Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-21 16:27:18 +02:00
Code Issues Packages Projects Releases Wiki Activity

Automatically import and update project repository skills (#16066)

Browse Source 
Automatically import skills from new GitHub project repositories and update enabled sources in the background. Preserve project-derived access, saved selections, immutable execution versions and explicit pins; add legacy adoption, refresh and connection repair in Sources.

Include durable scheduling and bounded work credits, production Storybook journeys, real GitHub-to-routine acceptance receipts and final-head CI/security/Greptile qualification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-13 16:11:43 -05:00
1 parent de335b42d4
commit 473c80d13f
101 files changed
+5991 -186
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

doc/DATABASE.md
+44
View File
@@ -756,6 +756,37 @@ register before writing, or they can remain unseen until the next startup or
another notification. Tests should verify both the producer and its outer
transaction boundary.
### Project repository skill synchronization
`project_skill_sources` stores durable workspace import jobs and originating project
access provenance. Active bindings track their repository/ref; inactive bindings retain
access to installed versions. Project deletion removes bindings, but retained skills
with `access_source_id` stay restricted when no originating project remains.
Removing a project source hides tracking using `company_skill_sources.removed_at`
while retaining its access identity for editable copies and forks. Bindings become
inactive and refresh leases are invalidated transactionally. Reattachment preserves
removal; explicit adoption creates or joins an active source with a separate identity.
Migration 0341 limits repository/ref uniqueness to active sources. Removed sources
retain their original project audience; editable forks capture independent retained
provenance so later attachments cannot expose their local edits. Membership and
project visibility are still evaluated dynamically.
`company_skill_sources` stores automatic-update and future-import preferences, setup
user/agent and connection/grant IDs, next-check time, failure count and existing refresh
leases. Existing sources default to automatic updates with their selection preserved.
Credential values remain in the connection secret system. No project membership lists
are copied into skills. Background publication uses normal immutable versions and
transactional activity attribution. Setup agent and connection identifiers remain after
those identities are deleted. Refresh then requires explicit repair; deletion cannot
silently substitute board authority or another connection. Execution snapshots keep
only the delivered skill keys and their immutable version IDs. New executions resolve
the latest installed versions without an upstream fetch.
Project-derived sources retain `HEAD` for an unspecified repository ref. It follows
GitHub's current default branch. An explicitly configured branch has a separate
source identity even if it currently resolves to that default branch. API-key
agents without an authenticated run may attach repositories; their durable import
jobs have no setup person and require adoption by an authorized person.
### Task workspace records (2026-10-09)
@@ -800,3 +831,16 @@ source captures leave no new records. Worktree realization uses a verified check
as its Git command directory, including when metadata sits inside another repository.
No migration scans or copies existing agent home directories. Keep admitted native descriptors and recovery references until compatible controllers finish their persistence obligations. See [the architecture rollout contract](plans/2026-10-09-task-workspaces.md).
Migration 0341 stores the last scanned Git tree and a durable automatic-refresh
work allowance. The allowance starts at 512 MiB, refills at 512 MiB/hour, and
reserves the existing 128 MiB upper bound before automatic downloads. Completed
small scans refund unused capacity; interrupted/failed work keeps its bounded
reservation. These internal fields do not appear in source responses.
Automatic initial repository imports reserve durable company work credits before
provider reads: 512 MiB capacity, refilling 512 MiB per hour, with a 128 MiB
reservation per attempt. Failed downloads and package fallback keep the full
reservation. Successfully linked small imports refund unused measured work.
Completed bindings do not reset company credits; waiting jobs retry on refill.
The company_skill_import_budgets table stores no credentials or source contents.
doc/DEVELOPING.md
+13
View File
@@ -2320,3 +2320,16 @@ companion broker profile and Slack approval. Do not add manager client credentia
to a local `.env` or an open-source deployment. See
[Slack setup and Cloud qualification](connections/slack-setup.md) for the gates,
recovery behavior, and production-component stories.
### Project repository skills
New GitHub project workspace attachments enqueue imports transactionally. A bounded
server worker drains imports and checks enabled sources every five minutes. Shutdown
cancels owned scans and releases their leases. Sources can be refreshed manually or
paused through Skills → Sources. Legacy repositories appear there without being
imported until the user chooses Import skills.
Storybook: **Skills → Project repository skills**. Its production UI journeys use
in-memory provider and execution fixtures. Live GitHub refresh and execution need
separate acceptance evidence. Run the focused browser suite against built Storybook,
or set `STORYBOOK_TEST_BASE_URL` to an owned Storybook dev server.
doc/PRODUCT.md
+27 -10
View File
@@ -100,16 +100,33 @@ The governing rule is: **skill permissions are opt-in restrictions, not opt-in c
Core Paperclip owns the skill runtime, company-boundary enforcement, policy evaluation contract, API denials, validation, path containment, secret redaction, and activity logging. Those safety invariants cannot be disabled by policy. Open-by-default skill work never authorizes arbitrary host-path reads, unsafe executable content, or policy edits: local imports and scans must stay within Paperclip-known workspace or managed-skill roots, remote sources must resolve to validated immutable content, and platform safety denials must stay distinct from optional administrative restrictions. Paperclip EE may provide detailed administration for per-agent, per-role, per-action, per-source, and protected-skill rules, but EE is not required to use skills and is not an enforcement boundary. Without EE, companies remain open by default and any already-configured restrictions continue to be enforced by core.
GitHub repositories are first-class sources within Skills. People import selected
skills through their existing GitHub connection and manually refresh installed
snapshots. The source records repository, branch, path, and installed commit;
the connection supplies caller-authorized access. Originals remain viewable and
testable, with independent editable copies. New upstream skills require selection,
and deselection or upstream removal retains installed content. Disconnecting removes
the source after an explicit choice: keep independent editable copies in My Skills,
or delete its imported skills. Copies preserve agent assignments; skills still
used by agents must be detached before deletion. Already-disconnected sources
can be removed through the same flow.
GitHub repositories attached to new projects automatically become skill sources.
Paperclip imports their skills and checks the configured branch every five minutes,
using the setup person's authorized GitHub connection. New skills import automatically
unless excluded. Existing projects can adopt their repositories from Skills → Sources;
existing manually imported sources preserve their selections and gain automatic updates.
Refresh remains available, and operators can pause automatic updates.
If a tracked repository grows beyond the supported package size, its installed
repository skills and shared dependencies remain available until a safe refresh
can succeed. Initial oversized imports can use self-contained skill packages;
packages with omitted dependencies are invalid and are not installed.
Project-derived skills inherit current project access. Access through any originating
project is sufficient, including the responsible-user restrictions on agent execution.
Existing explicit company imports retain their visibility. Editable copies inherit
project restrictions; private content cannot be published or exported implicitly.
Detachment retains installed content and access provenance. Deleting its last project
never makes a skill company-visible.
Task and routine skill mentions remain stable references to the latest installed version.
Each execution records the immutable versions it uses; resuming keeps those versions,
while a new execution resolves latest again. Upstream failures and removals preserve
installed snapshots. GitHub refresh does not delay task execution.
Disconnecting removes a repository from Sources after an explicit choice: keep
independent editable copies in My Skills, or delete its imported skills. Kept
copies and derived forks retain their project access restrictions. Copies preserve
agent assignments; skills used by agents must be detached before deletion.
An explicit restricted policy may deny selected operations or switch to a default-deny preset with explicit allow rules. Core exposes a stable versioned policy API so EE and other administrative clients configure and simulate the same evaluator used by skill mutation routes. Core Skill Studio only needs to perform normal skill work, explain an explicit denial, and point administrators to EE when its richer policy UI is available; it must not recreate a partial enterprise permission editor.
doc/SPEC-implementation.md
+78 -6
View File
@@ -2016,9 +2016,11 @@ unavailable. Preserve current ownership and newer-work fences. See
reduced-motion-aware animations and shows indeterminate progress while saving
complete packages, without inventing completion percentages.
- Sources and entries are company-scoped. Reuse GitHub connection grants and credential
refresh; every provider read authorizes the current caller. Saved connection IDs do
not confer access to another user's token. Agent reads use managed run identities.
- Sources and entries are company-scoped. Discovery and connection changes authorize
the current caller. Automatic and manual source refresh use the stored setup person's
connection and grant, revalidating their access and refreshing credentials on each
check. Saved connection IDs do not independently grant library visibility. Agent
setup derives the responsible person from the authenticated run identity.
- The import picker searches the deduplicated union of repositories from all connections
the caller can access, automatically choosing an authorized connection. Adding accounts
uses the standard GitHub setup in Apps and retains the import draft. Repository URLs
@@ -2079,7 +2081,7 @@ unavailable. Preserve current ownership and newer-work fences. See
and history. New key collisions cannot overwrite unrelated skills. Only changed package
bytes or executable modes create versions. Library reads, tests, and subsequent unpinned
runs use local installed snapshots; pinned/active runs retain their versions.
- Additions require reviewed selection, with new candidates checked by default and
- Manually selected sources retain reviewed selection, with new candidates checked by default and
explicit folder exclusions retained. Upstream deletion keeps installed content with
removed status; moves are removal plus discovery. Deselect retains content
and assignments. Originals are read-only, with the existing independent fork workflow.
@@ -2096,8 +2098,38 @@ unavailable. Preserve current ownership and newer-work fences. See
resolve missing repository IDs and full snapshots on successful refresh. Exclude bundled,
catalog, local/project, skills.sh, and unsupported hosts. Existing GitHub import and update
endpoints delegate to sources while retaining response shapes.
- GitHub.com, manual refresh only. No upstream editing, polling, webhook sync, commits,
or pull-request creation in this milestone.
- GitHub.com project repository attachments enqueue durable skill-source import jobs in
the project mutation transaction. Skill import completes asynchronously and cannot
fail project creation. Configured refs use the existing project workspace precedence;
absent refs track the default branch. Sources without skills remain tracked.
- Enabled sources check upstream every five minutes; unchanged commits avoid downloads.
Manual Refresh uses the same serialized publication pipeline. Automatic updates may
be paused independently of selection. Project sources select future additions unless
excluded; existing manually imported sources retain their selections. Legacy projects
adopt through an explicit company-scoped, project-authorized endpoint, never a GET.
- Repository size fallback must not discard installed shared dependencies or
publish packages with detected omitted dependencies. Retain affected versions
and report the size problem. Self-contained packages can still update through
fallback. Project sources retry complete repository packaging and return to it
when supported; selections and exclusions remain unchanged.
- Unattended refresh stores setup identity and connection/grant identifiers, never token
values. Current membership, connection authorization and skill policy are revalidated.
Unrecoverable legacy setup identities require explicit connection repair. Retries are
bounded and backed off; leases survive restarts and shutdown cancels owned downloads.
- Project-derived skill visibility is the union of current originating project audiences.
Core uses project authorization, including agent/responsible-user intersection, for
discovery, reads, mutations, files, versions, tests and runtime delivery. Unauthorized
IDs return not found. Company imports retain their existing visibility. Forks preserve
project provenance. Unsupported public publication or portable export is rejected.
- Detachment retains installed content and project access provenance while stopping that
project's tracking. Ref replacement never rewrites existing references. An orphaned
project source remains restricted after its last originating project is deleted.
- Executions resolve unpinned references from local immutable versions, persist the
resolved inventory, and reuse it on resume after rechecking access. Explicit unavailable
references fail actionably rather than silently omitting instructions. Background
GitHub checks never delay execution. Existing version pins remain supported.
- Upstream editing, push webhooks, commit/PR creation and advanced reference pin UI
remain outside this release.
## Public assistant connection (opt-in)
@@ -2188,6 +2220,17 @@ See [the layer architecture](plans/2026-10-09-task-workspaces.md) for resolution
Companies may configure an optional shared API connection and model for short acknowledgements of accepted human turns. Fast responses use direct SDK inference alongside normal agent work, with a three-second admission deadline, bounded prompts/output, current authorization and independent `fast_response` accounting. They are server-marked platform receipts attributed to the agent, never completion, interaction or recovery evidence. Company-sponsored unlinked external senders are explicit; linked-user permission denials are never bypassed. See [fast-responses.md](fast-responses.md) for provider, delivery, history and endpoint contracts.
Project-derived source removal retains dynamic project access for kept editable
copies and forks. A removed source is absent from Sources and cannot refresh;
ordinary repository attachment does not restart it. Explicit project adoption can
restart tracking without widening the retained content’s access.
Codex execution homes are scoped to the agent’s authorized skill inventory,
including its immutable source versions and an empty inventory. Revocation and
version changes select a separate home and session store while retaining the
selected model account’s credential source. Repository skill auto-update trusts
its upstream authors; operators can pause updates to review changes manually.
### Task collection creation visibility (2026-10-13)
Tasks and project collections default to hiding agent/system-created tasks and
@@ -2207,3 +2250,32 @@ read-only activity do not admit scheduled work. Inbox discovery/Blocked preserve
participated automation and hide unrelated automation by default. Agent profile
history, routine history, and task-detail subtasks remain complete. API consumers
that omit the optional creation filter retain their existing list behavior.
Project-derived editable forks retain the project audience present when created.
Later repository attachments do not grant access to those edits. Re-adopting a
removed source uses a separate active tracking identity; retained copies keep
their original provenance. Project membership and visibility remain dynamic.
Automatic skill checks compare tree identity as well as commit identity, so
empty commits do not download or rescan content. Automatic scan work uses a
durable per-source 512 MiB allowance with 512 MiB/hour refill; installed versions
remain usable when it is temporarily limited, and authorized manual Refresh
remains available. Initial imports authorize skills.import before provider work.
At most 20 jobs per project and 100 per company enter the active initial queue;
additional repository bindings remain durable and are admitted automatically
when capacity frees. Project workspace create/update/delete use the existing
workspace-write authorization evaluator. Policy-denied initial jobs await explicit
adoption/repair rather than repeatedly scanning.
Agent-created project sources may reuse only the GitHub connection and grant
already authorized in the persisted active run identity. The responsible person
is recorded for attribution; that identity alone does not permit searching other
personal connections. Without a selected run grant, Sources requires an explicit
human connection repair before background import or update.
Automatic initial repository imports reserve durable company work credits before
provider reads: 512 MiB capacity, refilling 512 MiB per hour, with a 128 MiB
reservation per attempt. Failed downloads and package fallback keep the full
reservation. Successfully linked small imports refund unused measured work.
Completed bindings do not reset company credits; waiting jobs retry on refill.
The company_skill_import_budgets table stores no credentials or source contents.
doc/plans/2026-10-12-project-repository-skills.md
+201
View File
@@ -0,0 +1,201 @@
# Project repository skills
## Finish line
Deliver automatic imports and five-minute refresh, project-derived visibility,
latest-version execution with immutable execution snapshots, legacy adoption,
production UI and Storybook coverage. Completion requires passing repository
checks, a live GitHub-to-routine update journey and a reviewable PR with evidence.
## Decisions
- Track the configured project branch, defaulting to GitHub's default branch.
- Import future skills unless excluded. Existing manually imported sources keep selections.
- Refresh using the setup person's existing authorized GitHub connection.
- Access through any originating project is sufficient; evaluate current project ACLs.
- Retain installed versions and project provenance after detach or upstream removal.
- New executions use latest installed versions; resumed executions keep their snapshot.
- Legacy projects adopt explicitly. Existing imported sources update automatically.
- Advanced reference pin UI and GitHub push webhooks are deferred.
## Current qualification — 2026-10-13 UTC
Implementation and live acceptance passed, including upstream updates, private
access/recovery matrices, eleven rebuilt browser journeys, responsive inspection
and a final real v2 execution after master reconciliation. All standard local test
phases completed across documented retries. The final safe-fallback correction
passed 89 pinned Node 24 cases and server typecheck/build. Project fallback retains
shared dependencies, permits safe package updates and restores complete packaging
when the repository fits. Final review/check results and authorized merge state
are recorded on PR #16066; only its exact green head may be merged.
GitHub CLI authentication was restored by the user. Local and browser-published
histories were reconciled with an unchanged, identical tree. Final-head checks
must pass before merging. Merge is already authorized; no further merge
permission is needed.
The temporary reconciliation checkout and obsolete previews have been stopped
and removed. Final local acceptance data and evidence are retained. See the
[acceptance report](2026-10-12-project-skills-acceptance.md) for current outcomes
and limits. The process notes below are historical and do not supersede this
checkpoint.
## Historical progress
- Repository implementation and verification paths inspected; worktree started clean.
- Persistence, attachment jobs, background checks, manual refresh, project access, runtime snapshots and Sources UI implemented; integration review remains.
- Initial preview migration was `0336_lovely_bloodstorm.sql`; master reconciliation replaces preview migrations with replay-safe generated `0341_abnormal_bucky.sql`.
- Focused source persistence tests passed (13 existing cases); new project source tests passed (3 lifecycle/access cases).
- Source API/UI regression tests passed: 69 tests. Runtime-skill regression suite also passed.
- UI/server direct TypeScript checks passed after building workspace dependencies; repeat after latest changes.
- Token gates passed. Storybook build passed.
- Storybook Playwright journeys passed: legacy adoption/navigation, automatic additions, failure/retry, preference keyboard behavior and mobile overflow (5 tests).
- Manual Sources review completed in IAB; fixed cramped intermediate-width action layout. Project creation and routine Storybook walkthroughs remain.
- GitHub account `cryppadotta` authenticated. Claude and Codex CLI sessions authenticated; no model API key in environment. Live test must use an authorized existing CLI session or connection.
- Live GitHub acceptance, expanded permission/worker/runtime coverage, required full checks and PR handoff have not run.
## Current process/evidence pointers
- Branch: `codex/project-repository-skills`; checkout `/Users/dotta/.codex/worktrees/4631/paperclip`.
- Owned Storybook: `http://127.0.0.1:6187`, exec session 14926. Port 6007 belongs to another process and was not changed.
- IAB tab 1 shows the full Sources journey; browser binding 4. Preserve/reuse the binding when continuing.
- Runner dependency build completed; exec session 59507 was its build handle.
- Logs in `/private/tmp/project-skills-*.log`; browser screenshots under `tests/storybook-visual/test-results/` (ignored).
- Full feature remains the finish line. Next: expand authorization/runtime tests, inspect project/routine journeys, launch isolated live instance and complete GitHub update execution acceptance.
## Verification
Source/worker lifecycle and failure tests; authenticated human/agent access matrix;
runtime latest/resume tests; Storybook browser journeys and responsive review;
live disposable GitHub repository and supported runner; token/map gates, Storybook
build, recursive typecheck, Vitest and production build. Record actual results here.
## Live acceptance progress (2026-10-12)
- Created the project and attached the disposable private GitHub repository through
the production browser UI. The worker imported `acceptance-marker` without an
import action. No agent assignment was added automatically.
- Created a routine through the UI with a stable skill mention. The real
`codex_local` runner (`gpt-6.1-sol`, existing ChatGPT CLI login) completed it and
wrote the exact first marker from the installed skill reference file.
- Pushed an updated marker and a new skill. The production worker imported both
at 23:21:03 UTC, without Refresh, about five minutes after initial import.
- Ran the unchanged routine through the UI again. The runner wrote the exact
updated marker. The first successful run retained version
`1ccdfa45-4a82-44ad-96cb-9ffa6e20fc64`; the second selected
`2bd9801c-1b96-417b-b081-2867e65afdcf`.
- A real agent bearer key could not list/read the private skill before project
membership (404), and could after membership (200). Removing membership revoked
access. Granting membership through a second private project restored access;
its source response exposed only that accessible project's name. Both projects
reused one source and the original skill identity.
- Restarted the owned instance with its retained data and latest source changes.
Doctor passed for PID 40476 at `http://127.0.0.1:3110`; source links, installed
skill identities and selections persisted.
- Manual refresh succeeded. Disabling the disposable GitHub connection produced
a clear source error while retaining both installed skills. The connection was
restored; the browser connection-repair journey is in progress.
- Initial provider setup attempted an unsupported model (`gpt-5.4`); corrected
the disposable agent to the supported model before claiming live execution.
Agent provisioning was an API fixture. Project/routine creation and execution
used the real UI. No synthetic model response was used.
Evidence directory: `/private/tmp/paperclip-project-skills-proof-BGunNG`.
Non-secret records include `fixture.json`, `execution-v1.json`,
`execution-v2.json`, marker files, `automatic-update.json`,
`private-agent-access.json`, `private-union-access.json`, `connection-failure.json`
and `execution-v2.jpg`. The separate test key file is credential material and must
not be published or attached.
## Latest verification and remaining work
- Live legacy adoption reused the existing source and skill IDs. Listing the older
repository did not import it. The real Sources action performed adoption.
- Seeded disposable human identities and real bearer authentication passed direct
skill/file/version/source checks. Access through either project worked. Removing
membership revoked access. Agent responsible-user mismatch failed closed.
This is API identity proof, not a multi-person browser sign-in test.
- Linking the repository through an open legacy project made its skills visible.
Changing that project to private revoked access for the unrelated company member.
- Real UI connection repair persisted after reload and kept installed selections.
- Current instance restarted at http://127.0.0.1:3109 (owned PID 73371). Startup
applied migration 0337. Doctor passed. The unchanged routine produced the exact
v2 marker again in successful run f003baa3-37d7-4203-bd4d-a0c4b45171ca.
Its snapshot includes four selected immutable versions and excludes the new,
unassigned project skill. Older execution version evidence remains unchanged.
- Recursive typecheck, production build, Storybook build, maps and token gates
passed. Six browser journeys passed against built Storybook (no retries).
- Latest focused server/UI check passed 94 tests. Worker/startup check passed 44.
The isolated recovery test passed. A GitHub review fixture was corrected to admit
a new event before checking changed configuration; that focused test passed.
- Full suite is running in exec session 45645, log
/private/tmp/project-skills-final-full-suite.log. Observe this exact handle.
- Manual mobile review found cramped provenance names. Source headings now use
the full narrow-screen width, with actions below. Token/UI checks are rerunning.
Desktop, intermediate and mobile real-app light captures and mobile dark capture
are in the evidence directory. Finish both-theme Storybook and recovery-dialog
review before final handoff.
- Still required: final full-suite result, remaining requirements audit, final UI
build/browser verification, sanitized inspectable acceptance report, commits,
rebase and template-compliant PR/review handoff.
- Owned current server session 94907; development Storybook 14926; built static
Storybook 34961 (6188). Preserve ownership and use the stop helper for cleanup.
## Rebased checkpoint — 2026-10-12
- Implementation is in four logical commits on `codex/project-repository-skills`,
rebased onto master `4b958ecbd8`. Current committed head is `afc5f22a0c`.
- Master added the downstream Storybook catalog. The new suites now live in the
Skills feature home, with explicit stable IDs preserving every existing URL.
Shared simulation lives in `ui/storybook/support/projectRepositorySkills.ts`.
Catalog structural checks passed (225 files / 2,235 stories) and nine catalog
tests passed. Existing source dependency records preserve their classifications.
- Post-rebase recursive typecheck, production build, Storybook build, map/token
gates and all six built-browser journeys passed. No Linux visual baseline was
compared or updated.
- The earlier full-suite process was stopped before rebase. The authoritative
post-rebase full suite is exec session 12928, log
`/private/tmp/project-skills-rebased-full-suite.log`. It is live; do not restart
it because output is quiet.
- The owned instance restarted at `http://127.0.0.1:3110`, PID 71855, server
session 62515, and Doctor proved committed head `afc5f22a0c` with ready UI/API.
Its retained source, skill identities, routine and execution history persisted.
- Manual mobile connection repair passed via keyboard. A missing focus return
was observed after saving; the UI now returns to the source actions button.
Its eleven focused UI tests and UI typecheck passed. The updated Storybook
build is session 73545 and must be followed by the browser focus assertion.
- Keep the full objective active. Still required: final suite result, complete
requirements audit, remaining theme/state review, acceptance evidence handoff,
template-compliant PR and review/check loops. No PR has been opened yet.
## Review fixes and evidence checkpoint
- PR #16066 is open in draft and attached to the Codex task. Initial CI passed;
Greptile requested fixes for API-key repository creation, default-branch tracking,
local file version freshness, and missing screenshots.
- API-key agents can attach repositories without an authenticated run. Their jobs
retain no invented setup person. Default-branch bindings preserve HEAD, separate
from fixed branches. Fresh unpinned local executions capture changed installed
files; resumed/pinned executions keep their immutable selections.
- A local-import path was proved to discard private snapshot provenance. It now
rejects known private snapshots/materialized copies as company-wide imports,
with an opaque not-found response for unauthorized readers.
- All 27 focused source/runtime tests passed after these fixes. Six browser cases
passed with real emulated touch taps, keyboard controls and repair focus return.
- The first post-rebase full run finished: 17,961 passed, six failed. Two failures
used pre-review module implementations alongside added tests. Four other cases
timed out or lost an HTTP/database connection and are being rerun in isolation.
Final qualification must use a fresh full run on the frozen reviewed code.
- The approved initial Storybook preview published through the existing CODEOWNER
workflow. Its immutable build is 38329548539-1 for commit 3e82dada92. Publish the
reviewed head before handoff. Acceptance screenshots and minimized execution
receipts are now in ui/storybook/public/project-skills-evidence/ for the existing
Storybook artifact workflow. No credential or identity key files are included.
## Reconciliation checkpoint — 2026-10-12 evening
Master integration preserves the durable task-workspace changes alongside project skill authorization and execution snapshots. The regenerated migration follows master's 0338 and its real database replay regression passed. Recursive typecheck and production build passed in the temporary reconciliation worktree. Ten focused Storybook journeys passed, including project creation, second repository attachment and immutable simulated routine revisions. Final built-artifact checks, full-suite qualification, updated PR review and a current implementation acceptance run remain.
The reviewed-head full run ended with 17,687 tests passing, two failing, 383 skipped and 18 failed files. Database startup errors identify host shared-memory exhaustion; public MCP also timed out. This is a failed run, not completion evidence. The owned acceptance server was stopped after recording successful reviewed-head run e2cbfc2b-aa07-4fa0-b240-82033d6d1035, and its data/evidence remain available.
The user authorized merging once final PR checks pass. Do not merge an incomplete or conflicting head.
doc/plans/2026-10-12-project-skills-acceptance.md
+517
View File
@@ -0,0 +1,517 @@
# Project repository skills: acceptance evidence
Live GitHub updates, immutable routine executions, access/recovery checks,
production-component browser journeys, and manual responsive review passed.
Local standard test phases were completed across documented environment retries.
Final review/check results and merge state are recorded on PR #16066; merge is
gated on its exact head. This report separates real outcomes from test fixtures.
## Environment and artifacts
Implementation checkout: `~/.codex/worktrees/4631/paperclip`, branch
`codex/project-repository-skills`, PR [#16066](https://github.com/paperclipai/paperclip/pull/16066).
The isolated instance uses embedded PostgreSQL and a retained disposable data
directory outside the checkout. Doctor verified the served source revision and
loopback port ownership before the live journeys. No production company data was
used. The test GitHub repository is private and contains only generated skill
fixtures. The supported live adapter is `codex_local`, using model `gpt-6.1-sol`
through the existing authenticated Codex subscription.
The tracked [evidence bundle](../../ui/storybook/public/project-skills-evidence/)
contains the original four captures and a final post-reconciliation capture and an explicit, minimized
[version receipt](../../ui/storybook/public/project-skills-evidence/versions.json).
It is included in the existing Storybook build artifact and branch publication.
Credential files, full provider logs, human authentication keys, and connection
secrets are excluded. This direct Codex task has no driving Paperclip issue or
injected artifact-upload context; the PR and Storybook artifact provide the
review access path.
## Live GitHub → import → execution
1. Provisioned an empty company, one authorized GitHub connection and a supported
test agent through normal APIs. This is fixture setup, not onboarding proof.
2. Created a project and attached its repository through the real board UI.
The background worker imported `acceptance-marker` without a second skills
setup step. Import did not assign the skill to every agent.
3. Created a routine through the real UI and selected the stable slash mention.
The runner read the installed skill and reference, then wrote the exact
`project-skill-v1` marker in the task workspace.
4. Pushed a changed skill and a new skill to the configured branch. The production
worker imported both at **23:21:03 UTC**, without Refresh, about five minutes
after initial import.
5. Ran the unchanged routine through the UI again. The runner wrote the exact
`project-skill-v2` marker. The earlier execution still identified its original
immutable version.
6. Restarted the owned instance with retained data. A further real execution
succeeded with v2. Its snapshot contained only four delivered immutable
versions; the newly imported, unassigned skill was absent.
| Execution | Immutable project skill version | Exact output |
| --- | --- | --- |
| `2db73573-eeaf-420e-aad8-898a87377342` | `1ccdfa45-4a82-44ad-96cb-9ffa6e20fc64` | `project-skill-v1` |
| `ae685356-02a9-4369-9bb3-b76bb112bcce` | `2bd9801c-1b96-417b-b081-2867e65afdcf` | `project-skill-v2` |
| `f003baa3-37d7-4203-bd4d-a0c4b45171ca` | `2bd9801c-1b96-417b-b081-2867e65afdcf` | `project-skill-v2` |
| `e2cbfc2b-aa07-4fa0-b240-82033d6d1035` | `2bd9801c-1b96-417b-b081-2867e65afdcf` | `project-skill-v2` |
| `b91818ac-9e9a-4a98-bd6b-63c0a58e4a1a` | `2bd9801c-1b96-417b-b081-2867e65afdcf` | `project-skill-v2` |
| `83f6f277-5a4a-4814-ba55-8a7ed4a517e1` | `2bd9801c-1b96-417b-b081-2867e65afdcf` | `project-skill-v2` |
The earlier reviewed real execution passed on implementation
`614a2d1d72892128c7fc949d2485308756e7c687`. It delivered four selected immutable
versions, excluded the unassigned addition, and preserved the first run's v1
snapshot. The uploaded output was exactly 17 bytes, with the v2 SHA-256 hash in
the receipt. The model was `gpt-6.1-sol`; its successful heartbeat execution is
`b91818ac-9e9a-4a98-bd6b-63c0a58e4a1a`.
After master reconciliation, another real execution passed on
`cea9e474ba462ed9385ce2f8acb58c245585b310`. The unchanged routine completed
through the real API and supported runner; the completed task was inspected in
the browser. It wrote a fresh 17-byte workspace file matching v2 exactly, delivered
the same four immutable versions, and left the original execution on v1. Startup
checked the unchanged upstream source successfully at 11:45:25 UTC. This output
is a verified workspace file; the earlier reviewed execution also proved upload.
The new receipt and capture distinguish those two evidence paths.
Initial upstream commit: `d7e95f01903678531981ce684e721191ecb0a7da`.
Updated upstream commit: `f4b35ea0082150bda320cda7b3c0777d263bf430`.
The marker files and their SHA-256 hashes are in the receipt. These runs used
real model responses. An initial unsupported model selection failed; the agent
was corrected to the supported model before recording successful execution.
## Recovery, adoption, and access
- **Manual refresh:** the real Sources action completed without replacing skill
identities or selections.
- **Connection failure:** disabling the disposable connection produced an
actionable source error while retaining both installed skills. Re-enabling it
and saving an authorized connection through the real repair dialog recovered
updates. Reload preserved the result.
- **Legacy adoption:** an existing workspace was seeded without a binding to
represent pre-feature data. Listing it did not import anything. The real
**Import skills** action reused the existing repository source and skill IDs.
- **Private agent access:** a real bearer key could not list/read the private
skill before project membership. Membership granted access; removal revoked
it. Membership in another qualifying private project restored access.
- **Human access:** separate disposable users and real hashed bearer credentials
passed skill, file, version, and source reads only through their qualifying
projects. Responses omitted inaccessible project names. An unrelated company
member received not-found responses. Revocation took effect on subsequent reads.
- **Responsible person:** an agent with project access could not read on behalf
of an unrelated person. The same agent could read on behalf of an authorized
project member.
- **Open/private transition:** linking through an open project made the shared
source visible. Changing that project to private revoked the unrelated
company member's access.
The multi-person evidence uses seeded identities followed by production bearer
middleware and APIs. It does not claim separate browser sign-in journeys. The
private GitHub repository requires the authorized owner's access for inspection.
## Automated proof
The final focused project-source suite passed **12 cases**. The source and runtime
suites passed **16 cases**. The replay-safe migration regression passed with a real
embedded database, retaining installed content, private provenance, scheduling,
and setup identities when renumbered SQL runs again.
The service and authorization suites cover transactional enqueue/rollback,
legacy edits without adoption, empty repositories, unchanged commits, additions,
exclusions, invalid siblings, source deduplication, ref changes, detach/delete,
lease competition/expiry, restart recovery, graceful shutdown, backoff/rate limits,
revoked setup access, and explicit connection repair. They also cover private
reads, forks and publication, responsible-person restrictions, immutable resume,
explicit pins, inaccessible references, and fresh local file snapshots. Known
project-derived local snapshots/materialized copies cannot become company-wide
imports. Default-branch HEAD and explicit branches retain distinct identities.
OAuth expiry and larger failure matrices use deterministic provider boundaries;
no live OAuth-expiry claim is made.
**Eleven browser journeys passed against the built Storybook artifact** on the
final implementation. They cover automatic import after project creation and
second repository attachment, empty-source guidance, legacy adoption/navigation,
background additions, refresh failure/retry, keyboard preferences, mobile touch,
connection repair/focus, repair after a concurrent update, and latest routine
execution with immutable earlier versions. Stories mount shipped pages and
components with simulated provider/import/execution data; they do not replace
the live acceptance above.
Recursive typecheck, production build, Storybook build, token gates, map checks,
and catalog checks passed after master reconciliation. The final fallback follow-up
also passed server typecheck/build and **89 pinned Node 24 cases**: scanner/source,
project worker, and both workspace authorization suites. Eleven built-browser
journeys passed again after master reconciliation. The signoff retry passed ten
browser/API cases across two runs.
The standard `pnpm test:run` phases on source d8ab984 were completed across retries:
| Phase | Passed files | Passed cases |
| --- | ---: | ---: |
| General server | 900 | 18,429 |
| UI | 729 | 8,181 |
| CLI | 66 | 527 |
| Shared | 89 | 908 |
| Skills catalog | 5 | 20 |
| Adapter utilities | 73 | 1,736 |
| Other workspaces | 130 | 2,079 |
| Database | 51 | 194 |
| Serialized server | 158 | 3,113 |
This is complete phase coverage, not a claim that one uninterrupted monolithic
command exited zero. The serialized run passed 156 files, then hit a 15-second
route-test timeout; the failed file and the last unrun file passed all 21 cases in
isolation. The final five-suite run passed all 89 cases on source
`40bafbb7a2f7753ac48765081bd2583f611cf5f2`, preserving assertions and deadlines.
Full final-head CI covers the reconciled master and subsequent corrections;
its authoritative results are linked from the PR.
GitHub CLI authentication was restored by the user. Earlier browser publication
and local source trees were identical and their histories were reconciled without
content changes. No host credentials or production data were mounted in the Linux
verifier. Review findings were fixed with regressions and resolved through the
[check-pr workflow](../../.agents/skills/check-pr/SKILL.md).
Earlier local full runs failed. macOS runs encountered database startup/hook
failures from shared-memory exhaustion and a public-MCP timeout. The first Linux
setup omitted generated SDK builds; the second lacked standard test utilities
and was stopped after that diagnosis. Those attempts are not passing evidence.
Two pre-existing login tests also assumed an x64 host. Their intended native
and legacy platforms are now explicit, and all 13 login cases passed on ARM64.
The subsequent 2 GB heap run exited unsuccessfully: 18,422 tests passed,
76 skipped, and one worker aborted from JavaScript heap exhaustion in the
existing setup-token route suite. Its 30 cases passed with an explicit 4 GB
heap. With the 4 GB heap, all 900 server files passed: 18,429 tests passed and
76 skipped. The UI group then reported 8179 passed and one failure. The reset
reproduced in isolation; its editor now holds accepted selections until parent
data refreshes. Both focused UI suites passed all 38 cases. Subsequent phase completion and final focused verification are recorded above.
The final Linux run uses Node 24, built workspace dependencies, pinned Rust,
and the standard process, shell, SSH and JSON test tools in an isolated container.
No host credentials or production data are mounted.
The ownership-checked acceptance cleanup passed: runtime, loopback port 3107,
and its embedded database stopped cleanly; disposable data and evidence remain
retained. All owned Storybook previews and the temporary reconciliation checkout
were removed after their captures and verification logs were preserved.
## Experience review and limitations
Manual real-app inspection covered mobile, intermediate and desktop in both
themes. Storybook inspection included long names, validation warnings, empty and
paused sources, retained removed skills, private project unions, and an 80-skill
list. The list had no horizontal overflow and its footer was keyboard reachable.
A cramped provenance layout, missing repair focus return, and blank simulated
routine section were found, fixed and covered by regression journeys.
The final concurrent-repair story also passed manual inspection at 390px in dark
mode. The selected account remained checked after a 409, its error and footer
fit on screen, keyboard retry succeeded, and focus returned to source actions.
The shared task-defaults editor was also inspected after saving on mobile in dark
mode: the acknowledged selection remained visible and Save became disabled.
Sampling does not claim interaction coverage for all **27 new exports**. No Linux
screenshot baseline was compared or updated.
The existing Storybook deployment workflow published commit 5869973 in run
38394544972, including the sixth real execution receipt and final task capture.
Subsequent fallback changes affect server code; shipped UI and story sources remain
identical to this published build:
[review guide](https://github.com/paperclipai/paperclip/blob/586997352198b2dbf025445706762e27d38e6764/ui/storybook/stories/skills/project-repository-skills/README.md),
[project creation](https://d1p6rlowie26tp.cloudfront.net/storybook/branches/codex-project-repository-skills-8ad0b8be51ae75bc/builds/38394544972-1/?path=/story/skills-project-repository-skills-02-journeys--create-project),
[repair retry](https://d1p6rlowie26tp.cloudfront.net/storybook/branches/codex-project-repository-skills-8ad0b8be51ae75bc/builds/38394544972-1/?path=/story/skills-project-repository-skills-02-journeys--repair-conflict).
Published deployment metadata and story IDs were verified over HTTP. The in-app
browser blocked CloudFront navigation, so hands-on inspection used the identical
local built artifact. No browser security warning was bypassed.
## Inspectable captures
Real isolated Sources page:
![Real Sources page](../../ui/storybook/public/project-skills-evidence/sources-desktop.jpg)
Shipped mobile repair dialog with Storybook fixtures:
![Mobile connection repair](../../ui/storybook/public/project-skills-evidence/connection-repair-mobile.jpg)
Real completed routine execution:
![Real routine execution](../../ui/storybook/public/project-skills-evidence/routine-execution-final.jpg)
## Review fixes
Initial-import and scheduled-refresh queries exclude live leases before bounding
batches. A real database regression places ten leased records ahead of eligible
work and verifies import and refresh still proceed. Connection repair derives
its source from the current query and reloads the revision after a conflict while
retaining the chosen account. Runtime-test teardown drains tracked execution
writes before truncation, resolving a reproduced database teardown deadlock.
![Mobile concurrent repair](../../ui/storybook/public/project-skills-evidence/repair-conflict-mobile.jpg)
The final review identified an oversized-repository refresh that could drop shared
dependencies. Repository sources retain installed versions when fallback would omit shared
files; safe self-contained packages can still update. Initial and subsequent project fallback scans reject detected omitted
dependencies. Project sources automatically restore complete packaging when it fits. All 56 scanner
and source-service cases passed, including manual and scheduled oversized refresh,
retained shared/sibling files, unchanged versions, and recovery after size reduction.
The signoff fixture retry also passed ten real browser/API cases across two runs.
Environment retries also preserved the original failures: a telemetry opt-out
variable conflicted with a mocked CLI test; Docker overlay storage affected probe
identity tests; a native temporary mount initially disallowed execution; and an
oversized stream fixture failed under that mount. The normal-storage cases and
all 56 probe cases on an executable native temporary filesystem passed. A later
host rerun used Node 26 and hit database/subprocess timeouts. The pinned Node 24
run passed the unchanged audit deadline, project suites and authorization cases.
No assertions or timeout thresholds were relaxed to obtain these results.
## Final source-removal integration
Integrated master’s keep/delete choice with project provenance. Removed project
sources retain an internal access identity, invalidate refresh and attachment leases,
and stop tracking. Kept copies and forks remain dynamically restricted. Ordinary
reattachment retains removal; explicit adoption restarts tracking. The rebuilt Storybook passes all 13 interactive project-skill browser cases, including
kept-copy access wording and cancelled removal; the Sources UI/API suites pass 21
cases. The initial Linux pass exposed two harness mismatches: source authorization
now precedes request validation, and repository skill instructions use a wrapper.
Both assertions retain their intended contract with corrected fixture setup/file
paths; the final service pass and current PR checks are recorded on PR #16066.
## Final review corrections
The security review’s per-revision approval recommendation conflicts with the
approved automatic/latest behavior. The response records repository-author trust,
per-refresh credential/access/policy revalidation and the manual-update option.
The scanner subsequently passed. Greptile then identified two concrete defects:
legacy adoption without a saved source ID did not restore removed tracking, and
Codex’s reused home could retain a revoked private skill link. The initial correction resolved the removed repository/ref; the final privacy
review superseded reuse with separate active tracking identity, described below. Codex launches against a
home scoped to its permitted immutable inventory, including an empty inventory,
with the selected credential source preserved. Three Codex launch/injection/sync
suites pass 32 cases; final service and real-provider results are recorded on the PR.
Post-review verification passed 212 cases across 13 pinned Node 24 suites,
plus the full typecheck and build. A fresh real ACP routine execution (`PRO-7`,
`95d14fcf-5643-4d5f-9512-d160efcb2446`) delivered four pinned versions and
reported the v2 marker. The CLI probe exposed the host’s old Codex 0.153.4
client: both requested models were rejected before model work. The repository
already pins Codex 0.160.0. With that client and the original `gpt-6.1-sol` model,
`PRO-9` succeeded in run `d94d7009-8787-4483-81e4-71ec0b512ed6`, marked its
task done, and independently verified the existing 17-byte v2 evidence file
without rewriting it. Its invocation used `skill-runtimes/<agent>/<inventory digest>`;
the selected account remained usable. The v1 execution still identified version
`1ccdfa45-4a82-44ad-96cb-9ffa6e20fc64`. Eight successful execution receipts
now distinguish newly written files from provider-reported or preserved outputs.
Failed probes are recorded separately. The disposable agent’s original settings
were restored and the superseded probe task was cancelled.
The live walkthrough also reproduced a withdrawn AI connection request still
showing an active “Fix connection” action after a successful task. A failing
regression test captured it. Withdrawn requests now use the existing terminal
card renderer; 70 interaction/Sources UI/API cases pass. A production-component
Storybook state and a narrow light-theme browser case preserve this recovery state.
The final compatibility review also found provider settings skipped for generated
Codex homes. A launch-level regression failed for both default and explicit
credential homes. Provider settings now apply to generated skill homes while
caller-owned homes remain untouched; the three Codex suites pass 33 cases.
Final qualification passed 213 service/runtime cases on pinned Node 24, 70
interaction/Sources UI/API cases, and 14 rebuilt Storybook browser journeys.
Typecheck, build, token gates, maps and catalog checks passed. The restarted
real app on source `c9a5897bb0843ee5ac89ee0bb9210ca864d6ceca` shows the
withdrawn request as inactive after reload, alongside the completed CLI result.
The updated CLI capture records this corrected recovery state. Final head checks
and merge status remain authoritative on the PR.
The final retained-content privacy correction separates tracking from provenance.
After removal, adoption creates or joins an active source; it never adds readers
to the removed source holding editable copies. Forks capture the originating
project set independently, so later open-project attachments cannot expose private
local edits. Membership, project visibility and deletion remain dynamic. Generated
migration 0341 makes source identity uniqueness apply only to active rows. The
regression matrix covers public re-adoption, direct private file/version denial,
private fork preservation, original-workspace adoption, and membership revocation.
Final qualification of the retained-audience correction (`4f7df5e683`) passed
232 tests across 14 pinned Node 24 suites. One scanner subprocess deadline failed
under concurrent build/install load; the unchanged full batch passed on retry.
The 37 source/privacy cases also passed on macOS. Final typecheck and production
build passed after installing two test dependencies added by concurrent master.
Repository policy leaves lockfile publication to its scheduled refresh bot; no
lockfile change remains in this PR. The rebuilt artifact passes all 14 interactive
browser cases. Maps, token gates and catalog tests pass; the catalog inventories
227 files and 2,252 stories without claiming all were rendered. Application and
preview processes owned by this task were stopped with their evidence retained.
Current-head GitHub checks and review results are recorded on the PR.
The final catalog review found stale immutable versions after installed catalog
bytes changed. A reference-only update regression failed before the fix. New
unpinned executions now compare installed catalog files, as they already did
for local files, without network fetching; unchanged content reuses its version
and pinned resumes preserve the old bytes. The runtime/catalog suites pass all
17 cases. This final correction changes no UI; the published Storybook build
38427815861 pins 6219538f75 and contains all eight successful execution receipts
and the two explicitly classified old-client failures.
A fresh review then identified a removal/import race and stale account selection
in generated Codex homes. Both were reproduced before correction. Final source
linking locks the source before the binding, follows removal's lock order, and
rechecks removal and the binding lease. A losing import retries with independent
tracking; it never adds readers to retained edits. Generated homes now include
the selected subscription account and credential source in their digest. Switching
accounts uses a new home; same-account token refresh keeps the home. The regression
uses a regular credential file to model remote copy-back. The source/Codex suites
pass 71 cases, and recursive typecheck and production build pass. These are
deterministic race and launch tests; they do not claim another live model run.
The final default-account follow-up reproduced a mismatch between host and
company credentials: hashing the host source while seeding the saved company
login could reuse a previous account's generated home. The digest now reads
the same prepared company or explicitly selected home used for seeding. The
default-company switch regression fails before correction and passes afterward.
All 33 Codex launch/injection/sync cases and recursive typecheck pass. The
source-removal correction passed fresh review; final CI/review results remain
recorded on the PR. No extra live-provider result is claimed for this correction.
The final security scan identified API-key imports persisting a responsible user
as unattended refresh authority. Direct imports now share project attachments'
setup resolver: API keys retain attribution without a setup person; authenticated
runs derive the person from the persisted active run. Legacy recovery cannot
restore deliberately absent API-key authority and accepts agent attribution only
with a matching recorded run. With valid membership fixtures, both API-key
regressions fail on the previous route setup and pass after correction. All 101
affected route/source/database cases pass, including scheduled repair and legacy
audit recovery. Recursive typecheck and production build also pass. Current-head
security and CI results are recorded on the PR.
Final master reconciliation includes `da9af6741585aeb439b5d38af398eab841131d6e`.
Master added migration 0339, so the unmerged feature migrations were regenerated
through `pnpm db:generate` as 0340 (project tracking), 0341 (retained source removal)
and 0342 (active-source identity indexes). Their replay-safe SQL preserves installed
content and provenance. Migration numbering/safety and five-snapshot retention
pass. All 151 affected source, authorization, catalog-version and Codex cases
pass across eight suites on the merged checkout, along with recursive typecheck,
production build, maps, catalog and token gates. Inherited budget/secret review
sources retain their prior classifications; no extra provider proof is inferred.
The final merged Storybook build succeeds and all 14 project-skills browser
journeys pass against it. This includes the inherited shell updates; no Linux
visual baseline comparison is claimed. Final publication and current-head PR
gates are recorded on the PR.
CI caught a stale replay-test filename after regeneration. The test now opens
0340_salty_vance_astro.sql; its preservation assertions remain unchanged. The
wider macOS Node 26 database run hit eight unrelated native PostgreSQL deadlines.
The qualified Node 24 rerun passed all 51 database files and all 195 tests, with
no skips or changed deadlines. Its input records e3b107976f plus the one-line
fixture correction and the corrected fixture hash. This supplements the 151
merged service/runtime cases and 14 rebuilt browser journeys. Final PR checks
remain the authoritative merge record.
The last concurrent catalog merge integrates master 252d9b5077. Its background
trace-context and runtime-skill suites pass all 12 cases; recursive typecheck
and build pass. Feature controls and fixtures are unchanged from the 14-case
built-browser qualification. The verification inventory now records 59 remaining
gaps and preserves variable-dialog and production obligations for the partially
covered routine-run journey. Catalog/map checks pass without treating source
classification or test presence as additional execution proof.
The final resource review adds generated migration 0343 and durable automatic
refresh credits: unchanged Git trees require no download, including empty commits;
512 MiB of scan/download allowance refills per hour, with conservative reservations
and refunds for small repositories. Manual Refresh remains available. Initial
imports admit 20 jobs per project and 100 per company; overflow remains durable
and is admitted as capacity frees. Repository attachment requires workspace-write
permission, and skill-import policy is checked before provider reads. Policy denials
stop automatic retries until an explicit repair/adoption. These changes passed
all 61 qualified Node 24 files and 397 tests without skips or deadline changes.
The final PUT-route and legacy-tree-learning follow-up passed all 48 affected
cases; recursive typecheck and production build pass afterward.
The final running checkout (32c5cdc4ee plus the recorded PUT-route/tree-learning
follow-up) also passed isolated runtime doctor. Through the real board UI at
127.0.0.1:3107, a new disposable project was created without repositories, then
the approved private acceptance repository was attached through Configuration.
The saved workspace persisted after reload. Sources showed both skills imported
automatically under the new default-branch source without Refresh or adoption.
The earlier projects track main explicitly, so their existing source and versions
remain independent. The picker warned that another connection could not load;
the selected Acceptance GitHub connection successfully listed and attached the
repository. This is an additional real attachment check, not another model run.
The screenshot is retained in the inspectable evidence gallery.
A later master merge introduced 0340_open_warlock. All unmerged feature schema
changes were regenerated together as 0341_abnormal_bucky through pnpm db:generate,
with the same preview replay safeguards and five newest snapshots. Agent setup
now pins only a GitHub connection/grant already authorized in the persisted active
run identity. Responsibility remains attributable but does not authorize searching
the person's other connections. Missing run selection and legacy agent sources
require an explicit human connection repair before unattended reads.
Final-head 684105f683 passed all 402 qualified Node 24 cases across 62 files
and all 14 rebuilt browser journeys; CI and security also passed. Greptile then
reported a deferred-admission/detach race and stale generated Codex settings.
Both regressions failed before correction. Admission now conditionally updates
only still-pending, capacity-waiting bindings with a workspace. Generated homes
include selected config.toml content in their identity, so edits and removals
reach the launched process without carrying old private inventory. Focused
verification and fresh final-head review are recorded on the PR.
Both final review fixes pass all 54 affected worker/Codex cases, recursive
typecheck and production build. Master c7f4b74111 changed workspace preparation
and its catalog notes; the catalog conflict preserves both sets of stable IDs
and classifications. Its affected workspace tests and final CI/review are
recorded on the PR. No new feature canvas or provider run is inferred from
this source reconciliation.
A subsequent queue review reproduced two database failures: adoption held the
binding while waiting for the company lock, and 100 deferred rows from a full
project hid another project's work. Both regressions fail on the previous head.
Adoption now captures run identity and takes the company queue lock before
source/binding locks, reusing that validated setup while enqueueing. Admission
excludes full projects before its bounded waiting-row selection. Verification
and current-head gates are recorded on the PR.
The final initial-import resource finding is corrected with durable company
credits, reserved before provider work. The failing-download regression observed
five reads before correction instead of the allowed four. New cases cover
worker recreation/refill, small-import refunds and completed large measured
jobs retaining their charge. These are simulated download sizes with real DB
transactions, not new oversized live downloads. Generated migration and final
qualification results are recorded on the PR.
The initial-budget commit passes all 251 qualified Node 24 database/source/setup
cases across 54 files. The final refill review reproduced a backwards timestamp
from interleaved older/newer sweeps. Both initial-import and source-refresh
reservations now clamp time to the persisted budget timestamp. The real database
regression fails before correction and verifies no repeated refill afterward.
Focused final verification and current-head CI/review are recorded on the PR.
Qualified Node 24 reproduced the earlier timestamp assertion failure, then passed
all 57 affected cases on the corrected source with no skips or deadline changes.
The macOS reproduction timed out in native DB setup; its broader attempt hit a
privacy-test deadline. Those failures are retained separately from the qualified
pass. Recursive typecheck and production build pass after the correction.
Final head 0fa7fd008b passed fresh security and Greptile 5/5; tests completed
without failures before another master merge introduced a catalog conflict.
Master 7eea9d8214 moved Storybook metadata into each story and made generated
indexes ignored. The two project-skill suites now carry their existing capability
associations, production/fixture classification and durable limits inline, keeping
all explicit meta IDs and exports. Both feature recipes and master's verification
sections are retained. The deleted aggregate catalog/review ledger stays deleted.
Map/generator validation passes; rebuilt browser and final-head gates are recorded
on the PR. This structural migration is separate from the earlier live receipts.
The final catalog-merge CI attempt lost eight jobs to the same runner shutdown;
the logs show cancellation rather than failed assertions. Greptile separately
reproduced adoption restoring a stale workspace selection. Adoption now shares
the editor's root-provenance/project/workspace lock order, re-reads the selected
repository, rechecks project access and rejects a changed selection before queue
mutations. Its real edit-interleaving regression fails before correction. Final
focused results and fresh CI/review are recorded on the PR.
The stale-adoption correction passes all 61 affected Node 24 cases across four
suites with no skipped tests or changed deadlines. The macOS broader attempt
timed out starting two native databases; it did not establish a code failure.
Recursive typecheck and production build pass. The final rebuilt Storybook
publication 38464595162 pins 60537b0695 and contains all eight live receipts;
the adoption-only correction leaves its controls and fixtures unchanged.
maps/agent-access-map/projects.md
+9
View File
@@ -111,3 +111,12 @@ remaining operations before qualifying the whole capability.
Gaps: Description/goal/target selection exist; host workspace command mutation is denied
to agents and budget policy is board-managed.
Repository attachment also writes a durable skill import binding in its transaction.
Agent-created bindings derive their refresh person from authenticated responsible-user
context. Project selection does not grant runtime credentials or override private
project skill access. Verify automatic import and denial with the actual agent identity.
Workspace creation, update and deletion require project:write_workspace through
the shared authorization evaluator in addition to project read access. Repository
attachment does not bypass skill-import policy for its background discovery.
maps/agent-access-map/routines.md
+6
View File
@@ -146,3 +146,9 @@ remaining operations before qualifying the whole capability.
Gaps: Pause/archive mutations have a generic native REST path under routine
ownership rules. Plugin-managed routine rules remain separate.
Execution resolves referenced project skills against agent/responsible-user access,
then saves immutable version selections in the heartbeat execution context. New runs
resolve latest installed content; resumed runs retain selections and recheck access.
An unavailable explicit reference produces an actionable configuration error.
Live agent execution across an upstream update remains required acceptance evidence.
maps/agent-access-map/skills.md
+34
View File
@@ -142,3 +142,37 @@ remaining operations before qualifying the whole capability.
Gaps: No dedicated native orchestration binding; generic REST calls retain
skills.test policy, task-assignment authorization, responsible-user intersection and
admitted skill-test JWT scope.
Project-source API additions: GET `/api/companies/{companyId}/skill-sources/project-repositories`,
POST `/api/companies/{companyId}/skill-sources/adopt-project` with `workspaceId`, and
PATCH `/api/companies/{companyId}/skill-sources/{sourceId}/settings` with expected revision
and update/connection preferences. Native API discovery uses the OpenAPI registration.
Project-derived read/use permission follows current project authorization, with access
through either originating project. File/version reads, source metadata, mutations,
editable copies, tests and runtime delivery retain that boundary. Unattended refresh
uses the revalidated setup identity and connection, not the requesting agent's token.
These additions need actual permitted/denied agent-run acceptance; service tests alone
are not live credential proof.
Direct GitHub source imports share the project setup-identity boundary. API-key
agents may import permitted public content, but their responsible-user field
does not authorize unattended credential use. They need human connection repair
for automatic refresh. Authenticated agent runs derive the setup person from
the persisted active run. Legacy agent activity without a matching run cannot
recover a refresh person. Route and database tests cover these distinctions;
this addition makes no new live-provider acceptance claim.
Initial project imports check skills.import before provider reads; denied jobs
do not retry automatically. Deferred jobs remain durable behind per-project and
company admission limits. Scheduled refresh compares immutable tree identity and
uses a persistent work allowance; manual refresh still requires its existing
policy/access checks. These limits grant no content or execution access.
Agent repository setup preserves the active run's selected GitHub connection/grant
IDs. Missing selections require human repair; the worker never searches the
responsible person's other grants for an agent-created source.
Initial automatic imports reserve company work credits before provider reads.
Failures retain the reservation; completed jobs cannot replenish the queue's
work allowance. Jobs remain durable and resume as company credits refill.
maps/feature-map/connection-setup.md
+2
View File
@@ -108,6 +108,8 @@ Test no-auth and authenticated endpoints separately; never infer one from the ot
Automated: [interaction card](../../ui/src/components/IssueThreadInteractionCard.test.tsx)
checks addressed-user controls. AppsConnect tests cover page/dialog parity,
task-bound OAuth, popup handoff, and verifying a saved connection before closing.
Withdrawn requests render a terminal acknowledgement with no setup action; the
interaction-body test and narrow Storybook journey cover this recovery state.
The connection-intents browser suite above exercises fixture-backed continuation.
Manual: ask an agent to use an unavailable app, open its request as the named
maps/feature-map/projects.md
+8
View File
@@ -56,6 +56,14 @@ Manual: Attach a small repository, save, and create a project task. Inspect the
- See [workspaces](./workspaces.md), [goals](./goals.md), and [budgets](./budgets-costs.md) for those features.
- Do not assume a deleted or archived project should erase its historical task evidence.
### Automatic repository skills
Creating or attaching a GitHub repository enqueues skill import without another setup
step. Check Skills → Sources and the installed library afterward. Ordinary edits to
older projects do not adopt their repositories; explicit adoption belongs in Sources.
Removing a repository retains installed skills and originating project access but stops
its automatic tracking. Changing the ref does not retarget existing skill references.
### Documentation scope: Projects and execution workspaces
Provider-specific provisioning, multi-repository/submodule edge cases, Git reconciliation variants, and all service exposure combinations require deeper recipes.
maps/feature-map/routines.md
+8
View File
@@ -60,6 +60,14 @@ Manual: Set a near-term disposable schedule or authorized test webhook, verify o
and verify the same generation before stopping. Local execution and catch-up
remain unchanged. Unsupported triggers and other pending work keep it awake.
### Latest skill references
Routine instructions retain stable skill mentions. After a project source syncs an
upstream change, the next execution uses the latest installed version without waiting
on GitHub. Record the resolved version before/after and verify a resumed execution
keeps its original version. Private skill references require current project access
for both the agent and its responsible person.
### Storybook verification
#### `run-now`
maps/feature-map/skills.md
+15
View File
@@ -66,6 +66,21 @@ Manual: Enable the skill for one test agent and leave another without access. Ru
- Read-only GitHub/bundled skills may require a writable copy rather than an in-place edit.
- Studio interaction coverage is in [questions and approvals](./questions-and-approvals.md); a passing mock is not a live harness test.
### Project repository source behavior
New GitHub project repositories import skills automatically. Skills → Sources lists
accessible legacy repositories with Import skills, shows source/project provenance,
and provides Refresh and Automatic updates in its source menu. Verify future additions,
retained exclusions, error recovery and reload persistence. Private-only sources must
be absent for unrelated people/agents; access through either originating project is sufficient.
Storybook/browser coverage uses scoped fixtures. Live GitHub-to-routine execution
must be verified separately; inspect resolved version IDs across an upstream update.
### Connection repair during an update
Open Choose connection while a source is being refreshed. Select an authorized account and save. If the source changes, the dialog must retain that selection, load the current revision, and allow a second save without reopening. The [project repository browser suite](../../tests/storybook-visual/project-repository-skills.spec.ts) covers this recovery with simulated upstream data.
### Storybook verification
#### `testing`
packages/adapters/codex-local/src/server/execute.ts
+48 -22
View File
@@ -1,5 +1,6 @@
import { createProviderStoppedBoundary } from "@paperclipai/adapter-utils/provider-stopped-boundary";
import { createUsageCheckpointLog } from "@paperclipai/adapter-utils/usage-checkpoint";
import { createHash } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
@@ -9,6 +10,7 @@ import { copyBackCodexAuth } from "./codex-auth-copyback.js";
import {
ensureCodexAuthCacheEntryDir,
isCodexAuthCacheEnabled,
readSubscriptionAccountId,
resolveCodexAuthCacheEntryPath,
selectVendCredential,
} from "./codex-auth-cache.js";
@@ -642,27 +644,50 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
typeof envConfig.CODEX_HOME === "string" && envConfig.CODEX_HOME.trim().length > 0
? path.resolve(envConfig.CODEX_HOME.trim())
: null;
const connectorSourceHome = configuredCodexHome;
const connectorSkillDigest = typeof config.paperclipConnectorSkillDigest === "string"
&& /^[a-f0-9]{64}$/.test(config.paperclipConnectorSkillDigest) ? config.paperclipConnectorSkillDigest : null;
if (connectorSkillDigest) {
// Never mount assignment-specific skills into the shared company/user home.
// A different skill revision gets a new home, so revoked/changed resources
// cannot survive as stale symlinks or bleed into another agent's session.
configuredCodexHome = path.join(resolveManagedCodexHomeDir(process.env, agent.companyId),
"connector-runtimes", agent.id, connectorSkillDigest);
}
const codexSkillEntries = (await readPaperclipRuntimeSkillEntries(config, __moduleDir))
// A missing-source entry would become a dangling skill symlink; skip it.
.filter((entry) => !isPaperclipSkillSourceMissing(entry));
const desiredSkillNames = resolveCodexDesiredSkillNames(config, codexSkillEntries);
if (!executionTargetIsRemote) {
await ensureAbsoluteDirectory(cwd, { createIfMissing: true });
}
const configuredOpenAiApiKey =
typeof envConfig.OPENAI_API_KEY === "string" && envConfig.OPENAI_API_KEY.trim().length > 0
? envConfig.OPENAI_API_KEY.trim()
: null;
const skillSourceHome = configuredCodexHome;
const connectorSkillDigest = typeof config.paperclipConnectorSkillDigest === "string"
&& /^[a-f0-9]{64}$/.test(config.paperclipConnectorSkillDigest) ? config.paperclipConnectorSkillDigest : null;
const codexSkillEntries = (await readPaperclipRuntimeSkillEntries(config, __moduleDir))
.filter((entry) => !isPaperclipSkillSourceMissing(entry));
const desiredSkillNames = resolveCodexDesiredSkillNames(config, codexSkillEntries);
const desiredSet = new Set(desiredSkillNames);
// An explicit inventory is the server's authorized execution selection. Include
// empty inventories: revocation must not fall back to a previously populated home.
let runtimeSkillDigest = Array.isArray(config.paperclipRuntimeSkills)
? createHash("sha256").update(JSON.stringify({ connectorSkillDigest,
skills: codexSkillEntries.filter(entry => desiredSet.has(entry.key))
.map(entry => [entry.key, entry.source]).sort((a, b) => JSON.stringify(a).localeCompare(JSON.stringify(b))),
})).digest("hex")
: connectorSkillDigest;
const runtimeCredentialHome = runtimeSkillDigest
? skillSourceHome ?? await prepareManagedCodexHome(process.env, onLog, agent.companyId, { apiKey: configuredOpenAiApiKey })
: null;
if (runtimeSkillDigest) {
const credentialHome = runtimeCredentialHome!;
const credentialBytes = await fs.readFile(path.join(credentialHome, "auth.json")).catch(() => null);
const selectedConfig = await fs.readFile(path.join(credentialHome, "config.toml")).catch(() => null);
// Keep selected-home provider/MCP edits (and removals) in the launched home.
const selectedConfigDigest = selectedConfig ? createHash("sha256").update(selectedConfig).digest("hex") : null;
// Copy-back may retain a refreshed regular credential in a generated home.
// Account switches must select a different home, while token refreshes for
// the same subscription identity preserve its cache and session state.
const credentialIdentity = credentialBytes
? readSubscriptionAccountId(credentialBytes) ?? createHash("sha256").update(credentialBytes).digest("hex") : null;
runtimeSkillDigest = createHash("sha256").update(JSON.stringify({
runtimeSkillDigest, credentialHome, credentialIdentity, selectedConfigDigest, apiKey: envConfig.OPENAI_API_KEY ?? null,
})).digest("hex");
// Different authorized versions/assignments get separate homes, including
// session state. Previously permitted private links cannot survive revocation.
configuredCodexHome = path.join(resolveManagedCodexHomeDir(process.env, agent.companyId),
connectorSkillDigest ? "connector-runtimes" : "skill-runtimes", agent.id, runtimeSkillDigest);
}
if (!executionTargetIsRemote) {
await ensureAbsoluteDirectory(cwd, { createIfMissing: true });
}
// A configured CODEX_HOME that lives under the Paperclip-managed company tree
// (the per-agent home set by the server isolation guard) still needs auth
// seeded — it ships with no credentials and OPENAI_API_KEY="" by default.
@@ -697,14 +722,14 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
void error;
});
}
if (configuredCodexHome == null || (connectorSkillDigest && connectorSourceHome == null)) {
if (configuredCodexHome == null) {
await prepareManagedCodexHome(process.env, onLog, agent.companyId, {
apiKey: configuredOpenAiApiKey,
});
}
if (configuredHomeIsManaged && configuredCodexHome) {
const seedEnv = connectorSkillDigest ? {
...process.env, CODEX_HOME: connectorSourceHome ?? resolveManagedCodexHomeDir(process.env, agent.companyId),
const seedEnv = runtimeSkillDigest ? {
...process.env, CODEX_HOME: runtimeCredentialHome ?? resolveManagedCodexHomeDir(process.env, agent.companyId),
} : process.env;
await seedManagedCodexHome(configuredCodexHome, seedEnv, onLog, {
apiKey: configuredOpenAiApiKey,
@@ -737,7 +762,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
// Merge custom model providers (PAPERCLIP_CODEX_PROVIDERS) into the managed
// CODEX_HOME's config.toml BEFORE the home is shipped to a remote execution
// target, so both local and sandboxed Codex processes pick up the routing.
// An explicit env.CODEX_HOME override is treated as user-managed and skipped.
// A caller-owned home stays untouched. A generated skill home receives the
// configured provider routing even when it seeds credentials from that home.
const envConfigStrings = Object.fromEntries(
Object.entries(envConfig).filter(
(entry): entry is [string, string] => typeof entry[1] === "string",
@@ -745,7 +771,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
);
const preparedRuntimeConfig = await prepareCodexRuntimeConfig({
env: envConfigStrings,
codexHome: configuredCodexHome ? null : effectiveCodexHome,
codexHome: configuredCodexHome && !runtimeSkillDigest ? null : effectiveCodexHome,
});
// Curated allowlist dir staged for the remote `home` asset (see below). Held
// here so the outer `finally` can remove it on every exit path (teardown and
packages/db/src/migrations/0341_abnormal_bucky.sql
+78
View File
@@ -0,0 +1,78 @@
-- Replay-safe after renumbering: preserve existing imports, provenance and scheduling.
CREATE TABLE IF NOT EXISTS "project_skill_sources" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"company_id" uuid NOT NULL,
"project_id" uuid NOT NULL,
"workspace_id" uuid,
"source_id" uuid,
"repository_url" text NOT NULL,
"tracking_ref" text NOT NULL,
"active" boolean DEFAULT true NOT NULL,
"status" text DEFAULT 'pending' NOT NULL,
"refresh_user_id" text,
"refresh_agent_id" uuid,
"refresh_local_trusted" boolean DEFAULT false NOT NULL,
"connection_id" uuid,
"refresh_grant_id" uuid,
"next_sync_at" timestamp with time zone DEFAULT now() NOT NULL,
"failure_count" integer DEFAULT 0 NOT NULL,
"last_error" text,
"lease_token" uuid,
"lease_expires_at" timestamp with time zone,
"created_at" timestamp with time zone DEFAULT now() NOT NULL
);
--> statement-breakpoint
ALTER TABLE "company_skill_sources" DROP CONSTRAINT IF EXISTS "company_skill_sources_connection_id_tool_connections_id_fk";
--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "automatic_updates" boolean DEFAULT true NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "import_new_skills" boolean DEFAULT false NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "access_scope" text DEFAULT 'company' NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "refresh_user_id" text;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "refresh_agent_id" uuid;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "refresh_local_trusted" boolean DEFAULT false NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "refresh_grant_id" uuid;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "next_sync_at" timestamp with time zone DEFAULT now() NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "failure_count" integer DEFAULT 0 NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skills" ADD COLUMN IF NOT EXISTS "access_source_id" uuid;--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'project_skill_sources_company_id_companies_id_fk' AND conrelid = 'public.project_skill_sources'::regclass) THEN
ALTER TABLE "project_skill_sources" ADD CONSTRAINT "project_skill_sources_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'project_skill_sources_project_id_projects_id_fk' AND conrelid = 'public.project_skill_sources'::regclass) THEN
ALTER TABLE "project_skill_sources" ADD CONSTRAINT "project_skill_sources_project_id_projects_id_fk" FOREIGN KEY ("project_id") REFERENCES "public"."projects"("id") ON DELETE cascade ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'project_skill_sources_workspace_id_project_workspaces_id_fk' AND conrelid = 'public.project_skill_sources'::regclass) THEN
ALTER TABLE "project_skill_sources" ADD CONSTRAINT "project_skill_sources_workspace_id_project_workspaces_id_fk" FOREIGN KEY ("workspace_id") REFERENCES "public"."project_workspaces"("id") ON DELETE set null ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'project_skill_sources_source_id_company_skill_sources_id_fk' AND conrelid = 'public.project_skill_sources'::regclass) THEN
ALTER TABLE "project_skill_sources" ADD CONSTRAINT "project_skill_sources_source_id_company_skill_sources_id_fk" FOREIGN KEY ("source_id") REFERENCES "public"."company_skill_sources"("id") ON DELETE set null ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
CREATE UNIQUE INDEX IF NOT EXISTS "project_skill_sources_workspace_repo_ref_idx" ON "project_skill_sources" USING btree ("workspace_id","repository_url","tracking_ref");--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "project_skill_sources_source_project_idx" ON "project_skill_sources" USING btree ("company_id","source_id","project_id");--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "project_skill_sources_sync_idx" ON "project_skill_sources" USING btree ("active","status","next_sync_at");
--> statement-breakpoint
-- Retain setup identity if a connection or agent is deleted, including preview schemas.
ALTER TABLE "company_skill_sources" DROP CONSTRAINT IF EXISTS "company_skill_sources_refresh_agent_id_agents_id_fk";
--> statement-breakpoint
ALTER TABLE "project_skill_sources" DROP CONSTRAINT IF EXISTS "project_skill_sources_refresh_agent_id_agents_id_fk";
--> statement-breakpoint
ALTER TABLE "project_skill_sources" DROP CONSTRAINT IF EXISTS "project_skill_sources_connection_id_tool_connections_id_fk";
--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "removed_at" timestamp with time zone;
--> statement-breakpoint
DROP INDEX IF EXISTS "company_skill_sources_repository_ref_idx";--> statement-breakpoint
DROP INDEX IF EXISTS "company_skill_sources_identity_ref_idx";--> statement-breakpoint
CREATE UNIQUE INDEX "company_skill_sources_repository_ref_idx" ON "company_skill_sources" USING btree ("company_id","repository_url","tracking_ref") WHERE "company_skill_sources"."removed_at" is null;--> statement-breakpoint
CREATE UNIQUE INDEX "company_skill_sources_identity_ref_idx" ON "company_skill_sources" USING btree ("company_id","repository_id","tracking_ref") WHERE "company_skill_sources"."removed_at" is null;
--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "last_scan_tree_sha" text;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "automatic_budget_bytes" integer DEFAULT 536870912 NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN IF NOT EXISTS "automatic_budget_at" timestamp with time zone DEFAULT now() NOT NULL;
packages/db/src/migrations/0342_dry_shen.sql
+7
View File
@@ -0,0 +1,7 @@
CREATE TABLE "company_skill_import_budgets" (
"company_id" uuid PRIMARY KEY NOT NULL,
"available_bytes" integer DEFAULT 536870912 NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
);
--> statement-breakpoint
ALTER TABLE "company_skill_import_budgets" ADD CONSTRAINT "company_skill_import_budgets_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;
packages/db/src/migrations/meta/0337_snapshot.json → packages/db/src/migrations/meta/0341_snapshot.json
Generated
+569 -9
View File
@@ -1,6 +1,6 @@
{
"id": "65f48b10-7f94-449e-b2ce-a68f220910f3",
"prevId": "a9f48522-347d-4290-83dd-07299afb8d03",
"id": "927d7532-059b-4a4f-846a-5e3445a81891",
"prevId": "b32fa655-79c7-4793-a021-22258b6c369c",
"version": "7",
"dialect": "postgresql",
"tables": {
@@ -16324,6 +16324,18 @@
"primaryKey": false,
"notNull": false
},
"owner_agent_id": {
"name": "owner_agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"owner_plugin_id": {
"name": "owner_plugin_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"user_secret_definition_id": {
"name": "user_secret_definition_id",
"type": "uuid",
@@ -16552,6 +16564,40 @@
"method": "btree",
"with": {}
},
"company_secrets_instance_owner_key_uq": {
"name": "company_secrets_instance_owner_key_uq",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "owner_agent_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "owner_plugin_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "key",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"where": "\"company_secrets\".\"scope\" = 'instance' and \"company_secrets\".\"deleted_at\" is null",
"concurrently": false,
"method": "btree",
"with": {}
},
"company_secrets_provider_config_idx": {
"name": "company_secrets_provider_config_idx",
"columns": [
@@ -16654,6 +16700,19 @@
"onDelete": "no action",
"onUpdate": "no action"
},
"company_secrets_owner_plugin_id_plugins_id_fk": {
"name": "company_secrets_owner_plugin_id_plugins_id_fk",
"tableFrom": "company_secrets",
"tableTo": "plugins",
"columnsFrom": [
"owner_plugin_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
},
"company_secrets_user_secret_definition_id_user_secret_definitions_id_fk": {
"name": "company_secrets_user_secret_definition_id_user_secret_definitions_id_fk",
"tableFrom": "company_secrets",
@@ -16692,6 +16751,21 @@
],
"onDelete": "set null",
"onUpdate": "no action"
},
"company_secrets_company_id_owner_agent_id_agents_company_id_id_fk": {
"name": "company_secrets_company_id_owner_agent_id_agents_company_id_id_fk",
"tableFrom": "company_secrets",
"tableTo": "agents",
"columnsFrom": [
"company_id",
"owner_agent_id"
],
"columnsTo": [
"company_id",
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
@@ -16700,7 +16774,7 @@
"checkConstraints": {
"company_secrets_scope_shape_check": {
"name": "company_secrets_scope_shape_check",
"value": "(\n \"company_secrets\".\"scope\" = 'company'\n and \"company_secrets\".\"owner_user_id\" is null\n and \"company_secrets\".\"user_secret_definition_id\" is null\n ) or (\n \"company_secrets\".\"scope\" = 'user'\n and \"company_secrets\".\"owner_user_id\" is not null\n and \"company_secrets\".\"user_secret_definition_id\" is not null\n )"
"value": "(\n \"company_secrets\".\"scope\" = 'company'\n and \"company_secrets\".\"owner_user_id\" is null\n and \"company_secrets\".\"user_secret_definition_id\" is null\n and \"company_secrets\".\"owner_agent_id\" is null and \"company_secrets\".\"owner_plugin_id\" is null\n ) or (\n \"company_secrets\".\"scope\" = 'user'\n and \"company_secrets\".\"owner_user_id\" is not null\n and \"company_secrets\".\"user_secret_definition_id\" is not null\n and \"company_secrets\".\"owner_agent_id\" is null and \"company_secrets\".\"owner_plugin_id\" is null\n ) or (\n \"company_secrets\".\"scope\" = 'instance'\n and \"company_secrets\".\"owner_user_id\" is null and \"company_secrets\".\"user_secret_definition_id\" is null\n and \"company_secrets\".\"owner_agent_id\" is not null and \"company_secrets\".\"owner_plugin_id\" is not null\n )"
}
},
"isRLSEnabled": false
@@ -17017,6 +17091,72 @@
"notNull": true,
"default": true
},
"removed_at": {
"name": "removed_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": false
},
"automatic_updates": {
"name": "automatic_updates",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": true
},
"import_new_skills": {
"name": "import_new_skills",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": false
},
"access_scope": {
"name": "access_scope",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'company'"
},
"refresh_user_id": {
"name": "refresh_user_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"refresh_agent_id": {
"name": "refresh_agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"refresh_local_trusted": {
"name": "refresh_local_trusted",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": false
},
"refresh_grant_id": {
"name": "refresh_grant_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"next_sync_at": {
"name": "next_sync_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"failure_count": {
"name": "failure_count",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 0
},
"revision": {
"name": "revision",
"type": "integer",
@@ -17042,6 +17182,26 @@
"primaryKey": false,
"notNull": false
},
"last_scan_tree_sha": {
"name": "last_scan_tree_sha",
"type": "text",
"primaryKey": false,
"notNull": false
},
"automatic_budget_bytes": {
"name": "automatic_budget_bytes",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 536870912
},
"automatic_budget_at": {
"name": "automatic_budget_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"last_error": {
"name": "last_error",
"type": "text",
@@ -17092,6 +17252,7 @@
}
],
"isUnique": true,
"where": "\"company_skill_sources\".\"removed_at\" is null",
"concurrently": false,
"method": "btree",
"with": {}
@@ -17119,6 +17280,7 @@
}
],
"isUnique": true,
"where": "\"company_skill_sources\".\"removed_at\" is null",
"concurrently": false,
"method": "btree",
"with": {}
@@ -17137,13 +17299,275 @@
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.project_skill_sources": {
"name": "project_skill_sources",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "uuid",
"primaryKey": true,
"notNull": true,
"default": "gen_random_uuid()"
},
"company_skill_sources_connection_id_tool_connections_id_fk": {
"name": "company_skill_sources_connection_id_tool_connections_id_fk",
"tableFrom": "company_skill_sources",
"tableTo": "tool_connections",
"company_id": {
"name": "company_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"project_id": {
"name": "project_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"workspace_id": {
"name": "workspace_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"source_id": {
"name": "source_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"repository_url": {
"name": "repository_url",
"type": "text",
"primaryKey": false,
"notNull": true
},
"tracking_ref": {
"name": "tracking_ref",
"type": "text",
"primaryKey": false,
"notNull": true
},
"active": {
"name": "active",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": true
},
"status": {
"name": "status",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'pending'"
},
"refresh_user_id": {
"name": "refresh_user_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"refresh_agent_id": {
"name": "refresh_agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"refresh_local_trusted": {
"name": "refresh_local_trusted",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": false
},
"connection_id": {
"name": "connection_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"refresh_grant_id": {
"name": "refresh_grant_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"next_sync_at": {
"name": "next_sync_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"failure_count": {
"name": "failure_count",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 0
},
"last_error": {
"name": "last_error",
"type": "text",
"primaryKey": false,
"notNull": false
},
"lease_token": {
"name": "lease_token",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"lease_expires_at": {
"name": "lease_expires_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
}
},
"indexes": {
"project_skill_sources_workspace_repo_ref_idx": {
"name": "project_skill_sources_workspace_repo_ref_idx",
"columns": [
{
"expression": "workspace_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "repository_url",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "tracking_ref",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"concurrently": false,
"method": "btree",
"with": {}
},
"project_skill_sources_source_project_idx": {
"name": "project_skill_sources_source_project_idx",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "source_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "project_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"project_skill_sources_sync_idx": {
"name": "project_skill_sources_sync_idx",
"columns": [
{
"expression": "active",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "status",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "next_sync_at",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {
"project_skill_sources_company_id_companies_id_fk": {
"name": "project_skill_sources_company_id_companies_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "companies",
"columnsFrom": [
"connection_id"
"company_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
},
"project_skill_sources_project_id_projects_id_fk": {
"name": "project_skill_sources_project_id_projects_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "projects",
"columnsFrom": [
"project_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
},
"project_skill_sources_workspace_id_project_workspaces_id_fk": {
"name": "project_skill_sources_workspace_id_project_workspaces_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "project_workspaces",
"columnsFrom": [
"workspace_id"
],
"columnsTo": [
"id"
],
"onDelete": "set null",
"onUpdate": "no action"
},
"project_skill_sources_source_id_company_skill_sources_id_fk": {
"name": "project_skill_sources_source_id_company_skill_sources_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "company_skill_sources",
"columnsFrom": [
"source_id"
],
"columnsTo": [
"id"
@@ -18603,6 +19027,12 @@
"notNull": true,
"default": "'company'"
},
"access_source_id": {
"name": "access_source_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"public_share_token": {
"name": "public_share_token",
"type": "text",
@@ -26839,6 +27269,136 @@
"checkConstraints": {},
"isRLSEnabled": false
},
"public.execution_workspace_repository_access": {
"name": "execution_workspace_repository_access",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "uuid",
"primaryKey": true,
"notNull": true,
"default": "gen_random_uuid()"
},
"company_id": {
"name": "company_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"root_key": {
"name": "root_key",
"type": "text",
"primaryKey": false,
"notNull": true
},
"canonical_cwd": {
"name": "canonical_cwd",
"type": "text",
"primaryKey": false,
"notNull": true
},
"kind": {
"name": "kind",
"type": "text",
"primaryKey": false,
"notNull": true
},
"scope_id": {
"name": "scope_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"issue_id": {
"name": "issue_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"agent_id": {
"name": "agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"responsible_user_id": {
"name": "responsible_user_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"updated_at": {
"name": "updated_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
}
},
"indexes": {
"execution_workspace_repository_access_scope_idx": {
"name": "execution_workspace_repository_access_scope_idx",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "root_key",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "kind",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "scope_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {
"execution_workspace_repository_access_company_id_companies_id_fk": {
"name": "execution_workspace_repository_access_company_id_companies_id_fk",
"tableFrom": "execution_workspace_repository_access",
"tableTo": "companies",
"columnsFrom": [
"company_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.execution_workspace_repository_roots": {
"name": "execution_workspace_repository_roots",
"schema": "",
@@ -45996,7 +46556,7 @@
},
"resource_lifecycle_events_action_check": {
"name": "resource_lifecycle_events_action_check",
"value": "\"resource_lifecycle_events\".\"action\" = 'create' OR (\"resource_lifecycle_events\".\"resource_type\" = 'project' AND \"resource_lifecycle_events\".\"action\" IN ('update', 'archive')) OR (\"resource_lifecycle_events\".\"resource_type\" = 'agent' AND \"resource_lifecycle_events\".\"action\" IN ('pause', 'resume', 'terminate'))"
"value": "\"resource_lifecycle_events\".\"action\" = 'create' OR (\"resource_lifecycle_events\".\"resource_type\" = 'project' AND \"resource_lifecycle_events\".\"action\" IN ('update', 'archive', 'delete')) OR (\"resource_lifecycle_events\".\"resource_type\" = 'agent' AND \"resource_lifecycle_events\".\"action\" IN ('pause', 'resume', 'terminate'))"
}
},
"isRLSEnabled": false
packages/db/src/migrations/meta/0336_snapshot.json → packages/db/src/migrations/meta/0342_snapshot.json
Generated
+710 -9
View File
@@ -1,6 +1,6 @@
{
"id": "a9f48522-347d-4290-83dd-07299afb8d03",
"prevId": "18c156f2-8344-4282-98b7-f702f0035c81",
"id": "e9a4bbc2-b22c-48a1-a74c-f108cd7b20ec",
"prevId": "927d7532-059b-4a4f-846a-5e3445a81891",
"version": "7",
"dialect": "postgresql",
"tables": {
@@ -16324,6 +16324,18 @@
"primaryKey": false,
"notNull": false
},
"owner_agent_id": {
"name": "owner_agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"owner_plugin_id": {
"name": "owner_plugin_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"user_secret_definition_id": {
"name": "user_secret_definition_id",
"type": "uuid",
@@ -16552,6 +16564,40 @@
"method": "btree",
"with": {}
},
"company_secrets_instance_owner_key_uq": {
"name": "company_secrets_instance_owner_key_uq",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "owner_agent_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "owner_plugin_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "key",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"where": "\"company_secrets\".\"scope\" = 'instance' and \"company_secrets\".\"deleted_at\" is null",
"concurrently": false,
"method": "btree",
"with": {}
},
"company_secrets_provider_config_idx": {
"name": "company_secrets_provider_config_idx",
"columns": [
@@ -16654,6 +16700,19 @@
"onDelete": "no action",
"onUpdate": "no action"
},
"company_secrets_owner_plugin_id_plugins_id_fk": {
"name": "company_secrets_owner_plugin_id_plugins_id_fk",
"tableFrom": "company_secrets",
"tableTo": "plugins",
"columnsFrom": [
"owner_plugin_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
},
"company_secrets_user_secret_definition_id_user_secret_definitions_id_fk": {
"name": "company_secrets_user_secret_definition_id_user_secret_definitions_id_fk",
"tableFrom": "company_secrets",
@@ -16692,6 +16751,21 @@
],
"onDelete": "set null",
"onUpdate": "no action"
},
"company_secrets_company_id_owner_agent_id_agents_company_id_id_fk": {
"name": "company_secrets_company_id_owner_agent_id_agents_company_id_id_fk",
"tableFrom": "company_secrets",
"tableTo": "agents",
"columnsFrom": [
"company_id",
"owner_agent_id"
],
"columnsTo": [
"company_id",
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
@@ -16700,7 +16774,7 @@
"checkConstraints": {
"company_secrets_scope_shape_check": {
"name": "company_secrets_scope_shape_check",
"value": "(\n \"company_secrets\".\"scope\" = 'company'\n and \"company_secrets\".\"owner_user_id\" is null\n and \"company_secrets\".\"user_secret_definition_id\" is null\n ) or (\n \"company_secrets\".\"scope\" = 'user'\n and \"company_secrets\".\"owner_user_id\" is not null\n and \"company_secrets\".\"user_secret_definition_id\" is not null\n )"
"value": "(\n \"company_secrets\".\"scope\" = 'company'\n and \"company_secrets\".\"owner_user_id\" is null\n and \"company_secrets\".\"user_secret_definition_id\" is null\n and \"company_secrets\".\"owner_agent_id\" is null and \"company_secrets\".\"owner_plugin_id\" is null\n ) or (\n \"company_secrets\".\"scope\" = 'user'\n and \"company_secrets\".\"owner_user_id\" is not null\n and \"company_secrets\".\"user_secret_definition_id\" is not null\n and \"company_secrets\".\"owner_agent_id\" is null and \"company_secrets\".\"owner_plugin_id\" is null\n ) or (\n \"company_secrets\".\"scope\" = 'instance'\n and \"company_secrets\".\"owner_user_id\" is null and \"company_secrets\".\"user_secret_definition_id\" is null\n and \"company_secrets\".\"owner_agent_id\" is not null and \"company_secrets\".\"owner_plugin_id\" is not null\n )"
}
},
"isRLSEnabled": false
@@ -16778,6 +16852,53 @@
"checkConstraints": {},
"isRLSEnabled": false
},
"public.company_skill_import_budgets": {
"name": "company_skill_import_budgets",
"schema": "",
"columns": {
"company_id": {
"name": "company_id",
"type": "uuid",
"primaryKey": true,
"notNull": true
},
"available_bytes": {
"name": "available_bytes",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 536870912
},
"updated_at": {
"name": "updated_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
}
},
"indexes": {},
"foreignKeys": {
"company_skill_import_budgets_company_id_companies_id_fk": {
"name": "company_skill_import_budgets_company_id_companies_id_fk",
"tableFrom": "company_skill_import_budgets",
"tableTo": "companies",
"columnsFrom": [
"company_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.company_skill_source_entries": {
"name": "company_skill_source_entries",
"schema": "",
@@ -17017,6 +17138,72 @@
"notNull": true,
"default": true
},
"removed_at": {
"name": "removed_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": false
},
"automatic_updates": {
"name": "automatic_updates",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": true
},
"import_new_skills": {
"name": "import_new_skills",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": false
},
"access_scope": {
"name": "access_scope",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'company'"
},
"refresh_user_id": {
"name": "refresh_user_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"refresh_agent_id": {
"name": "refresh_agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"refresh_local_trusted": {
"name": "refresh_local_trusted",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": false
},
"refresh_grant_id": {
"name": "refresh_grant_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"next_sync_at": {
"name": "next_sync_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"failure_count": {
"name": "failure_count",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 0
},
"revision": {
"name": "revision",
"type": "integer",
@@ -17042,6 +17229,26 @@
"primaryKey": false,
"notNull": false
},
"last_scan_tree_sha": {
"name": "last_scan_tree_sha",
"type": "text",
"primaryKey": false,
"notNull": false
},
"automatic_budget_bytes": {
"name": "automatic_budget_bytes",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 536870912
},
"automatic_budget_at": {
"name": "automatic_budget_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"last_error": {
"name": "last_error",
"type": "text",
@@ -17092,6 +17299,7 @@
}
],
"isUnique": true,
"where": "\"company_skill_sources\".\"removed_at\" is null",
"concurrently": false,
"method": "btree",
"with": {}
@@ -17119,6 +17327,7 @@
}
],
"isUnique": true,
"where": "\"company_skill_sources\".\"removed_at\" is null",
"concurrently": false,
"method": "btree",
"with": {}
@@ -17137,13 +17346,275 @@
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.project_skill_sources": {
"name": "project_skill_sources",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "uuid",
"primaryKey": true,
"notNull": true,
"default": "gen_random_uuid()"
},
"company_skill_sources_connection_id_tool_connections_id_fk": {
"name": "company_skill_sources_connection_id_tool_connections_id_fk",
"tableFrom": "company_skill_sources",
"tableTo": "tool_connections",
"company_id": {
"name": "company_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"project_id": {
"name": "project_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"workspace_id": {
"name": "workspace_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"source_id": {
"name": "source_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"repository_url": {
"name": "repository_url",
"type": "text",
"primaryKey": false,
"notNull": true
},
"tracking_ref": {
"name": "tracking_ref",
"type": "text",
"primaryKey": false,
"notNull": true
},
"active": {
"name": "active",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": true
},
"status": {
"name": "status",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'pending'"
},
"refresh_user_id": {
"name": "refresh_user_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"refresh_agent_id": {
"name": "refresh_agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"refresh_local_trusted": {
"name": "refresh_local_trusted",
"type": "boolean",
"primaryKey": false,
"notNull": true,
"default": false
},
"connection_id": {
"name": "connection_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"refresh_grant_id": {
"name": "refresh_grant_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"next_sync_at": {
"name": "next_sync_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"failure_count": {
"name": "failure_count",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 0
},
"last_error": {
"name": "last_error",
"type": "text",
"primaryKey": false,
"notNull": false
},
"lease_token": {
"name": "lease_token",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"lease_expires_at": {
"name": "lease_expires_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
}
},
"indexes": {
"project_skill_sources_workspace_repo_ref_idx": {
"name": "project_skill_sources_workspace_repo_ref_idx",
"columns": [
{
"expression": "workspace_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "repository_url",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "tracking_ref",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"concurrently": false,
"method": "btree",
"with": {}
},
"project_skill_sources_source_project_idx": {
"name": "project_skill_sources_source_project_idx",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "source_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "project_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"project_skill_sources_sync_idx": {
"name": "project_skill_sources_sync_idx",
"columns": [
{
"expression": "active",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "status",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "next_sync_at",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {
"project_skill_sources_company_id_companies_id_fk": {
"name": "project_skill_sources_company_id_companies_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "companies",
"columnsFrom": [
"connection_id"
"company_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
},
"project_skill_sources_project_id_projects_id_fk": {
"name": "project_skill_sources_project_id_projects_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "projects",
"columnsFrom": [
"project_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
},
"project_skill_sources_workspace_id_project_workspaces_id_fk": {
"name": "project_skill_sources_workspace_id_project_workspaces_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "project_workspaces",
"columnsFrom": [
"workspace_id"
],
"columnsTo": [
"id"
],
"onDelete": "set null",
"onUpdate": "no action"
},
"project_skill_sources_source_id_company_skill_sources_id_fk": {
"name": "project_skill_sources_source_id_company_skill_sources_id_fk",
"tableFrom": "project_skill_sources",
"tableTo": "company_skill_sources",
"columnsFrom": [
"source_id"
],
"columnsTo": [
"id"
@@ -18603,6 +19074,12 @@
"notNull": true,
"default": "'company'"
},
"access_source_id": {
"name": "access_source_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"public_share_token": {
"name": "public_share_token",
"type": "text",
@@ -26839,6 +27316,230 @@
"checkConstraints": {},
"isRLSEnabled": false
},
"public.execution_workspace_repository_access": {
"name": "execution_workspace_repository_access",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "uuid",
"primaryKey": true,
"notNull": true,
"default": "gen_random_uuid()"
},
"company_id": {
"name": "company_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"root_key": {
"name": "root_key",
"type": "text",
"primaryKey": false,
"notNull": true
},
"canonical_cwd": {
"name": "canonical_cwd",
"type": "text",
"primaryKey": false,
"notNull": true
},
"kind": {
"name": "kind",
"type": "text",
"primaryKey": false,
"notNull": true
},
"scope_id": {
"name": "scope_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"issue_id": {
"name": "issue_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"agent_id": {
"name": "agent_id",
"type": "uuid",
"primaryKey": false,
"notNull": false
},
"responsible_user_id": {
"name": "responsible_user_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"updated_at": {
"name": "updated_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
}
},
"indexes": {
"execution_workspace_repository_access_scope_idx": {
"name": "execution_workspace_repository_access_scope_idx",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "root_key",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "kind",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "scope_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {
"execution_workspace_repository_access_company_id_companies_id_fk": {
"name": "execution_workspace_repository_access_company_id_companies_id_fk",
"tableFrom": "execution_workspace_repository_access",
"tableTo": "companies",
"columnsFrom": [
"company_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.execution_workspace_repository_roots": {
"name": "execution_workspace_repository_roots",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "uuid",
"primaryKey": true,
"notNull": true,
"default": "gen_random_uuid()"
},
"company_id": {
"name": "company_id",
"type": "uuid",
"primaryKey": false,
"notNull": true
},
"canonical_cwd": {
"name": "canonical_cwd",
"type": "text",
"primaryKey": false,
"notNull": true
},
"root_key": {
"name": "root_key",
"type": "text",
"primaryKey": false,
"notNull": true
},
"mode": {
"name": "mode",
"type": "text",
"primaryKey": false,
"notNull": true
},
"created_at": {
"name": "created_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
},
"updated_at": {
"name": "updated_at",
"type": "timestamp with time zone",
"primaryKey": false,
"notNull": true,
"default": "now()"
}
},
"indexes": {
"execution_workspace_repository_roots_company_root_idx": {
"name": "execution_workspace_repository_roots_company_root_idx",
"columns": [
{
"expression": "company_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "root_key",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": true,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {
"execution_workspace_repository_roots_company_id_companies_id_fk": {
"name": "execution_workspace_repository_roots_company_id_companies_id_fk",
"tableFrom": "execution_workspace_repository_roots",
"tableTo": "companies",
"columnsFrom": [
"company_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.execution_workspace_runtime_leases": {
"name": "execution_workspace_runtime_leases",
"schema": "",
@@ -45902,7 +46603,7 @@
},
"resource_lifecycle_events_action_check": {
"name": "resource_lifecycle_events_action_check",
"value": "\"resource_lifecycle_events\".\"action\" = 'create' OR (\"resource_lifecycle_events\".\"resource_type\" = 'project' AND \"resource_lifecycle_events\".\"action\" IN ('update', 'archive')) OR (\"resource_lifecycle_events\".\"resource_type\" = 'agent' AND \"resource_lifecycle_events\".\"action\" IN ('pause', 'resume', 'terminate'))"
"value": "\"resource_lifecycle_events\".\"action\" = 'create' OR (\"resource_lifecycle_events\".\"resource_type\" = 'project' AND \"resource_lifecycle_events\".\"action\" IN ('update', 'archive', 'delete')) OR (\"resource_lifecycle_events\".\"resource_type\" = 'agent' AND \"resource_lifecycle_events\".\"action\" IN ('pause', 'resume', 'terminate'))"
}
},
"isRLSEnabled": false
packages/db/src/migrations/meta/_journal.json
Generated
+14
View File
@@ -2374,6 +2374,20 @@
"when": 1791911743153,
"tag": "0340_open_warlock",
"breakpoints": true
},
{
"idx": 341,
"version": "7",
"when": 1791919024567,
"tag": "0341_abnormal_bucky",
"breakpoints": true
},
{
"idx": 342,
"version": "7",
"when": 1791921858146,
"tag": "0342_dry_shen",
"breakpoints": true
}
]
}
packages/db/src/project-skill-sources-migration.test.ts
+54
View File
@@ -0,0 +1,54 @@
import { randomUUID } from "node:crypto";
import { readFile } from "node:fs/promises";
import postgres from "postgres";
import { describe, expect, it } from "vitest";
import { EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./test-embedded-postgres.js";
const support = await getEmbeddedPostgresTestSupport();
const describePostgres = support.supported ? describe : describe.skip;
describePostgres("project skill source migration replay", () => {
it("keeps imported content, private provenance and scheduling when renumbered SQL runs again", async () => {
const database = await startEmbeddedPostgresTestDatabase("paperclip-project-skill-migration-");
const sql = postgres(database.connectionString, { max: 1, onnotice: () => {} });
try {
const companyId = randomUUID(), projectId = randomUUID(), workspaceId = randomUUID();
const sourceId = randomUUID(), skillId = randomUUID(), bindingId = randomUUID(), agentId = randomUUID();
const connectionId = randomUUID();
await sql`INSERT INTO companies (id, name, issue_prefix) VALUES (${companyId}, 'Migration company', 'PSM')`;
await sql`INSERT INTO projects (id, company_id, name, visibility) VALUES (${projectId}, ${companyId}, 'Private project', 'private')`;
await sql`INSERT INTO project_workspaces (id, company_id, project_id, name, repo_url) VALUES (${workspaceId}, ${companyId}, ${projectId}, 'Repository', 'https://github.com/acme/skills')`;
await sql`INSERT INTO agents (id, company_id, name) VALUES (${agentId}, ${companyId}, 'Setup agent')`;
await sql`INSERT INTO company_skill_sources (id, company_id, repository_url, full_name, tracking_ref, access_scope, automatic_updates, import_new_skills, refresh_user_id, refresh_agent_id, connection_id, excluded_folders, revision, failure_count, next_sync_at)
VALUES (${sourceId}, ${companyId}, 'https://github.com/acme/skills', 'acme/skills', 'HEAD', 'projects', false, true, 'setup-person', ${agentId}, ${connectionId}, '["excluded"]', 7, 3, '2030-01-01')`;
await sql`INSERT INTO company_skills (id, company_id, key, slug, name, markdown, access_source_id) VALUES (${skillId}, ${companyId}, 'repo/review', 'review', 'Review', 'Keep installed content', ${sourceId})`;
await sql`INSERT INTO project_skill_sources (id, company_id, project_id, workspace_id, source_id, repository_url, tracking_ref, active, status, refresh_user_id, refresh_agent_id, connection_id)
VALUES (${bindingId}, ${companyId}, ${projectId}, ${workspaceId}, ${sourceId}, 'https://github.com/acme/skills', 'HEAD', false, 'ready', 'setup-person', ${agentId}, ${connectionId})`;
// Preview builds used an agent FK that would erase the authorizing identity.
await sql.unsafe('ALTER TABLE project_skill_sources ADD CONSTRAINT project_skill_sources_refresh_agent_id_agents_id_fk FOREIGN KEY (refresh_agent_id) REFERENCES agents(id) ON DELETE SET NULL');
const source = () => sql`SELECT * FROM company_skill_sources WHERE id = ${sourceId}`;
const binding = () => sql`SELECT * FROM project_skill_sources WHERE id = ${bindingId}`;
const skill = () => sql`SELECT * FROM company_skills WHERE id = ${skillId}`;
const relations = () => sql`SELECT oid::text, relname FROM pg_class WHERE relname IN ('company_skill_sources', 'project_skill_sources', 'company_skills') ORDER BY relname`;
const before = { source: await source(), binding: await binding(), skill: await skill(), relations: await relations() };
const migration = await readFile(new URL("./migrations/0341_abnormal_bucky.sql", import.meta.url), "utf8");
for (let attempt = 0; attempt < 2; attempt++) {
for (const statement of migration.split("--> statement-breakpoint")) {
if (statement.trim()) await sql.unsafe(statement);
}
expect(await source()).toEqual(before.source);
expect(await binding()).toEqual(before.binding);
expect(await skill()).toEqual(before.skill);
expect(await relations()).toEqual(before.relations);
}
await sql`DELETE FROM agents WHERE id = ${agentId}`;
expect((await source())[0]?.refresh_agent_id).toBe(agentId);
expect((await binding())[0]?.refresh_agent_id).toBe(agentId);
expect((await binding())[0]?.connection_id).toBe(connectionId);
expect((await skill())[0]?.access_source_id).toBe(sourceId);
} finally {
await sql.end();
await database.cleanup();
}
}, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
});
packages/db/src/schema/company_skill_sources.ts
+56 -4
View File
@@ -1,8 +1,10 @@
import { sql } from 'drizzle-orm';
import { pgTable, uuid, text, timestamp, jsonb, integer, boolean, uniqueIndex, index } from 'drizzle-orm/pg-core';
import type { SkillPackageInspection, SkillRepositoryPackage } from '@paperclipai/shared';
import { companies } from './companies.js';
import { companySkills } from './company_skills.js';
import { toolConnections } from './tool_access.js';
import { projects } from './projects.js';
import { projectWorkspaces } from './project_workspaces.js';
export const companySkillSources = pgTable('company_skill_sources', {
id: uuid('id').primaryKey().defaultRandom(),
@@ -11,22 +13,65 @@ export const companySkillSources = pgTable('company_skill_sources', {
repositoryUrl: text('repository_url').notNull(),
fullName: text('full_name').notNull(),
trackingRef: text('tracking_ref').notNull(),
connectionId: uuid('connection_id').references(() => toolConnections.id, { onDelete: 'set null' }),
// Retain the selected identity after deletion so refresh requires repair.
connectionId: uuid('connection_id'),
packageMode: text('package_mode').$type<'skills' | 'repository'>().notNull().default('skills'),
repositoryPackage: jsonb('repository_package').$type<SkillRepositoryPackage>(),
excludedFolders: jsonb('excluded_folders').$type<string[]>().notNull().default([]),
enabled: boolean('enabled').notNull().default(true),
// Removed project sources retain access provenance for editable copies and forks.
removedAt: timestamp('removed_at', { withTimezone: true }),
automaticUpdates: boolean('automatic_updates').notNull().default(true),
importNewSkills: boolean('import_new_skills').notNull().default(false),
accessScope: text('access_scope').$type<'company' | 'projects'>().notNull().default('company'),
refreshUserId: text('refresh_user_id'),
refreshAgentId: uuid('refresh_agent_id'),
refreshLocalTrusted: boolean('refresh_local_trusted').notNull().default(false),
refreshGrantId: uuid('refresh_grant_id'),
nextSyncAt: timestamp('next_sync_at', { withTimezone: true }).notNull().defaultNow(),
failureCount: integer('failure_count').notNull().default(0),
revision: integer('revision').notNull().default(0),
lastAttemptAt: timestamp('last_attempt_at', { withTimezone: true }),
lastSuccessAt: timestamp('last_success_at', { withTimezone: true }),
lastScanCommit: text('last_scan_commit'),
lastScanTreeSha: text('last_scan_tree_sha'),
automaticBudgetBytes: integer('automatic_budget_bytes').notNull().default(512 * 1024 * 1024),
automaticBudgetAt: timestamp('automatic_budget_at', { withTimezone: true }).notNull().defaultNow(),
lastError: text('last_error'),
leaseToken: uuid('lease_token'),
leaseExpiresAt: timestamp('lease_expires_at', { withTimezone: true }),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
}, table => ({
repositoryRef: uniqueIndex('company_skill_sources_repository_ref_idx').on(table.companyId, table.repositoryUrl, table.trackingRef),
repositoryIdentity: uniqueIndex('company_skill_sources_identity_ref_idx').on(table.companyId, table.repositoryId, table.trackingRef),
repositoryRef: uniqueIndex('company_skill_sources_repository_ref_idx').on(table.companyId, table.repositoryUrl, table.trackingRef).where(sql`${table.removedAt} is null`),
repositoryIdentity: uniqueIndex('company_skill_sources_identity_ref_idx').on(table.companyId, table.repositoryId, table.trackingRef).where(sql`${table.removedAt} is null`),
}));
/** Durable attachment jobs and retained project access provenance. */
export const projectSkillSources = pgTable('project_skill_sources', {
id: uuid('id').primaryKey().defaultRandom(),
companyId: uuid('company_id').notNull().references(() => companies.id, { onDelete: 'cascade' }),
projectId: uuid('project_id').notNull().references(() => projects.id, { onDelete: 'cascade' }),
workspaceId: uuid('workspace_id').references(() => projectWorkspaces.id, { onDelete: 'set null' }),
sourceId: uuid('source_id').references(() => companySkillSources.id, { onDelete: 'set null' }),
repositoryUrl: text('repository_url').notNull(),
trackingRef: text('tracking_ref').notNull(),
active: boolean('active').notNull().default(true),
status: text('status').$type<'pending' | 'ready' | 'error'>().notNull().default('pending'),
refreshUserId: text('refresh_user_id'),
refreshAgentId: uuid('refresh_agent_id'),
refreshLocalTrusted: boolean('refresh_local_trusted').notNull().default(false),
connectionId: uuid('connection_id'),
refreshGrantId: uuid('refresh_grant_id'),
nextSyncAt: timestamp('next_sync_at', { withTimezone: true }).notNull().defaultNow(),
failureCount: integer('failure_count').notNull().default(0),
lastError: text('last_error'),
leaseToken: uuid('lease_token'),
leaseExpiresAt: timestamp('lease_expires_at', { withTimezone: true }),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
}, table => ({
workspaceRepository: uniqueIndex('project_skill_sources_workspace_repo_ref_idx').on(table.workspaceId, table.repositoryUrl, table.trackingRef),
sourceProject: index('project_skill_sources_source_project_idx').on(table.companyId, table.sourceId, table.projectId),
pending: index('project_skill_sources_sync_idx').on(table.active, table.status, table.nextSyncAt),
}));
export const companySkillSourceEntries = pgTable('company_skill_source_entries', {
id: uuid('id').primaryKey().defaultRandom(),
@@ -44,3 +89,10 @@ export const companySkillSourceEntries = pgTable('company_skill_source_entries',
sourcePath: uniqueIndex('company_skill_source_entries_source_path_idx').on(table.sourceId, table.path),
companySkill: index('company_skill_source_entries_company_skill_idx').on(table.companyId, table.skillId),
}));
/** Sustained company credits for automatic initial imports, including failures. */
export const companySkillImportBudgets = pgTable('company_skill_import_budgets', {
companyId: uuid('company_id').primaryKey().references(() => companies.id, { onDelete: 'cascade' }),
availableBytes: integer('available_bytes').notNull().default(512 * 1024 * 1024),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
});
packages/db/src/schema/company_skills.ts
+2
View File
@@ -40,6 +40,8 @@ export const companySkills = pgTable(
homepageUrl: text("homepage_url"),
categories: text("categories").array().notNull().default([]),
sharingScope: text("sharing_scope").$type<CompanySkillSharingScope>().notNull().default("company"),
// Source access is inherited by editable copies too. Orphaned sources fail closed.
accessSourceId: uuid("access_source_id"),
publicShareToken: text("public_share_token"),
forkedFromSkillId: uuid("forked_from_skill_id").references((): AnyPgColumn => companySkills.id, { onDelete: "set null" }),
forkedFromCompanyId: uuid("forked_from_company_id").references(() => companies.id, { onDelete: "set null" }),
packages/shared/src/types/company-skill.ts
+2
View File
@@ -55,6 +55,7 @@ export interface CompanySkill {
authorName: string | null;
homepageUrl: string | null;
categories: string[];
accessSourceId?: string | null;
sharingScope: CompanySkillSharingScope;
publicShareToken: string | null;
forkedFromSkillId: string | null;
@@ -89,6 +90,7 @@ export interface CompanySkillListItem {
authorName: string | null;
homepageUrl: string | null;
categories: string[];
accessSourceId?: string | null;
sharingScope: CompanySkillSharingScope;
publicShareToken: string | null;
forkedFromSkillId: string | null;
packages/shared/src/types/skill-source.ts
+24
View File
@@ -67,6 +67,12 @@ export interface SkillSource {
connectionId: string | null;
excludedFolders: string[];
enabled: boolean;
automaticUpdates?: boolean;
importNewSkills?: boolean;
accessScope?: 'company' | 'projects';
trackingActive?: boolean;
syncState?: 'pending' | 'refreshing' | 'ready' | 'error' | 'paused';
projects?: Array<{ id: string; name: string; visibility: 'open' | 'private'; active: boolean }>;
revision: number;
lastAttemptAt: Date | null;
lastSuccessAt: Date | null;
@@ -74,6 +80,24 @@ export interface SkillSource {
lastError: string | null;
entries: SkillSourceEntry[];
}
export interface SkillSourceSettingsRequest {
revision: number;
automaticUpdates?: boolean;
importNewSkills?: boolean;
connectionId?: string | null;
}
export interface ProjectSkillRepository {
projectId: string;
projectName: string;
projectVisibility: 'open' | 'private';
workspaceId: string;
repositoryUrl: string;
trackingRef: string;
bindingId: string | null;
sourceId: string | null;
status: 'available' | 'pending' | 'ready' | 'error';
lastError: string | null;
}
export interface SkillSourceDiscoveryRequest {
packageMode?: SkillSourcePackageMode;
repositoryUrl: string;
packages/shared/src/validators/skill-source.ts
+9
View File
@@ -25,6 +25,15 @@ export const skillSourceSelectionSchema = z.object({
excludedFolders: z.array(repoPath).max(10000),
connectionId: z.string().uuid().nullable().optional(),
});
export const skillSourceSettingsSchema = z.object({
revision: z.number().int().nonnegative(),
automaticUpdates: z.boolean().optional(),
importNewSkills: z.boolean().optional(),
connectionId: z.string().uuid().nullable().optional(),
});
export const projectSkillSourceAdoptSchema = z.object({
workspaceId: z.string().uuid(),
});
export const skillSourceDisconnectSchema = z.object({
revision: z.number().int().nonnegative(),
server/src/__tests__/chat-channels.integration.test.ts
+10 -1
View File
@@ -2422,7 +2422,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
const saved = await f.management.configuration(f.endpoint.id, "owner-user");
await f.management.saveConfiguration(f.endpoint.id, { expectedRevision: saved.revision, configuration: {
...saved.configuration, defaults: { ...saved.configuration.defaults,
instructions: `Use ${ref(common)} and ${ref(foreign)}.`,
instructions: `Use ${ref(common)}.`,
prompts: { ...saved.configuration.defaults.prompts, mention: `Use ${ref(mention)}.`, opened: ref(injected) },
},
} }, "owner-user");
@@ -2452,6 +2452,15 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
expect(await resolveRunScopedMentionedSkillKeys(args)).toEqual([common.key, mention.key]);
expect(await resolveRunScopedMentionedSkillKeys({ ...args, commentIds: all.map((c) => c.id) })).toEqual(expect.arrayContaining([common.key, mention.key, next.key]));
expect(await resolveRunScopedMentionedSkillKeys({ ...args, commentIds: [] })).toEqual([next.key]);
const invalid = await f.management.configuration(f.endpoint.id, "owner-user");
await f.management.saveConfiguration(f.endpoint.id, { expectedRevision: invalid.revision, configuration: {
...invalid.configuration, defaults: { ...invalid.configuration.defaults, instructions: ref(foreign) },
} }, "owner-user");
// Instructions are captured on admission; changing configuration must
// not change the already admitted wake above.
await deliverMessage({ callbacks: f.callbacks, endpointId: f.endpoint.id, provider: "github", thread,
trigger: "subscribed_message", message: makeMessage({ id: "41903", text: 'Check again', userId: "42", userName: "octocat" }) });
await expect(resolveRunScopedMentionedSkillKeys({ ...args, commentIds: [] })).rejects.toThrow("referenced skill is unavailable");
});
it.each([
"github:paperclipai/paperclip:issue:419",
server/src/__tests__/codex-local-execute.test.ts
+122 -2
View File
@@ -1590,7 +1590,8 @@ process.exit(1);
await fs.mkdir(workspace, { recursive: true });
await fs.mkdir(sourceHome, { recursive: true });
await fs.mkdir(skillSource, { recursive: true });
await fs.writeFile(path.join(sourceHome, "auth.json"), fakeCodexAuthJson);
const selectedAuth = JSON.stringify({ tokens: { account_id: "original-selected-account", access_token: "test-subscription-token" } });
await fs.writeFile(path.join(sourceHome, "auth.json"), selectedAuth);
await fs.writeFile(path.join(skillSource, "SKILL.md"), "# AgentMail\nAssigned inbox one.");
await writeFakeCodexCommand(command);
const keys = ["PAPERCLIP_HOME", "PAPERCLIP_INSTANCE_ID", "CODEX_HOME"] as const;
@@ -1634,8 +1635,127 @@ process.exit(1);
await expect(fs.stat(path.join(sourceHome, "skills/agentmail"))).rejects.toMatchObject({ code: "ENOENT" });
const removed = await invoke("agent-1", null);
expect(removed.prompt).not.toContain("inline@example.test");
expect(removed.codexHome).toBe(sourceHome);
expect(removed.codexHome).not.toBe(sourceHome);
expect(removed.codexHome).toContain("skill-runtimes/agent-1/");
await expect(fs.stat(path.join(removed.codexHome!, "skills/agentmail"))).rejects.toMatchObject({ code: "ENOENT" });
// Remote credential copy-back leaves a regular file. A later authorized
// account switch must not reuse that previous account's generated home.
await fs.unlink(path.join(first.codexHome!, "auth.json"));
await fs.writeFile(path.join(first.codexHome!, "auth.json"), selectedAuth);
const nextAccount = JSON.parse(selectedAuth);
nextAccount.tokens.account_id = "new-selected-account";
await fs.writeFile(path.join(sourceHome, "auth.json"), JSON.stringify(nextAccount));
const switched = await invoke("agent-1", "a".repeat(64));
expect(switched.codexHome).not.toBe(first.codexHome);
expect(JSON.parse(await fs.readFile(path.join(switched.codexHome!, "auth.json"), "utf8")).tokens.account_id).toBe("new-selected-account");
nextAccount.tokens.access_token = "refreshed-same-account-token";
await fs.writeFile(path.join(sourceHome, "auth.json"), JSON.stringify(nextAccount));
expect((await invoke("agent-1", "a".repeat(64))).codexHome).toBe(switched.codexHome);
expect(await fs.readFile(path.join(first.codexHome!, "auth.json"), "utf8")).toBe(selectedAuth);
} finally {
for (const key of keys) { if (previous[key] === undefined) delete process.env[key]; else process.env[key] = previous[key]; }
await fs.rm(root, { recursive: true, force: true });
}
});
it.each([false, true])("isolates private skills and keeps custom routing with an explicit credential home: %s", async (explicitHome) => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-private-codex-"));
const workspace = path.join(root, "workspace");
const command = path.join(root, "codex");
const capture = path.join(root, "capture.json");
const sourceHome = path.join(root, "selected-account");
const skillSource = path.join(root, "skill-v1");
await fs.mkdir(workspace, { recursive: true });
await fs.mkdir(sourceHome, { recursive: true });
await fs.mkdir(skillSource, { recursive: true });
await fs.writeFile(path.join(sourceHome, "auth.json"), fakeCodexAuthJson);
await fs.writeFile(path.join(skillSource, "SKILL.md"), "# AgentMail\nAssigned inbox one.");
await writeFakeCodexCommand(command);
const keys = ["PAPERCLIP_HOME", "PAPERCLIP_INSTANCE_ID", "CODEX_HOME"] as const;
const previous = Object.fromEntries(keys.map((key) => [key, process.env[key]]));
process.env.PAPERCLIP_HOME = path.join(root, "paperclip");
process.env.PAPERCLIP_INSTANCE_ID = "connectors";
process.env.CODEX_HOME = sourceHome;
const invoke = async (agentId: string, digest: string | null, source = skillSource, connectorSkillInstructions = "") => {
const config = {
engine: "cli", command, cwd: workspace,
env: { ...(explicitHome ? { CODEX_HOME: sourceHome } : {}), PAPERCLIP_TEST_CAPTURE_PATH: capture,
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({ providers: { test_gateway: {
name: "Fixture gateway", base_url: "https://gateway.example/v1", wire_api: "responses",
} }, model_provider: "test_gateway" }),
},
paperclipSkillSync: { desiredSkills: digest ? ["paperclipai/paperclip/agentmail"] : [] },
paperclipRuntimeSkills: digest ? [{ key: "paperclipai/paperclip/agentmail", runtimeName: "agentmail", source }] : [],
};
const result = await execute({ runId: `run-${agentId}-${digest?.slice(0, 1) ?? "none"}`,
agent: { id: agentId, companyId: "company-1", name: "Email agent", adapterType: "codex_local", adapterConfig: config },
runtime: { sessionId: null, sessionParams: null, sessionDisplayId: null, taskKey: null },
config, context: { paperclipWake: { connectorSkillInstructions } }, authToken: "test-token", onLog: async () => {},
});
expect(result.errorMessage).toBeNull();
expect(result.exitCode).toBe(0);
return JSON.parse(await fs.readFile(capture, "utf8")) as CapturePayload;
};
try {
const first = await invoke("agent-1", "a".repeat(64));
expect(first.codexHome).toContain("skill-runtimes/agent-1/");
expect(first.codexConfigContents).toContain('model_provider = "test_gateway"');
expect(first.codexConfigContents).toContain('base_url = "https://gateway.example/v1"');
expect(await fs.realpath(path.join(first.codexHome!, "auth.json"))).toBe(await fs.realpath(path.join(sourceHome, "auth.json")));
expect(await fs.readFile(path.join(first.codexHome!, "skills/agentmail/SKILL.md"), "utf8")).toContain("inbox one");
await expect(fs.stat(path.join(sourceHome, "skills/agentmail"))).rejects.toMatchObject({ code: "ENOENT" });
const unchanged = await invoke("agent-1", "a".repeat(64));
expect(unchanged.codexHome).toBe(first.codexHome);
const other = await invoke("agent-2", "a".repeat(64));
expect(other.codexHome).not.toBe(first.codexHome);
const nextSource = path.join(root, "skill-v2");
await fs.mkdir(nextSource);
await fs.writeFile(path.join(nextSource, "SKILL.md"), "# AgentMail\nAssigned inbox two.");
const next = await invoke("agent-1", "b".repeat(64), nextSource);
expect(next.codexHome).not.toBe(first.codexHome);
expect(await fs.readFile(path.join(next.codexHome!, "skills/agentmail/SKILL.md"), "utf8")).toContain("inbox two");
const inline = await invoke("agent-1", null, skillSource, "# AgentMail\nAssigned inbox inline@example.test");
expect(inline.prompt).toContain("Assigned inbox inline@example.test");
await expect(fs.stat(path.join(sourceHome, "skills/agentmail"))).rejects.toMatchObject({ code: "ENOENT" });
const removed = await invoke("agent-1", null);
expect(removed.prompt).not.toContain("inline@example.test");
// The old saved files still exist; exclusion is enforced by the launch home.
expect(await fs.readFile(path.join(first.codexHome!, "skills/agentmail/SKILL.md"), "utf8")).toContain("inbox one");
expect(removed.codexHome).not.toBe(first.codexHome);
expect(removed.codexConfigContents).toContain('model_provider = "test_gateway"');
expect(await fs.realpath(path.join(removed.codexHome!, "auth.json"))).toBe(await fs.realpath(path.join(sourceHome, "auth.json")));
expect(removed.codexHome).not.toBe(sourceHome);
expect(removed.codexHome).toContain("skill-runtimes/agent-1/");
await expect(fs.stat(path.join(removed.codexHome!, "skills/agentmail"))).rejects.toMatchObject({ code: "ENOENT" });
if (!explicitHome) {
// A promoted company login can differ from the host's default account.
// Digest and seeding must use that same effective credential source.
const companyHome = path.resolve(first.codexHome!, "../../..");
await fs.unlink(path.join(companyHome, "auth.json"));
const originalAccount = JSON.stringify({ tokens: { account_id: "company-account-one", access_token: "test-token-one" } });
await fs.writeFile(path.join(companyHome, "auth.json"), originalAccount);
const companyRun = await invoke("agent-1", "a".repeat(64));
expect(JSON.parse(await fs.readFile(path.join(companyRun.codexHome!, "auth.json"), "utf8")).tokens.account_id).toBe("company-account-one");
await fs.unlink(path.join(companyRun.codexHome!, "auth.json"));
await fs.writeFile(path.join(companyRun.codexHome!, "auth.json"), originalAccount);
await fs.writeFile(path.join(companyHome, "auth.json"), JSON.stringify({ tokens: { account_id: "company-account-two", access_token: "test-token-two" } }));
const companyNext = await invoke("agent-1", "a".repeat(64));
expect(companyNext.codexHome).not.toBe(companyRun.codexHome);
expect(JSON.parse(await fs.readFile(path.join(companyNext.codexHome!, "auth.json"), "utf8")).tokens.account_id).toBe("company-account-two");
expect(await fs.readFile(path.join(sourceHome, "auth.json"), "utf8")).toBe(fakeCodexAuthJson);
} else {
await fs.writeFile(path.join(sourceHome, "config.toml"), '[mcp_servers.changed]\ncommand = "new-selected-server"\n');
const settingsChanged = await invoke("agent-1", "a".repeat(64));
expect(settingsChanged.codexHome).not.toBe(first.codexHome);
expect(settingsChanged.codexConfigContents).toContain('command = "new-selected-server"');
expect(settingsChanged.codexConfigContents).toContain('model_provider = "test_gateway"');
await fs.unlink(path.join(sourceHome, "config.toml"));
const settingsRemoved = await invoke("agent-1", "a".repeat(64));
expect(settingsRemoved.codexConfigContents).not.toContain('new-selected-server');
}
} finally {
for (const key of keys) { if (previous[key] === undefined) delete process.env[key]; else process.env[key] = previous[key]; }
await fs.rm(root, { recursive: true, force: true });
server/src/__tests__/company-skills-routes.test.ts
+43
View File
@@ -3,6 +3,8 @@ import request from "supertest";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { hoistModuleGraph } from "./helpers/hoist-module-graph.js";
const mockCaptureRunIdentity = vi.hoisted(() => vi.fn());
const mockSkillSourceService = vi.hoisted(() => ({ sourceForSkill: vi.fn(), importFromUrl: vi.fn(), discover: vi.fn() }));
const mockAgentService = vi.hoisted(() => ({
@@ -133,6 +135,10 @@ function denySkillPolicy(action = "skills.import") {
}
function registerModuleMocks() {
vi.doMock("../services/run-identity.js", async () => ({
...await vi.importActual<typeof import("../services/run-identity.js")>("../services/run-identity.js"),
captureRunIdentity: mockCaptureRunIdentity,
}));
vi.doMock("../services/skill-sources.js", () => ({ skillSourceService: () => mockSkillSourceService }));
vi.doMock("../routes/authz.js", async () => vi.importActual("../routes/authz.js"));
@@ -159,6 +165,15 @@ function registerModuleMocks() {
agentService: () => mockAgentService,
}));
// This route-contract harness supplies no database. Private-source decisions
// are exercised with the real database in project-skill-sources.test.ts.
vi.doMock("../services/skill-access.js", async () => {
const actual = await vi.importActual<typeof import("../services/skill-access.js")>("../services/skill-access.js");
return { ...actual, skillAccessService: (db: Parameters<typeof actual.skillAccessService>[0]) => ({
...actual.skillAccessService(db), assertSource: async () => {},
}) };
});
vi.doMock("../services/company-skills.js", async () => {
const actual = await vi.importActual<typeof import("../services/company-skills.js")>(
"../services/company-skills.js",
@@ -295,6 +310,7 @@ describe("company skill mutation permissions", () => {
beforeEach(() => {
vi.clearAllMocks();
mockCaptureRunIdentity.mockResolvedValue({ run: { responsibleUserId: "validated-owner" } });
mockSkillSourceService.sourceForSkill.mockResolvedValue(null);
mockSkillSourceService.importFromUrl.mockImplementation((companyId, source) => mockCompanySkillService.importFromSource(companyId, source));
mockGetTelemetryClient.mockReturnValue({ track: vi.fn() });
@@ -1232,6 +1248,33 @@ describe("company skill mutation permissions", () => {
});
});
it.each(["responsible-person", "local-board"])("does not persist API-key refresh authority from %s", async (onBehalfOfUserId) => {
const agent = { ...assignedAgentActor(), source: "agent_key", runId: undefined, onBehalfOfUserId,
onBehalfOfMemberships: [{ companyId: "company-1", membershipRole: "member", status: "active" }] };
const response = await request(createApp(agent)).post("/api/companies/company-1/skills/import")
.send({ source: "https://github.com/acme/public-skills" });
expect(response.status, JSON.stringify(response.body)).toBe(201);
expect(mockSkillSourceService.importFromUrl).toHaveBeenCalledWith("company-1", expect.anything(),
expect.objectContaining({ setup: { userId: null, agentId: agent.agentId, localTrusted: false } }));
});
it("derives refresh authority from the authenticated run instead of caller responsibility", async () => {
const agent = { ...assignedAgentActor(), source: "agent_jwt", onBehalfOfUserId: "unverified-person",
onBehalfOfMemberships: [{ companyId: "company-1", membershipRole: "member", status: "active" }] };
const response = await request(createApp(agent)).post("/api/companies/company-1/skills/import")
.send({ source: "https://github.com/acme/public-skills" });
expect(response.status, JSON.stringify(response.body)).toBe(201);
expect(mockCaptureRunIdentity).toHaveBeenCalledWith(expect.anything(), { companyId: "company-1", agentId: agent.agentId, runId: "run-1" });
expect(mockSkillSourceService.importFromUrl).toHaveBeenCalledWith("company-1", expect.anything(),
expect.objectContaining({ setup: { userId: "validated-owner", agentId: agent.agentId, localTrusted: false } }));
});
it("rejects a run-authenticated import without a run identity", async () => {
await request(createApp({ ...assignedAgentActor(), source: "agent_jwt", runId: undefined }))
.post("/api/companies/company-1/skills/import").send({ source: "https://github.com/acme/public-skills" }).expect(403);
expect(mockSkillSourceService.importFromUrl).not.toHaveBeenCalled();
});
it('streams discovery updates and a final result while retaining JSON compatibility', async () => {
const result = { candidates: [], commitSha: 'a'.repeat(40) };
mockSkillSourceService.discover.mockImplementation(async (_input, _context, options) => {
server/src/__tests__/github-skill-source.test.ts
+26 -1
View File
@@ -6,13 +6,38 @@ import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { promisify } from 'node:util';
import { describe, expect, it, vi } from 'vitest';
import { scanGitHubSkills, previewGitHubSkillFile, parseSkillRepository } from '../services/github-skill-source.js';
import { scanGitHubSkills, scanGitHubSkillsWithPackageFallback, rejectIncompleteSkillPackages, previewGitHubSkillFile, parseSkillRepository } from '../services/github-skill-source.js';
import { skillSourceDiscoverySchema, skillSourcePreviewSchema, type SkillSourceScanUpdate } from '@paperclipai/shared';
import { skillFileBytes } from '../services/skill-snapshot.js';
const sha = 'a'.repeat(40);
const md = (name: string) => `---\nname: ${name}\ndescription: A useful skill\n---\nFollow these instructions.\n`;
describe('GitHub skill repository discovery', () => {
it('keeps bounded packages discoverable but rejects dependencies omitted by repository fallback', async () => {
const fixture = githubFixture({
'skills/main/SKILL.md': md('main') + '\nRead [shared](../../shared/guide.md) and [helper](../helper/SKILL.md).',
'skills/helper/SKILL.md': md('helper'),
'shared/guide.md': 'Shared instructions',
...Object.fromEntries(Array.from({ length: 62 }, (_, i) => [`generated/${i}.bin`, 'x'])),
});
const open = fixture.openSnapshot;
fixture.openSnapshot = async (...args) => {
const snapshot = await open(...args);
for (const entry of snapshot.entries) if (entry.path.startsWith('generated/')) entry.size = 1024 * 1024;
return snapshot;
};
const scan = await scanGitHubSkillsWithPackageFallback({ repositoryUrl: 'https://github.com/acme/skills', packageMode: 'repository' }, fixture);
expect(scan.packageMode).toBe('skills');
expect(scan.skills.find(skill => skill.name === 'helper')?.error).toBeNull();
expect(scan.skills.find(skill => skill.name === 'main')?.error).toContain('dependencies could not be preserved');
expect(scan.candidates.find(skill => skill.name === 'main')?.error).toContain('dependencies could not be preserved');
const later = await scanGitHubSkills({ repositoryUrl: 'https://github.com/acme/skills', packageMode: 'skills' }, fixture);
expect(later.skills.find(skill => skill.name === 'main')?.error).toBeNull();
rejectIncompleteSkillPackages(later);
expect(later.candidates.find(skill => skill.name === 'main')?.error).toContain('dependencies could not be preserved');
expect(later.skills.find(skill => skill.name === 'helper')?.error).toBeNull();
});
it('resolves moving branches before cache lookup and reuses the same pinned commit until the branch changes', async () => {
const fixture = githubFixture({ 'SKILL.md': md('one') });
const read = vi.fn(fixture) as unknown as typeof fixture;
server/src/__tests__/heartbeat-execution-configuration.test.ts
+43 -2
View File
@@ -1,9 +1,11 @@
import { randomUUID } from "node:crypto";
import { access, readFile } from "node:fs/promises";
import { access, readFile, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { eq } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, companies, companyMemberships, createDb, heartbeatRunEvents, heartbeatRuns, issues, projects } from "@paperclipai/db";
import { agents, companies, companyMemberships, companySkills, createDb, heartbeatRunEvents, heartbeatRuns, issues, projects } from "@paperclipai/db";
import { companySkillService } from "../services/company-skills.js";
import { registerServerAdapter, unregisterServerAdapter, type AdapterExecutionContext } from "../adapters/index.js";
import { HttpError } from "../errors.js";
import { heartbeatService } from "../services/heartbeat.js";
@@ -37,10 +39,14 @@ describePostgres("heartbeat execution configuration through adapter dispatch", (
const captured = new Map<string, AdapterExecutionContext>();
const inspect = new Map<string, (input: AdapterExecutionContext) => Promise<void>>();
const adapterType = "codex_local";
const oldHome = process.env.PAPERCLIP_HOME;
let home: string;
beforeAll(async () => {
temporary = await startEmbeddedPostgresTestDatabase("heartbeat-execution-config-");
db = createDb(temporary.connectionString);
home = await mkdtemp(path.join(tmpdir(), "paperclip-execution-skills-"));
process.env.PAPERCLIP_HOME = home;
heartbeat = heartbeatService(db);
// Exercise real preparation for a managed-capable adapter without calling a provider.
registerServerAdapter({
@@ -60,6 +66,8 @@ describePostgres("heartbeat execution configuration through adapter dispatch", (
unregisterServerAdapter(adapterType);
await db?.$client.end({ timeout: 0 });
await temporary?.cleanup();
if (oldHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = oldHome;
if (home) await rm(home, { recursive: true, force: true });
});
async function seed(managed = false) {
@@ -157,4 +165,37 @@ describePostgres("heartbeat execution configuration through adapter dispatch", (
const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agentId));
expect(runs.filter(run => run.id !== finished.id)).toEqual([expect.objectContaining({ status: "scheduled_retry", scheduledRetryReason: "ai_connection_busy" })]);
});
it("snapshots only selected immutable skills and omits inaccessible optional assignments", async () => {
const f = await seed();
const skills = companySkillService(db);
const selected = await skills.createLocalSkill(f.companyId, { name: "Selected workflow", markdown: "# Selected\nUse the original workflow.\n" });
const unused = await skills.createLocalSkill(f.companyId, { name: "Unused workflow" });
const restricted = await skills.createLocalSkill(f.companyId, { name: "Restricted workflow" });
await db.update(companySkills).set({ accessSourceId: randomUUID() }).where(eq(companySkills.id, restricted.id));
await db.update(agents).set({ adapterConfig: { paperclipSkillSync: { desiredSkills: [restricted.key] } } }).where(eq(agents.id, f.agentId));
await db.update(issues).set({ description: "Use [/selected](skill://" + selected.id + "?s=selected-workflow)." }).where(eq(issues.id, f.issueId));
const { finished, input } = await execute(f);
expect(finished.status, finished.error ?? undefined).toBe("succeeded");
const inventory = input?.config.paperclipRuntimeSkills as Array<{ key: string; source: string; versionId: string }>;
expect(inventory.some(entry => entry.key === unused.key || entry.key === restricted.key)).toBe(false);
const delivered = inventory.find(entry => entry.key === selected.key)!;
expect(delivered.versionId).toBe(selected.currentVersionId);
expect(delivered.source).toContain(selected.currentVersionId!);
expect(await readFile(path.join(delivered.source, "SKILL.md"), "utf8")).toContain("Use the original workflow.");
expect(finished.contextSnapshot).toMatchObject({ skillSelectionSnapshot: true, skillVersions: { [selected.key]: selected.currentVersionId } });
expect(Object.keys(finished.contextSnapshot!.skillVersions as object)).not.toContain(unused.key);
expect(Object.keys(finished.contextSnapshot!.skillVersions as object)).not.toContain(restricted.key);
});
it("rejects an inaccessible explicit skill reference before adapter dispatch", async () => {
const f = await seed();
const skill = await companySkillService(db).createLocalSkill(f.companyId, { name: "Private reference" });
await db.update(companySkills).set({ accessSourceId: randomUUID() }).where(eq(companySkills.id, skill.id));
await db.update(issues).set({ description: "Use [/private](skill://" + skill.id + "?s=private-reference)." }).where(eq(issues.id, f.issueId));
const { finished, input } = await execute(f);
expect(finished.status).toBe("failed");
expect(finished.error).toContain("referenced skill is unavailable");
expect(input).toBeUndefined();
});
});
server/src/__tests__/heartbeat-runtime-skills.test.ts
+38 -2
View File
@@ -8,6 +8,8 @@ import {
agents,
companies,
companySkills,
projects,
projectWorkspaces,
createDb,
toolApplications,
toolConnectionInstalls,
@@ -110,9 +112,10 @@ describeEmbeddedPostgres("heartbeat runtime skill version pins", () => {
}, 20_000);
afterEach(async () => {
// Terminal status precedes trailing writes; drain them before truncating the fixture.
await heartbeatService(db).drainActiveRunExecutions();
capturedRuns.length = 0;
await instanceSettingsService(db).updateExperimental({ enableBetaSkills: false });
await new Promise((resolve) => setTimeout(resolve, 100));
await db.execute(sql.raw(`
TRUNCATE TABLE
"activity_log",
@@ -144,6 +147,39 @@ describeEmbeddedPostgres("heartbeat runtime skill version pins", () => {
await tempDb?.cleanup();
});
it.each(["local_path", "catalog"] as const)("snapshots changed installed %s files for new executions and keeps resume versions", async (sourceType) => {
const companyId = randomUUID();
await db.insert(companies).values({ id: companyId, name: 'Local skill snapshots', issuePrefix: 'LSK' });
const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'paperclip-local-snapshot-'));
cleanupDirs.add(directory);
const [project] = await db.insert(projects).values({ companyId, name: 'Approved local workspace' }).returning();
await db.insert(projectWorkspaces).values({ companyId, projectId: project!.id, name: 'Local skill workspace', sourceType: 'local_path', cwd: directory, isPrimary: true });
await fs.mkdir(path.join(directory, 'references'));
await fs.writeFile(path.join(directory, 'SKILL.md'), '---\nname: local-snapshot\ndescription: Read the installed reference.\n---\nRead references/marker.txt.\n');
await fs.writeFile(path.join(directory, 'references/marker.txt'), 'v1\n');
const skills = companySkillService(db);
const imported = await skills.importFromSource(companyId, directory);
const key = imported.imported[0]!.key;
if (sourceType === "catalog") await db.update(companySkills).set({ sourceType }).where(eq(companySkills.key, key));
const selected = { snapshotLatest: true, onlyKeys: new Set([key]) };
const [first] = await skills.listRuntimeSkillEntries(companyId, selected);
expect(first!.versionId).toBeTruthy();
expect(await fs.readFile(path.join(first!.source, 'references/marker.txt'), 'utf8')).toBe('v1\n');
// A reference-only change must be discovered even when SKILL.md is unchanged.
await fs.writeFile(path.join(directory, 'references/marker.txt'), 'v2\n');
if (sourceType === "local_path") await skills.importFromSource(companyId, directory);
const [latest] = await skills.listRuntimeSkillEntries(companyId, selected);
expect(latest!.versionId).not.toBe(first!.versionId);
expect(await fs.readFile(path.join(latest!.source, 'references/marker.txt'), 'utf8')).toBe('v2\n');
const [unchanged] = await skills.listRuntimeSkillEntries(companyId, selected);
expect(unchanged!.versionId).toBe(latest!.versionId);
const [resumed] = await skills.listRuntimeSkillEntries(companyId, { ...selected, versionSelections: new Map([[key, first!.versionId!]]) });
expect(resumed!.versionId).toBe(first!.versionId);
expect(await fs.readFile(path.join(resumed!.source, 'references/marker.txt'), 'utf8')).toBe('v1\n');
});
it("materializes different pinned skill versions for different agents at runtime", async () => {
const companyId = randomUUID();
const skillId = randomUUID();
@@ -292,7 +328,7 @@ describeEmbeddedPostgres("heartbeat runtime skill version pins", () => {
?.skills.find((entry) => entry.key === skillKey);
expect(defaultSkill).toMatchObject({
key: skillKey,
versionId: null,
versionId: versionTwo.id,
currentVersionId: versionTwo.id,
sourceStatus: "available",
});
server/src/__tests__/helpers/github-skills.ts
+4 -2
View File
@@ -1,15 +1,17 @@
import { createHash } from "node:crypto";
import type { GitHubRead } from "../../services/github-skill-source.js";
export function githubFixture(files: Record<string, string | Buffer>, modes: Record<string, string> = {}, commit = "a".repeat(40)): GitHubRead {
const entries = Object.keys(files).map((path, i) => ({ path, type: modes[path] === '160000' ? 'commit' : 'blob', mode: modes[path] ?? '100644', sha: String(i), size: Buffer.byteLength(files[path]!) }));
const treeSha = createHash("sha1").update(JSON.stringify(entries.map(entry => [entry.path, entry.mode, files[entry.path]]).sort())).digest("hex");
const read = (async (url: string) => {
if (url === '/repos/acme/skills') return { id: 42, full_name: 'acme/skills', default_branch: 'main' };
if (url.startsWith('/repos/acme/skills/commits/')) return { sha: commit };
if (url.startsWith('/repos/acme/skills/commits/')) return { sha: commit, commit: { tree: { sha: treeSha } } };
throw new Error(`Unexpected request ${url}`);
}) as GitHubRead;
read.openSnapshot = async (_input, options) => {
options?.signal?.throwIfAborted();
await options?.onDownload?.({ stage: 'receiving', percent: 50, receivedBytes: 1024 });
return { commitSha: commit, entries, readBlob: async entry => Buffer.from(files[entry.path]!), release: async () => {} };
return { commitSha: commit, treeSha, downloadBytes: entries.reduce((sum, entry) => sum + (entry.size ?? 0), 0), entries, readBlob: async entry => Buffer.from(files[entry.path]!), release: async () => {} };
};
return read;
}
server/src/__tests__/project-routes-env.test.ts
+10
View File
@@ -170,6 +170,16 @@ describe("project env routes", () => {
mockSecretService.normalizeEnvBindingsForPersistence.mockImplementation(async (_companyId, env) => env);
});
it('requires workspace-write authority to attach a repository despite project read access', async () => {
mockProjectService.getById.mockResolvedValue(buildProject());
mockAccessService.decide.mockImplementation(async (input) => ({ allowed: input.action !== 'project:write_workspace',
action: input.action, reason: 'deny_no_grant', explanation: 'Workspace write permission required.' }));
const app = await createApp();
await request(app).post('/api/projects/project-1/workspaces').send({ name: 'Repository', repoUrl: 'https://github.com/acme/skills' }).expect(403);
await request(app).put('/api/projects/project-1/repositories').send({ repositoryIds: ['42'] }).expect(403);
expect(mockProjectService.createWorkspace).not.toHaveBeenCalled();
});
it("normalizes env bindings on create and logs only env keys", async () => {
const normalizedEnv = {
API_KEY: {
server/src/__tests__/project-skill-setup-identity.test.ts
+25
View File
@@ -0,0 +1,25 @@
import { beforeEach, expect, it, vi } from 'vitest';
import type { Db } from '@paperclipai/db';
import { projectSkillSetupIdentity } from '../services/project-skill-source-bindings.js';
const { capture } = vi.hoisted(() => ({ capture: vi.fn() }));
vi.mock('../services/run-identity.js', () => ({ captureRunIdentity: capture }));
const db = {} as Db;
const actor = { type: 'agent' as const, source: 'agent_jwt' as const, companyId: 'company', agentId: 'agent', runId: 'run', onBehalfOfUserId: 'forged' };
beforeEach(() => capture.mockReset());
it('keeps responsibility without searching personal GitHub grants when the run has no selected credential', async () => {
capture.mockResolvedValue({ run: { responsibleUserId: 'person' }, context: { github: null } });
expect(await projectSkillSetupIdentity(db, 'company', actor)).toMatchObject({ refreshUserId: 'person', connectionId: null, refreshGrantId: null });
});
it('pins only the connection and grant already authorized for the active run', async () => {
capture.mockResolvedValue({ run: { responsibleUserId: 'person' }, context: { github: { status: 'available', connectionId: 'selected', grantId: 'grant' } } });
expect(await projectSkillSetupIdentity(db, 'company', actor)).toMatchObject({ refreshUserId: 'person', connectionId: 'selected', refreshGrantId: 'grant' });
expect(capture).toHaveBeenCalledWith(db, { companyId: 'company', agentId: 'agent', runId: 'run' });
});
it('does not reuse an unavailable run credential', async () => {
capture.mockResolvedValue({ run: { responsibleUserId: 'person' }, context: { github: { status: 'unavailable', connectionId: 'old', grantId: 'old' } } });
expect(await projectSkillSetupIdentity(db, 'company', actor)).toMatchObject({ connectionId: null, refreshGrantId: null });
});
it('API keys do not obtain responsibility or a run credential', async () => {
expect(await projectSkillSetupIdentity(db, 'company', { ...actor, source: 'agent_key' })).toMatchObject({ refreshUserId: null, connectionId: null, refreshGrantId: null });
expect(capture).not.toHaveBeenCalled();
});
server/src/__tests__/project-skill-sources.test.ts
+893
View File
@@ -0,0 +1,893 @@
import { reserveInitialSkillWork } from '../services/skill-source-import-budget.js';
import { admitDeferredSkillImports, lockSkillImportQueue, MAX_COMPANY_PENDING_SKILL_IMPORTS, MAX_PROJECT_PENDING_SKILL_IMPORTS, SKILL_IMPORT_CAPACITY_WAIT } from '../services/skill-source-import-queue.js';
import { randomUUID } from 'node:crypto';
import { promises as fs } from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { and, eq, sql } from 'drizzle-orm';
import { activityLog, agents, authUsers, companies, companyMemberships, companySkillSources, companySkillImportBudgets, companySkills, createDb, projects,
projectWorkspaces, projectSkillSources, projectAccessMembers } from '@paperclipai/db';
import { startEmbeddedPostgresTestDatabase, getEmbeddedPostgresTestSupport } from './helpers/embedded-postgres.js';
import { githubFixture } from './helpers/github-skills.js';
import { enqueueProjectSkillSource, detachProjectSkillSource, projectSkillSetupIdentity } from '../services/project-skill-source-bindings.js';
import { adoptProjectSkillRepository, projectSkillRepositories, skillSourceSyncService } from '../services/skill-source-sync.js';
import { skillSourceService, type SkillSourceContext } from '../services/skill-sources.js';
import { companySkillService } from '../services/company-skills.js';
import { projectService } from '../services/projects.js';
import { skillAccessService, withSkillActor } from '../services/skill-access.js';
import type { AuthorizationActor } from '../services/authorization.js';
import { forbidden, unprocessable } from '../errors.js';
const support = await getEmbeddedPostgresTestSupport();
describe.skipIf(!support.supported)('automatic project skill sources', () => {
let testDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let home: string;
const oldHome = process.env.PAPERCLIP_HOME;
const companyId = randomUUID();
const md = (name: string) => `---\nname: ${name}\ndescription: Test workflow\n---\nFollow the workflow.\n`;
let files: Record<string, string> = {};
let commit = 'a'.repeat(40);
const downloads = vi.fn();
const context: SkillSourceContext = { actor: { type: 'system' }, setup: { userId: null, localTrusted: true },
read: () => {
const fixture = githubFixture(files, {}, commit), open = fixture.openSnapshot;
fixture.openSnapshot = async (...args) => { downloads(); return open(...args); };
return fixture;
}, authorize: async () => {}, audit: async () => {} };
beforeAll(async () => {
testDb = await startEmbeddedPostgresTestDatabase('paperclip-project-skills-');
db = createDb(testDb.connectionString);
home = await fs.mkdtemp(path.join(os.tmpdir(), 'paperclip-project-skills-home-'));
process.env.PAPERCLIP_HOME = home;
await db.insert(companies).values({ id: companyId, name: 'Project skills', issuePrefix: 'PSK' });
}, 30000);
it('requires explicit repair before an agent-created repository can search a person’s GitHub grants', async () => {
const row = await workspace('agent-repair', 'open');
const binding = await enqueueProjectSkillSource(db, row);
const [agent] = await db.insert(agents).values({ companyId, name: 'Agent repository setup', adapterType: 'codex_local', status: 'idle' }).returning();
await db.update(projectSkillSources).set({ refreshUserId: 'local-board', refreshAgentId: agent!.id, refreshLocalTrusted: true }).where(eq(projectSkillSources.id, binding!.id));
const provider = vi.spyOn(globalThis, 'fetch');
try {
await skillSourceSyncService(db).tick();
const [saved] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, binding!.id));
expect(saved!.lastError).toMatch(/Choose a GitHub connection/);
expect(saved!.active).toBe(false);
expect(provider).not.toHaveBeenCalled();
} finally { provider.mockRestore(); }
});
afterAll(async () => {
if (oldHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = oldHome;
await testDb?.cleanup();
if (home) await fs.rm(home, { recursive: true, force: true });
});
async function workspace(ref: string, visibility = 'open') {
const [project] = await db.insert(projects).values({ companyId, name: ref, visibility }).returning();
const [row] = await db.insert(projectWorkspaces).values({ companyId, projectId: project!.id, name: ref,
repoUrl: 'https://github.com/acme/skills', defaultRef: ref }).returning();
return row!;
}
const actor = (userId: string): AuthorizationActor => ({ type: 'board', userId, source: 'session', companyIds: [companyId], ignoreInstanceAdmin: true });
it('bounds per-project pending imports and admits deferred work after capacity frees', async () => {
await db.update(projectSkillSources).set({ active: false, lastError: null });
const root = await workspace('quota-root');
const rows = [];
for (let index = 0; index <= MAX_PROJECT_PENDING_SKILL_IMPORTS; index++) {
const [row] = await db.insert(projectWorkspaces).values({ companyId, projectId: root.projectId, name: `Repository ${index}`,
repoUrl: root.repoUrl, defaultRef: `project-quota-${index}` }).returning();
rows.push(row!); await enqueueProjectSkillSource(db, row!);
}
const pending = await db.select().from(projectSkillSources).where(eq(projectSkillSources.projectId, root.projectId));
expect(pending.filter(row => row.active)).toHaveLength(MAX_PROJECT_PENDING_SKILL_IMPORTS);
expect(pending.filter(row => !row.active)).toEqual([expect.objectContaining({ status: 'pending', lastError: SKILL_IMPORT_CAPACITY_WAIT })]);
await detachProjectSkillSource(db, companyId, rows[0]!.id);
await admitDeferredSkillImports(db, companyId, new Date());
const [admitted] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, rows.at(-1)!.id));
expect(admitted).toMatchObject({ active: true, lastError: null });
const [extra] = await db.insert(projectWorkspaces).values({ companyId, projectId: root.projectId, name: 'Detached waiting repository', repoUrl: root.repoUrl, defaultRef: 'quota-detached' }).returning();
await enqueueProjectSkillSource(db, extra!);
await detachProjectSkillSource(db, companyId, extra!.id);
await detachProjectSkillSource(db, companyId, rows[1]!.id);
await admitDeferredSkillImports(db, companyId, new Date());
const [detached] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, extra!.id));
expect(detached!.active).toBe(false);
await db.update(projectSkillSources).set({ active: false, lastError: null });
});
it('does not reactivate a deferred import detached after admission selection', async () => {
const row = await workspace('deferred-removal-race');
const binding = await enqueueProjectSkillSource(db, row);
await db.update(projectSkillSources).set({ active: false, lastError: SKILL_IMPORT_CAPACITY_WAIT }).where(eq(projectSkillSources.id, binding!.id));
let detached = false;
const database = new Proxy(db, { get(target, key) {
if (key !== 'transaction') return Reflect.get(target, key);
return (callback: (tx: unknown) => Promise<unknown>) => target.transaction(async tx => callback(new Proxy(tx, { get(target, key) {
if (key !== 'update') return Reflect.get(target, key);
return (table: unknown) => {
const update = target.update(table as typeof projectSkillSources);
if (table !== projectSkillSources) return update;
return { set: (values: Parameters<typeof update.set>[0]) => {
const set = update.set(values);
return { where: async (condition: Parameters<typeof set.where>[0]) => {
if (!detached) { detached = true; await detachProjectSkillSource(db, companyId, row.id); }
return set.where(condition);
} };
} };
};
} })));
} });
await admitDeferredSkillImports(database, companyId, new Date());
const [saved] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, binding!.id));
expect(detached).toBe(true);
expect(saved!.active).toBe(false);
expect(saved!.lastError).toBeNull();
});
it('bounds the company pending queue across projects without losing deferred provenance', async () => {
const projectsToInsert = Array.from({ length: MAX_COMPANY_PENDING_SKILL_IMPORTS + 1 }, (_, index) => ({ id: randomUUID(), companyId, name: `Quota project ${index}` }));
await db.insert(projects).values(projectsToInsert);
const workspaces = await db.insert(projectWorkspaces).values(projectsToInsert.map((project, index) => ({ companyId, projectId: project.id,
name: 'Repository', repoUrl: 'https://github.com/acme/skills', defaultRef: `company-quota-${index}` }))).returning();
for (const row of workspaces) await enqueueProjectSkillSource(db, row);
const pending = await db.select().from(projectSkillSources).where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.active, true)));
expect(pending.filter(row => !row.sourceId)).toHaveLength(MAX_COMPANY_PENDING_SKILL_IMPORTS);
const deferred = await db.select().from(projectSkillSources).where(eq(projectSkillSources.lastError, SKILL_IMPORT_CAPACITY_WAIT));
expect(deferred).toHaveLength(1);
expect(deferred[0]!.workspaceId).toBeTruthy();
await db.update(projectSkillSources).set({ active: false, lastError: null });
});
it('admits another project beyond a full project’s oldest waiting batch', async () => {
const full = await workspace('full-admission-project');
const other = await workspace('other-admission-project');
const rows = await db.insert(projectWorkspaces).values(Array.from({ length: 120 }, (_, index) => ({ companyId, projectId: full.projectId,
name: `Queued ${index}`, repoUrl: full.repoUrl, defaultRef: `starvation-${index}` }))).returning();
await db.insert(projectSkillSources).values(rows.map((row, index) => ({ companyId, projectId: row.projectId, workspaceId: row.id,
repositoryUrl: row.repoUrl!, trackingRef: row.defaultRef!, active: index < 20,
lastError: index < 20 ? null : SKILL_IMPORT_CAPACITY_WAIT, createdAt: new Date(0) })));
const [waiting] = await db.insert(projectSkillSources).values({ companyId, projectId: other.projectId, workspaceId: other.id,
repositoryUrl: other.repoUrl!, trackingRef: other.defaultRef!, active: false, lastError: SKILL_IMPORT_CAPACITY_WAIT }).returning();
await admitDeferredSkillImports(db, companyId, new Date());
const [saved] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, waiting!.id));
expect(saved!.active).toBe(true);
await db.update(projectSkillSources).set({ active: false, lastError: null });
});
it('adoption waits for the company queue before taking repository binding locks', async () => {
const row = await workspace('adoption-lock-order');
const binding = await enqueueProjectSkillSource(db, row);
const [source] = await db.insert(companySkillSources).values({ companyId, repositoryUrl: row.repoUrl!, fullName: 'acme/skills', trackingRef: row.defaultRef! }).returning();
await db.update(projectSkillSources).set({ sourceId: source!.id }).where(eq(projectSkillSources.id, binding!.id));
let unlock!: () => void, locked!: () => void;
const release = new Promise<void>(resolve => { unlock = resolve; });
const ready = new Promise<void>(resolve => { locked = resolve; });
let editTx!: typeof db;
const edit = db.transaction(async tx => { editTx = tx as unknown as typeof db; await lockSkillImportQueue(editTx, companyId); locked(); await release; });
await ready;
const adoption = withSkillActor({ type: 'board', source: 'local_implicit', companyIds: [companyId] },
() => adoptProjectSkillRepository(db, companyId, row.id, context.authorize));
try {
await vi.waitFor(async () => {
const result = await db.execute(sql`select 1 from pg_stat_activity where wait_event_type = 'Lock' and query ilike '%companies%for no key update%'`);
expect(result.length).toBeGreaterThan(0);
});
// A repository edit holding the company lock must still be able to lock
// this binding. The old inverse adoption order fails with lock_not_available.
await editTx.execute(sql`select id from project_skill_sources where id = ${binding!.id} for update nowait`);
} finally { unlock(); await edit; await adoption; }
});
it('meters failed initial downloads across workers and retries before provider reads', async () => {
const budgetCompany = randomUUID();
await db.insert(companies).values({ id: budgetCompany, name: 'Initial work credits', issuePrefix: 'IWC' });
const [project] = await db.insert(projects).values({ companyId: budgetCompany, name: 'Repeated failed downloads' }).returning();
const rows = await db.insert(projectWorkspaces).values(Array.from({ length: 5 }, (_, index) => ({ companyId: budgetCompany,
projectId: project!.id, name: `Failed ${index}`, repoUrl: 'https://github.com/acme/skills', defaultRef: `failed-${index}` }))).returning();
for (const row of rows) await enqueueProjectSkillSource(db, row);
const reads = vi.fn(() => {
const reader = githubFixture({});
reader.openSnapshot = async () => { throw unprocessable('Git download exceeds 128 MiB.'); };
return reader;
});
const factory = async (setup: { companyId: string }) => setup.companyId === budgetCompany ? { ...context, read: reads } : context;
const now = new Date();
try {
await skillSourceSyncService(db, { context: factory }).tick(now);
expect(reads).toHaveBeenCalledTimes(4);
const [budget] = await db.select().from(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, budgetCompany));
expect(budget!.availableBytes).toBe(0);
const worker = skillSourceSyncService(db, { context: factory });
await worker.tick(new Date(now.getTime() + 5 * 60_000));
expect(reads).toHaveBeenCalledTimes(4);
const waiting = await db.select().from(projectSkillSources).where(eq(projectSkillSources.companyId, budgetCompany));
expect(waiting.every(row => row.active && row.status === 'error' && row.lastError?.includes('limited'))).toBe(true);
await worker.tick(new Date(now.getTime() + 15 * 60_000));
expect(reads).toHaveBeenCalledTimes(5);
} finally { await db.update(companies).set({ status: 'archived' }).where(eq(companies.id, budgetCompany)); }
});
it('refunds measured small initial imports while completed jobs keep a company budget', async () => {
const budgetCompany = randomUUID();
await db.insert(companies).values({ id: budgetCompany, name: 'Small initial imports', issuePrefix: 'SII' });
const [project] = await db.insert(projects).values({ companyId: budgetCompany, name: 'Small repositories' }).returning();
const rows = await db.insert(projectWorkspaces).values(Array.from({ length: 6 }, (_, index) => ({ companyId: budgetCompany,
projectId: project!.id, name: `Small ${index}`, repoUrl: 'https://github.com/acme/skills', defaultRef: `small-${index}` }))).returning();
for (const row of rows) await enqueueProjectSkillSource(db, row);
const opened = vi.fn();
const reads = vi.fn(() => {
const reader = githubFixture({}), open = reader.openSnapshot!;
reader.openSnapshot = async (...args) => { opened(); return open(...args); };
return reader;
});
try {
await skillSourceSyncService(db, { context: async setup => setup.companyId === budgetCompany ? { ...context, read: reads } : context }).tick();
const bindings = await db.select().from(projectSkillSources).where(eq(projectSkillSources.companyId, budgetCompany));
expect(bindings.every(row => row.status === 'ready' && row.sourceId)).toBe(true);
expect(opened).toHaveBeenCalledTimes(6);
const [budget] = await db.select().from(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, budgetCompany));
expect(budget!.availableBytes).toBeGreaterThan(500 * 1024 * 1024);
} finally { await db.update(companies).set({ status: 'archived' }).where(eq(companies.id, budgetCompany)); }
});
it('does not replenish initial work credits when large measured jobs complete', async () => {
const budgetCompany = randomUUID();
await db.insert(companies).values({ id: budgetCompany, name: 'Completed work credits', issuePrefix: 'CWC' });
const [project] = await db.insert(projects).values({ companyId: budgetCompany, name: 'Large measured imports' }).returning();
const rows = await db.insert(projectWorkspaces).values(Array.from({ length: 5 }, (_, index) => ({ companyId: budgetCompany,
projectId: project!.id, name: `Measured ${index}`, repoUrl: 'https://github.com/acme/skills', defaultRef: `measured-${index}` }))).returning();
for (const row of rows) await enqueueProjectSkillSource(db, row);
const opened = vi.fn();
const reads = vi.fn(() => {
const reader = githubFixture({}), open = reader.openSnapshot!;
reader.openSnapshot = async (...args) => { opened(); return { ...await open(...args), downloadBytes: 128 * 1024 * 1024 }; };
return reader;
});
try {
await skillSourceSyncService(db, { context: async setup => setup.companyId === budgetCompany ? { ...context, read: reads } : context }).tick();
const bindings = await db.select().from(projectSkillSources).where(eq(projectSkillSources.companyId, budgetCompany));
expect(bindings.filter(row => row.status === 'ready' && row.sourceId)).toHaveLength(4);
expect(bindings.filter(row => row.status === 'error' && row.lastError?.includes('limited'))).toHaveLength(1);
expect(opened).toHaveBeenCalledTimes(4);
const [budget] = await db.select().from(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, budgetCompany));
expect(budget!.availableBytes).toBe(0);
} finally { await db.update(companies).set({ status: 'archived' }).where(eq(companies.id, budgetCompany)); }
});
it('never counts refill time twice when an older sweep reserves after a newer one', async () => {
const row = await workspace('monotonic-initial-work');
const binding = await enqueueProjectSkillSource(db, row);
const token = randomUUID();
await db.update(projectSkillSources).set({ leaseToken: token }).where(eq(projectSkillSources.id, binding!.id));
await db.delete(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, companyId));
const older = new Date(), newer = new Date(older.getTime() + 15 * 60_000);
try {
await reserveInitialSkillWork(db, binding!, token, newer);
await reserveInitialSkillWork(db, binding!, token, older);
const [afterOlder] = await db.select().from(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, companyId));
expect(afterOlder!.updatedAt).toEqual(newer);
await reserveInitialSkillWork(db, binding!, token, newer);
const [saved] = await db.select().from(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, companyId));
expect(saved!.availableBytes).toBe(128 * 1024 * 1024);
} finally {
await detachProjectSkillSource(db, companyId, row.id);
await db.delete(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, companyId));
}
});
it('does not adopt a stale repository after an editor commits a new selection', async () => {
const row = await workspace('adoption-stale-selection');
await enqueueProjectSkillSource(db, row);
try {
await withSkillActor({ type: 'board', source: 'local_implicit', companyIds: [companyId] }, async () => {
await expect(adoptProjectSkillRepository(db, companyId, row.id, async () => {
await projectService(db).updateWorkspace(row.projectId, row.id, { repoUrl: 'https://github.com/acme/skills-next', defaultRef: 'edited-branch' });
})).rejects.toThrow(/repository changed/i);
});
const saved = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
expect(saved.find(binding => binding.trackingRef === 'edited-branch')!.active).toBe(true);
expect(saved.find(binding => binding.trackingRef === row.defaultRef)!.active).toBe(false);
} finally { await detachProjectSkillSource(db, companyId, row.id); }
});
it('does not read GitHub or retry an initial job denied by import policy', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
const row = await workspace('denied-preflight');
await enqueueProjectSkillSource(db, row);
const read = vi.fn(context.read);
const worker = skillSourceSyncService(db, { context: async () => ({ ...context, read,
authorize: async () => { throw forbidden('Skill import denied by policy.'); } }) });
expect((await worker.tick()).imported).toBe(0);
expect(read).not.toHaveBeenCalled();
const [blocked] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
expect(blocked).toMatchObject({ active: false, status: 'error', sourceId: null });
await worker.tick(new Date(Date.now() + 60 * 60_000));
expect(read).not.toHaveBeenCalled();
});
it('does not add readers when an import overlaps private source removal', async () => {
files = { 'race/SKILL.md': md('race workflow') };
const originalWorkspace = await workspace('removal-race', 'private');
await enqueueProjectSkillSource(db, originalWorkspace);
await skillSourceSyncService(db, { context: async () => context }).tick();
const [origin] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, originalWorkspace.id));
const sources = skillSourceService(db), skills = companySkillService(db);
const source = await sources.detail(companyId, origin!.sourceId!);
const skillId = source.entries[0]!.skillId!;
const pendingWorkspace = await workspace('removal-race', 'open');
await enqueueProjectSkillSource(db, pendingWorkspace);
let removed = false, authorizations = 0;
const racingContext = { ...context, authorize: async () => {
if (++authorizations !== 2 || removed) return;
removed = true;
await sources.disconnect(companyId, source.id, context, { revision: source.revision, skills: 'copy' });
await skills.updateFile(companyId, skillId, 'SKILL.md', md('Restricted race edits'));
} };
const worker = skillSourceSyncService(db, { context: async () => racingContext });
expect((await worker.tick()).imported).toBe(0);
const [pending] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, pendingWorkspace.id));
expect(pending).toMatchObject({ sourceId: null, status: 'error' });
const readerId = randomUUID();
await db.insert(authUsers).values({ id: readerId, name: 'Open reader', email: `${readerId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
await db.insert(companyMemberships).values({ companyId, principalType: 'user', principalId: readerId, membershipRole: 'member', status: 'active' });
await withSkillActor(actor(readerId), async () => {
expect(await skills.getById(companyId, skillId)).toBeNull();
expect(await skills.readFile(companyId, skillId, 'SKILL.md')).toBeNull();
});
expect((await worker.tick(new Date(Date.now() + 6 * 60_000))).imported).toBe(1);
const [retried] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, pendingWorkspace.id));
expect(retried!.sourceId).not.toBe(source.id);
});
it('imports after durable enqueue, imports later additions and avoids unchanged downloads', async () => {
files = { 'one/SKILL.md': md('one') }; commit = 'a'.repeat(40); downloads.mockClear();
const row = await workspace('automatic');
const binding = await enqueueProjectSkillSource(db, row);
await enqueueProjectSkillSource(db, row);
expect(await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id))).toHaveLength(1);
expect((await db.select().from(companySkillSources)).filter(source => source.trackingRef === 'automatic')).toHaveLength(0);
const worker = skillSourceSyncService(db, { context: async () => context });
expect((await worker.tick()).imported).toBe(1);
const [ready] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, binding!.id));
const sourceId = ready!.sourceId!;
const first = await skillSourceService(db).detail(companyId, sourceId);
expect(first.accessScope).toBe('projects');
expect(first.importNewSkills).toBe(true);
const installedId = first.entries[0]!.skillId!;
const original = (await companySkillService(db).getById(companyId, installedId))!;
expect(original.accessSourceId).toBe(sourceId);
const downloadCount = downloads.mock.calls.length;
await worker.tick(new Date(Date.now() + 6 * 60_000));
expect(downloads).toHaveBeenCalledTimes(downloadCount);
files['one/SKILL.md'] = md('one improved'); files['two/SKILL.md'] = md('two'); commit = 'b'.repeat(40);
await worker.tick(new Date(Date.now() + 6 * 60_000));
const updated = await skillSourceService(db).detail(companyId, sourceId);
expect(updated.entries.filter(entry => entry.skillId)).toHaveLength(2);
const latest = (await companySkillService(db).getById(companyId, installedId))!;
expect(latest.id).toBe(original.id);
expect(latest.currentVersionId).not.toBe(original.currentVersionId);
const pinned = await companySkillService(db).getVersion(companyId, installedId, original.currentVersionId!);
expect(pinned!.fileInventory.find(file => file.path === 'SKILL.md')!.content).toBe(md('one'));
await detachProjectSkillSource(db, companyId, row.id);
const before = downloads.mock.calls.length;
await worker.tick(new Date(Date.now() + 6 * 60_000));
expect(downloads).toHaveBeenCalledTimes(before);
expect(await companySkillService(db).getById(companyId, installedId)).not.toBeNull();
expect(await skillSourceService(db).detail(companyId, sourceId)).toMatchObject({ trackingActive: false, syncState: 'paused' });
});
it('watches empty repositories, retains folder exclusions and coalesces worker sweeps', async () => {
files = {}; commit = 'c'.repeat(40);
const row = await workspace('empty');
await enqueueProjectSkillSource(db, row);
const worker = skillSourceSyncService(db, { context: async () => context });
const first = worker.tick();
expect(worker.tick()).toBe(first);
await first;
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
expect(binding!.status).toBe('ready');
const service = skillSourceService(db);
const source = await service.detail(companyId, binding!.sourceId!);
await service.refresh(companyId, source.id, context, { revision: source.revision, selectedPaths: [], excludedFolders: ['excluded'] });
files = { 'excluded/secret/SKILL.md': md('excluded'), 'new/SKILL.md': md('new') }; commit = 'd'.repeat(40);
await worker.tick(new Date(Date.now() + 6 * 60_000));
const result = await service.detail(companyId, source.id);
expect(result.entries.find(entry => entry.path === 'new/SKILL.md')!.skillId).toBeTruthy();
expect(result.entries.find(entry => entry.path.includes('secret'))!.selection).toBe('excluded');
});
it('inherits either project access, rechecks revocation and fails closed after project deletion', async () => {
files = { 'private/SKILL.md': md('private workflow') }; commit = 'e'.repeat(40);
const row = await workspace('private', 'private');
await enqueueProjectSkillSource(db, row);
await skillSourceSyncService(db, { context: async () => context }).tick();
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
expect(binding!.status, binding!.lastError ?? 'initial private binding').toBe('ready');
const source = await skillSourceService(db).detail(companyId, binding!.sourceId!);
const skillId = source.entries[0]!.skillId!;
let privateSnapshotPath = '';
const privateDerivatives: string[] = [];
for (const user of ['alice', 'bob']) {
await db.insert(authUsers).values({ id: user, name: user, email: `${user}@example.test`, createdAt: new Date(), updatedAt: new Date() });
await db.insert(companyMemberships).values({ companyId, principalType: 'user', principalId: user, membershipRole: 'member', status: 'active' });
}
await db.insert(projectAccessMembers).values({ companyId, projectId: row.projectId, subjectType: 'user', subjectId: 'alice' });
await withSkillActor(actor('alice'), async () => {
expect(await companySkillService(db).getById(companyId, skillId)).not.toBeNull();
expect((await companySkillService(db).list(companyId)).some(skill => skill.id === skillId)).toBe(true);
const skill = (await companySkillService(db).getById(companyId, skillId))!;
const [runtime] = await companySkillService(db).listRuntimeSkillEntries(companyId, { onlyKeys: new Set([skill.key]) });
privateSnapshotPath = runtime!.source;
await expect(companySkillService(db).importFromSource(companyId, runtime!.source)).rejects.toThrow(/Project-derived skill/);
const fork = await companySkillService(db).forkSkill(companyId, skillId, { slug: 'private-copy' });
expect(fork.skill.accessSourceId).not.toBe(source.id);
expect(fork.skill.accessSourceId).toBeTruthy();
privateDerivatives.push(fork.skill.id);
await expect(companySkillService(db).updateSkill(companyId, fork.skill.id, { sharingScope: 'public_link' })).rejects.toThrow(/public link/);
const copy = await companySkillService(db).createLocalSkill(companyId, { name: 'Private derived copy', forkedFromSkillId: skillId });
expect(copy.accessSourceId).not.toBe(source.id);
expect(copy.accessSourceId).toBeTruthy();
privateDerivatives.push(copy.id);
await expect(companySkillService(db).createLocalSkill(companyId, { name: 'Unsafe publication', forkedFromSkillId: skillId, sharingScope: 'public_link' })).rejects.toThrow(/public link/);
});
await withSkillActor(actor('bob'), async () => {
await expect(companySkillService(db).importFromSource(companyId, privateSnapshotPath)).rejects.toThrow(/not found/);
expect(await companySkillService(db).getById(companyId, skillId)).toBeNull();
expect(await companySkillService(db).readFile(companyId, skillId, 'SKILL.md')).toBeNull();
expect((await skillSourceService(db).list(companyId)).some(item => item.id === source.id)).toBe(false);
await expect(skillAccessService(db).assertSkill(companyId, skillId)).rejects.toThrow(/not found/);
});
const [reader] = await db.insert(agents).values({ companyId, name: 'Private skill reader', adapterType: 'codex_local', status: 'idle' }).returning();
const [outsider] = await db.insert(agents).values({ companyId, name: 'Unassigned reader', adapterType: 'codex_local', status: 'idle' }).returning();
for (const agent of [reader!, outsider!]) await db.insert(companyMemberships).values({ companyId, principalType: 'agent', principalId: agent.id, membershipRole: 'member', status: 'active' });
await db.insert(projectAccessMembers).values({ companyId, projectId: row.projectId, subjectType: 'agent', subjectId: reader!.id });
const agentActor = (agentId: string, userId: string): AuthorizationActor => ({
type: 'agent', source: 'agent_jwt', companyId, agentId, runId: randomUUID(), onBehalfOfUserId: userId,
});
await withSkillActor(agentActor(reader!.id, 'alice'), async () => {
expect(await companySkillService(db).readFile(companyId, skillId, 'SKILL.md')).not.toBeNull();
const skill = (await companySkillService(db).getById(companyId, skillId))!;
expect(await companySkillService(db).getVersion(companyId, skillId, skill.currentVersionId!)).not.toBeNull();
expect((await companySkillService(db).listRuntimeSkillEntries(companyId)).some(entry => entry.key === skill.key)).toBe(true);
});
for (const denied of [agentActor(reader!.id, 'bob'), agentActor(outsider!.id, 'alice'), { ...agentActor(reader!.id, 'alice'), companyId: randomUUID() }]) {
await withSkillActor(denied, async () => {
expect(await companySkillService(db).readFile(companyId, skillId, 'SKILL.md')).toBeNull();
expect((await companySkillService(db).listRuntimeSkillEntries(companyId)).some(entry => entry.key.includes('private'))).toBe(false);
});
}
const second = await workspace('private', 'private');
await db.insert(projectAccessMembers).values({ companyId, projectId: second.projectId, subjectType: 'user', subjectId: 'bob' });
await enqueueProjectSkillSource(db, second);
await skillSourceSyncService(db, { context: async () => context }).tick();
const [linked] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, second.id));
expect(linked!.status, linked!.lastError ?? 'second project binding').toBe('ready');
expect(linked!.sourceId).toBe(source.id);
await withSkillActor(actor('bob'), async () => expect(await companySkillService(db).getById(companyId, skillId)).not.toBeNull());
await db.delete(projectAccessMembers).where(and(eq(projectAccessMembers.projectId, second.projectId), eq(projectAccessMembers.subjectId, 'bob')));
await withSkillActor(actor('bob'), async () => expect(await companySkillService(db).getById(companyId, skillId)).toBeNull());
const opened = await workspace('private', 'open');
await enqueueProjectSkillSource(db, opened);
await skillSourceSyncService(db, { context: async () => context }).tick();
await withSkillActor(actor('bob'), async () => {
expect(await companySkillService(db).getById(companyId, skillId)).not.toBeNull();
for (const derivativeId of privateDerivatives) {
expect(await companySkillService(db).getById(companyId, derivativeId)).toBeNull();
expect(await companySkillService(db).readFile(companyId, derivativeId, 'SKILL.md')).toBeNull();
}
});
await withSkillActor(agentActor(reader!.id, 'bob'), async () => {
expect(await companySkillService(db).getById(companyId, skillId)).not.toBeNull();
for (const derivativeId of privateDerivatives) expect(await companySkillService(db).getById(companyId, derivativeId)).toBeNull();
});
await db.delete(projects).where(eq(projects.id, row.projectId));
await db.delete(projects).where(eq(projects.id, second.projectId));
await withSkillActor(actor('alice'), async () => {
for (const id of privateDerivatives) expect(await companySkillService(db).getById(companyId, id)).toBeNull();
});
await db.delete(projects).where(eq(projects.id, opened.projectId));
await withSkillActor(actor('alice'), async () => expect(await companySkillService(db).getById(companyId, skillId)).toBeNull());
expect((await db.select().from(companySkills).where(eq(companySkills.id, skillId)))[0]!.accessSourceId).toBe(source.id);
});
it.each(['copy', 'delete'] as const)('removes project tracking with %s while retaining dynamic copy/fork access', async choice => {
files = { 'private/SKILL.md': md('retained private workflow') }; commit = '9'.repeat(40);
const row = await workspace(`removed-${choice}`, 'private');
await enqueueProjectSkillSource(db, row);
const worker = skillSourceSyncService(db, { context: async () => context });
await worker.tick();
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
const service = skillSourceService(db), skills = companySkillService(db);
const source = await service.detail(companyId, binding!.sourceId!);
const original = (await skills.getById(companyId, source.entries[0]!.skillId!))!;
const userId = randomUUID();
await db.insert(authUsers).values({ id: userId, name: 'Source owner', email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
await db.insert(companyMemberships).values({ companyId, principalType: 'user', principalId: userId, membershipRole: 'member', status: 'active' });
await db.insert(projectAccessMembers).values({ companyId, projectId: row.projectId, subjectType: 'user', subjectId: userId });
await withSkillActor(actor(userId), async () => {
const fork = await skills.forkSkill(companyId, original.id, { slug: `retained-${choice}-fork` });
await skills.updateFile(companyId, fork.skill.id, 'SKILL.md', md('Private fork edits') + '\nConfidential original-project notes.\n');
const result = await service.disconnect(companyId, source.id, context, { revision: source.revision, skills: choice });
expect(result).toMatchObject({ copiedCount: choice === 'copy' ? 1 : 0, deletedCount: choice === 'delete' ? 1 : 0 });
await expect(service.detail(companyId, source.id)).rejects.toThrow(/not found/);
expect((await service.list(companyId)).some(item => item.id === source.id)).toBe(false);
expect((await projectSkillRepositories(db, companyId, actor(userId))).find(item => item.workspaceId === row.id)).toMatchObject({ sourceId: null, status: 'available' });
expect(await skills.getById(companyId, fork.skill.id)).not.toBeNull();
let retainedVersionId = original.currentVersionId;
if (choice === 'copy') {
expect(await skills.getById(companyId, original.id)).toMatchObject({ sourceType: 'local_path', accessSourceId: source.id });
expect((await skills.readFile(companyId, original.id, '.paperclip-repository/private/SKILL.md'))?.content).toBe(md('retained private workflow'));
expect((await skills.readFile(companyId, original.id, 'SKILL.md'))?.content).toContain('.paperclip-repository/private/SKILL.md');
await skills.updateFile(companyId, original.id, 'SKILL.md', md('Private copy edits') + '\nConfidential original-project notes.\n');
retainedVersionId = (await skills.createVersion(companyId, original.id, { label: 'Private edited copy' })).id;
} else expect(await skills.getById(companyId, original.id)).toBeNull();
await db.update(companySkillSources).set({ automaticUpdates: false }).where(eq(companySkillSources.companyId, companyId));
const before = downloads.mock.calls.length;
await worker.tick(new Date(Date.now() + 6 * 60_000));
expect(downloads).toHaveBeenCalledTimes(before);
// Reattachment retains disconnection; adoption is the deliberate restart action.
await enqueueProjectSkillSource(db, row);
await worker.tick(new Date(Date.now() + 6 * 60_000));
expect(downloads).toHaveBeenCalledTimes(before);
const legacy = await workspace(`removed-${choice}`, 'open');
await db.insert(projectAccessMembers).values({ companyId, projectId: legacy.projectId, subjectType: 'user', subjectId: userId });
// This older workspace has no binding/source ID; one explicit adoption must work.
await adoptProjectSkillRepository(db, companyId, legacy.id, context.authorize);
await worker.tick(new Date(Date.now() + 6 * 60_000));
await expect(service.detail(companyId, source.id)).rejects.toThrow(/not found/);
const [adopted] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, legacy.id));
expect(adopted!.status, adopted!.lastError ?? 'adopted binding').toBe('ready');
expect(adopted!.sourceId).toBeTruthy();
expect(adopted!.sourceId).not.toBe(source.id);
const restored = await service.detail(companyId, adopted!.sourceId!);
expect(restored.enabled).toBe(true);
expect(restored.lastError).toBeNull();
expect(restored.entries.filter(entry => entry.skillId)).toHaveLength(1);
expect(restored.entries[0]!.skillId).not.toBe(original.id);
if (choice === 'copy') expect(await skills.getById(companyId, original.id)).toMatchObject({ key: original.key, sourceType: 'local_path', currentVersionId: retainedVersionId });
const readerId = randomUUID();
await db.insert(authUsers).values({ id: readerId, name: 'Open project reader', email: `${readerId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
await db.insert(companyMemberships).values({ companyId, principalType: 'user', principalId: readerId, membershipRole: 'member', status: 'active' });
await withSkillActor(actor(readerId), async () => {
const upstream = restored.entries[0]!.skillId!;
expect(await skills.getById(companyId, upstream)).not.toBeNull();
expect(await skills.getById(companyId, fork.skill.id)).toBeNull();
expect(await skills.readFile(companyId, fork.skill.id, 'SKILL.md')).toBeNull();
if (choice === 'copy') {
expect(await skills.getById(companyId, original.id)).toBeNull();
expect(await skills.readFile(companyId, original.id, 'SKILL.md')).toBeNull();
expect(await skills.getVersion(companyId, original.id, retainedVersionId!)).toBeNull();
}
expect((await skills.readFile(companyId, upstream, '.paperclip-repository/private/SKILL.md'))?.content).not.toContain('Confidential');
// A guessed hidden slug cannot replace or erase someone else's private fork.
const publicFork = await skills.forkSkill(companyId, upstream, { slug: `retained-${choice}-fork` });
expect(publicFork.skill.id).not.toBe(fork.skill.id);
expect(publicFork.skill.slug).not.toBe(fork.skill.slug);
});
expect((await skills.readFile(companyId, fork.skill.id, 'SKILL.md'))?.content).toContain('Confidential original-project notes');
// Explicit adoption of the original workspace also moves tracking, not provenance.
await adoptProjectSkillRepository(db, companyId, row.id, context.authorize);
await worker.tick(new Date(Date.now() + 6 * 60_000));
const [rebound] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
expect(rebound!.sourceId).toBe(restored.id);
expect(await skills.getById(companyId, fork.skill.id)).not.toBeNull();
await db.delete(projects).where(eq(projects.id, legacy.projectId));
await db.delete(projectAccessMembers).where(and(eq(projectAccessMembers.projectId, row.projectId), eq(projectAccessMembers.subjectId, userId)));
expect(await skills.getById(companyId, fork.skill.id)).toBeNull();
if (choice === 'copy') expect(await skills.getById(companyId, original.id)).toBeNull();
await db.delete(projects).where(eq(projects.id, row.projectId));
expect(await skills.getById(companyId, fork.skill.id)).toBeNull();
});
});
it('recovers expired jobs, competes through durable leases and respects failed-refresh backoff', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = { 'leased/SKILL.md': md('leased') }; commit = 'f'.repeat(40);
const row = await workspace('leased');
const binding = await enqueueProjectSkillSource(db, row);
await db.update(projectSkillSources).set({ leaseToken: randomUUID(), leaseExpiresAt: new Date(Date.now() - 1_000) })
.where(eq(projectSkillSources.id, binding!.id));
let release!: () => void, scanned!: () => void;
const gate = new Promise<void>(resolve => { release = resolve; });
const started = new Promise<void>(resolve => { scanned = resolve; });
const blocking: SkillSourceContext = { ...context, read: () => {
const fixture = context.read(null), open = fixture.openSnapshot!;
fixture.openSnapshot = async (...args) => { scanned(); await gate; return open(...args); };
return fixture;
} };
const first = skillSourceSyncService(db, { context: async () => blocking }).tick();
await started;
expect((await skillSourceSyncService(db, { context: async () => context }).tick()).imported).toBe(0);
release(); expect((await first).imported).toBe(1);
const [ready] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, binding!.id));
const sourceId = ready!.sourceId!;
const service = skillSourceService(db);
const original = await service.detail(companyId, sourceId);
const failure: SkillSourceContext = { ...context, read: () => (async () => {
throw unprocessable('GitHub request limit reached.', { retryAfterMs: 7_200_000 });
}) as ReturnType<SkillSourceContext['read']> };
const failedAt = new Date();
await db.update(companySkillSources).set({ nextSyncAt: failedAt }).where(eq(companySkillSources.id, sourceId));
await skillSourceSyncService(db, { context: async () => failure }).tick(failedAt);
const [failed] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, sourceId));
expect(failed!.failureCount).toBe(1);
expect(failed!.nextSyncAt.getTime() - failedAt.getTime()).toBeGreaterThanOrEqual(7_200_000);
expect((await service.detail(companyId, sourceId)).entries[0]!.skillId).toBe(original.entries[0]!.skillId);
const retryRead = vi.fn(context.read);
await skillSourceSyncService(db, { context: async () => ({ ...context, read: retryRead }) })
.tick(new Date(failedAt.getTime() + 6 * 60_000));
expect(retryRead).not.toHaveBeenCalled();
await db.update(companySkillSources).set({ automaticUpdates: false, nextSyncAt: new Date() }).where(eq(companySkillSources.id, sourceId));
await skillSourceSyncService(db, { context: async () => ({ ...context, read: retryRead }) }).tick();
expect(retryRead).not.toHaveBeenCalled();
});
it('skips live leases before bounding import and refresh batches', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = { 'fair/SKILL.md': md('fair') }; commit = '9'.repeat(40);
const now = new Date();
const expires = new Date(now.getTime() + 60 * 60_000);
const busyJobs: string[] = [];
for (let index = 0; index < 10; index++) {
const row = await workspace(`busy-import-${index}`);
const binding = await enqueueProjectSkillSource(db, row);
busyJobs.push(binding!.id);
await db.update(projectSkillSources).set({ leaseToken: randomUUID(), leaseExpiresAt: expires,
nextSyncAt: new Date(now.getTime() - (20 - index) * 1_000) }).where(eq(projectSkillSources.id, binding!.id));
}
const ready = await enqueueProjectSkillSource(db, await workspace('ready-import'));
await db.update(projectSkillSources).set({ nextSyncAt: now }).where(eq(projectSkillSources.id, ready!.id));
const worker = skillSourceSyncService(db, { context: async () => context });
expect((await worker.tick(now)).imported).toBe(1);
const [imported] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, ready!.id));
expect(imported!.sourceId).toBeTruthy();
for (const id of busyJobs) {
const [job] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, id));
expect(job).toMatchObject({ sourceId: null, leaseExpiresAt: expires, status: 'pending' });
}
const [template] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, imported!.sourceId!));
await db.update(companySkillSources).set({ automaticUpdates: false });
const readySourceId = randomUUID();
await db.insert(companySkillSources).values(Array.from({ length: 11 }, (_, index) => ({
...template!, id: index === 10 ? readySourceId : randomUUID(), trackingRef: `fair-refresh-${index}`,
accessScope: 'company' as const, automaticUpdates: true, nextSyncAt: new Date(now.getTime() - (20 - index) * 1_000),
leaseToken: index === 10 ? null : randomUUID(), leaseExpiresAt: index === 10 ? null : expires,
})));
expect((await worker.tick(now)).checked).toBe(1);
const [refreshed] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, readySourceId));
expect(refreshed!.nextSyncAt.getTime()).toBeGreaterThan(now.getTime());
});
it('releases interrupted initial imports so a new worker can recover them', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = { 'recovered/SKILL.md': md('recovered') }; commit = '1'.repeat(40);
const row = await workspace('restart');
const binding = await enqueueProjectSkillSource(db, row);
let scanned!: () => void;
const started = new Promise<void>(resolve => { scanned = resolve; });
const blocking: SkillSourceContext = { ...context, read: () => {
const fixture = context.read(null);
fixture.openSnapshot = async (_input, options) => {
scanned();
await new Promise<void>((_resolve, reject) => {
options!.signal!.addEventListener('abort', () => reject(options!.signal!.reason), { once: true });
});
throw new Error('Unreachable');
};
return fixture;
} };
const worker = skillSourceSyncService(db, { context: async () => blocking });
const sweep = worker.tick();
await started;
await worker.stop();
await sweep;
const [interrupted] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.id, binding!.id));
expect(interrupted).toMatchObject({ status: 'pending', failureCount: 0, sourceId: null, leaseToken: null, lastError: null });
expect((await skillSourceSyncService(db, { context: async () => context }).tick()).imported).toBe(1);
});
it('keeps deleted setup agents attributable and requires explicit connection repair', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = { 'owned/SKILL.md': md('owned') }; commit = '2'.repeat(40);
const row = await workspace('deleted-owner');
const [owner] = await db.insert(agents).values({ companyId, name: 'Source setup agent', adapterType: 'codex_local', status: 'idle' }).returning();
await enqueueProjectSkillSource(db, row);
const ownerContext: SkillSourceContext = { ...context, setup: { userId: 'local-board', localTrusted: true, agentId: owner!.id } };
await skillSourceSyncService(db, { context: async () => ownerContext }).tick();
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
const sourceId = binding!.sourceId!;
await db.delete(agents).where(eq(agents.id, owner!.id));
await db.update(companySkillSources).set({ nextSyncAt: new Date() }).where(eq(companySkillSources.id, sourceId));
const [before] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, sourceId));
expect(before!.refreshAgentId).toBe(owner!.id);
await skillSourceSyncService(db).tick();
const [failed] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, sourceId));
expect(failed!.lastError).toContain('agent who connected');
expect(failed!.failureCount).toBe(1);
expect((await skillSourceService(db).detail(companyId, sourceId)).entries[0]!.skillId).toBeTruthy();
});
it('backs off a policy denial before acquiring a source lease', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = { 'policy/SKILL.md': md('policy') }; commit = '3'.repeat(40);
const row = await workspace('policy-denial');
await enqueueProjectSkillSource(db, row);
await skillSourceSyncService(db, { context: async () => context }).tick();
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
const now = new Date();
await db.update(companySkillSources).set({ nextSyncAt: now }).where(eq(companySkillSources.id, binding!.sourceId!));
const authorize = vi.fn(async () => { throw unprocessable('Automatic updates are denied by company policy.'); });
const worker = skillSourceSyncService(db, { context: async () => ({ ...context, authorize }) });
await worker.tick(now);
const [failed] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, binding!.sourceId!));
expect(failed).toMatchObject({ failureCount: 1, leaseToken: null, lastError: 'Automatic updates are denied by company policy.' });
expect(failed!.nextSyncAt.getTime()).toBeGreaterThan(now.getTime());
await worker.tick(new Date(now.getTime() + 1000));
expect(authorize).toHaveBeenCalledTimes(1);
});
it('uses bounded skill packages when a tracked repository outgrows the repository limit', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = { 'small/SKILL.md': md('small') }; commit = '4'.repeat(40);
const row = await workspace('growing-repository');
await enqueueProjectSkillSource(db, row);
await skillSourceSyncService(db, { context: async () => context }).tick();
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, row.id));
const service = skillSourceService(db), original = await service.detail(companyId, binding!.sourceId!);
expect(original.packageMode).toBe('repository');
const skillId = original.entries[0]!.skillId!;
files['small/SKILL.md'] = md('small updated'); commit = '5'.repeat(40);
const growing: SkillSourceContext = { ...context, read: () => {
const fixture = context.read(null), open = fixture.openSnapshot;
fixture.openSnapshot = async (...args) => {
const snapshot = await open(...args);
for (let index = 0; index < 10000; index++) snapshot.entries.push({ path: `unrelated/${index}.txt`, type: 'blob', mode: '100644', sha: String(index + 1), size: 0 });
return snapshot;
};
return fixture;
} };
const result = await service.refresh(companyId, original.id, growing);
expect(result.source.packageMode).toBe('skills');
expect(result.warnings.join(' ')).toContain('repository exceeds');
expect(result.source.entries[0]!.skillId).toBe(skillId);
expect(result.updated).toHaveLength(1);
files = { 'small/SKILL.md': md('small with shared guide') + '\nRead [guide](../shared/guide.md).', 'shared/guide.md': 'Now fits in the repository package.' };
commit = '6'.repeat(40);
const complete = await service.refresh(companyId, original.id, context);
expect(complete.source.packageMode).toBe('repository');
expect(complete.source.entries.find(entry => entry.path === 'small/SKILL.md')!.skillId).toBe(skillId);
expect((await companySkillService(db).readFile(companyId, skillId, '.paperclip-repository/shared/guide.md'))?.content)
.toBe('Now fits in the repository package.');
});
it('enqueues repository changes atomically, leaves ordinary legacy edits alone and retains old references', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
const legacy = await workspace('lifecycle');
const service = projectService(db, { captureWorkspaceUpdates: false });
await service.updateWorkspace(legacy.projectId, legacy.id, { name: 'Renamed older workspace' });
expect(await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, legacy.id))).toHaveLength(0);
await expect(db.transaction(async tx => {
await projectService(tx as unknown as typeof db, { captureWorkspaceUpdates: false }).createWorkspace(legacy.projectId,
{ repoUrl: 'https://github.com/acme/skills', defaultRef: 'rolled-back' });
throw new Error('Required project audit failed');
})).rejects.toThrow('Required project audit failed');
expect(await db.select().from(projectWorkspaces).where(eq(projectWorkspaces.defaultRef, 'rolled-back'))).toHaveLength(0);
expect(await db.select().from(projectSkillSources).where(eq(projectSkillSources.trackingRef, 'rolled-back'))).toHaveLength(0);
files = { 'lifecycle/SKILL.md': md('lifecycle') }; commit = '6'.repeat(40);
const created = (await service.createWorkspace(legacy.projectId,
{ repoUrl: 'https://github.com/acme/skills', defaultRef: 'lifecycle-created' }))!;
const worker = skillSourceSyncService(db, { context: async () => context });
await worker.tick();
const [originalBinding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, created.id));
const source = await skillSourceService(db).detail(companyId, originalBinding!.sourceId!);
const skillId = source.entries[0]!.skillId!;
const versionId = (await companySkillService(db).getById(companyId, skillId))!.currentVersionId;
await service.updateWorkspace(legacy.projectId, created.id, { defaultRef: 'lifecycle-next' });
await worker.tick();
const bindings = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, created.id));
expect(bindings).toHaveLength(2);
expect(bindings.find(binding => binding.id === originalBinding!.id)?.active).toBe(false);
const current = bindings.find(binding => binding.active)!;
expect(current.sourceId).not.toBe(source.id);
const retained = (await companySkillService(db).getById(companyId, skillId))!;
expect(retained.currentVersionId).toBe(versionId);
const newSource = await skillSourceService(db).detail(companyId, current.sourceId!);
expect(newSource.entries[0]!.skillId).not.toBe(skillId);
await service.updateWorkspace(legacy.projectId, created.id, { defaultRef: 'lifecycle-next' });
expect(await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, created.id))).toHaveLength(2);
await service.removeWorkspace(legacy.projectId, created.id);
expect(await skillSourceService(db).detail(companyId, current.sourceId!)).toMatchObject({ trackingActive: false });
expect(await companySkillService(db).getById(companyId, skillId)).not.toBeNull();
});
it('lets API-key agents attach repositories without inventing a setup person', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
const project = await workspace('key-agent-parent');
const [agent] = await db.insert(agents).values({ companyId, name: 'API-key project creator', adapterType: 'codex_local', status: 'idle' }).returning();
const keyActor: AuthorizationActor = { type: 'agent', source: 'agent_key', agentId: agent!.id, companyId, onBehalfOfUserId: 'unverified-person' };
const attached = await withSkillActor(keyActor, () => projectService(db, { captureWorkspaceUpdates: false }).createWorkspace(project.projectId,
{ repoUrl: 'https://github.com/acme/skills', defaultRef: 'key-agent' }));
expect(attached).not.toBeNull();
const [binding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, attached!.id));
expect(binding).toMatchObject({ status: 'pending', refreshUserId: null, refreshAgentId: agent!.id, refreshLocalTrusted: false, sourceId: null });
await detachProjectSkillSource(db, companyId, attached!.id);
});
it('requires connection repair before scheduled refresh of API-key public imports', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
files = {};
const [agent] = await db.insert(agents).values({ companyId, name: 'API-key source importer', adapterType: 'codex_local', status: 'idle' }).returning();
const setup = await projectSkillSetupIdentity(db, companyId, { type: 'agent', source: 'agent_key', companyId,
agentId: agent!.id, onBehalfOfUserId: 'local-board' });
const imported = await skillSourceService(db).create(companyId, { repositoryUrl: 'https://github.com/acme/skills',
trackingRef: 'key-generic-public', commitSha: commit, selectedPaths: [] }, { ...context,
actor: { type: 'agent', agentId: agent!.id }, setup: { userId: setup.refreshUserId, agentId: setup.refreshAgentId, localTrusted: setup.refreshLocalTrusted } });
const [stored] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, imported.source.id));
expect(stored).toMatchObject({ refreshUserId: null, refreshAgentId: agent!.id, refreshLocalTrusted: false, connectionId: null });
await db.insert(activityLog).values({ companyId, actorType: 'agent', actorId: agent!.id, agentId: agent!.id,
responsibleUserId: 'local-board', action: 'company.skill_source_refreshed', entityType: 'company_skill_source', entityId: imported.source.id });
await skillSourceSyncService(db).tick(new Date(Date.now() + 6 * 60_000));
const [checked] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, imported.source.id));
expect(checked!.lastError).toMatch(/Choose a GitHub connection/);
expect(checked!.refreshUserId).toBeNull();
expect(checked!.lastSuccessAt).toEqual(stored!.lastSuccessAt);
// Migrated sources may lack refreshAgentId; API-key audit responsibility
// still cannot stand in for an authenticated run's credential authority.
await db.update(companySkillSources).set({ refreshAgentId: null }).where(eq(companySkillSources.id, imported.source.id));
await skillSourceSyncService(db).tick(new Date(Date.now() + 12 * 60_000));
const [legacy] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, imported.source.id));
expect(legacy!.refreshUserId).toBeNull();
expect(legacy!.lastError).toMatch(/Choose a GitHub connection/);
});
it('follows a changed default branch while an explicit branch remains fixed', async () => {
await db.update(companySkillSources).set({ automaticUpdates: false });
let defaultBranch = 'main';
const originalFiles = { 'workflow/SKILL.md': md('original default') }, nextFiles = { 'workflow/SKILL.md': md('new default') };
const moving: SkillSourceContext = { ...context, read: () => {
const reader = (async (url: string) => {
if (url === '/repos/acme/skills') return { id: 42, full_name: 'acme/skills', default_branch: defaultBranch };
if (url.startsWith('/repos/acme/skills/commits/')) return { sha: decodeURIComponent(url.split('/commits/')[1]!) === 'main' ? '7'.repeat(40) : '8'.repeat(40) };
throw new Error('Unexpected request');
}) as ReturnType<SkillSourceContext['read']>;
reader.openSnapshot = async (input, options) => githubFixture(input.ref === 'main' ? originalFiles : nextFiles, {}, input.ref === 'main' ? '7'.repeat(40) : '8'.repeat(40)).openSnapshot(input, options);
return reader;
} };
const implicit = await workspace('HEAD');
await enqueueProjectSkillSource(db, implicit);
const worker = skillSourceSyncService(db, { context: async () => moving });
await worker.tick();
const [defaultBinding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, implicit.id));
const sourceService = skillSourceService(db);
const original = await sourceService.detail(companyId, defaultBinding!.sourceId!);
expect(original.trackingRef).toBe('HEAD');
const skillId = original.entries[0]!.skillId!;
const versionId = (await companySkillService(db).getById(companyId, skillId))!.currentVersionId;
const explicit = await workspace('main');
await enqueueProjectSkillSource(db, explicit);
await worker.tick();
const [fixedBinding] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, explicit.id));
expect(fixedBinding!.sourceId).not.toBe(original.id);
defaultBranch = 'next-default';
await worker.tick(new Date(Date.now() + 6 * 60_000));
expect(await sourceService.detail(companyId, original.id)).toMatchObject({ trackingRef: 'HEAD', lastScanCommit: '8'.repeat(40) });
expect(await sourceService.detail(companyId, fixedBinding!.sourceId!)).toMatchObject({ trackingRef: 'main', lastScanCommit: '7'.repeat(40) });
const latest = (await companySkillService(db).getById(companyId, skillId))!;
expect(latest.currentVersionId).not.toBe(versionId);
expect(latest.markdown).toBe(md('new default'));
expect((await companySkillService(db).getVersion(companyId, skillId, versionId!))!.fileInventory[0]!.content).toBe(md('original default'));
const repeated = await workspace('HEAD');
await enqueueProjectSkillSource(db, repeated);
await worker.tick();
const [reused] = await db.select().from(projectSkillSources).where(eq(projectSkillSources.workspaceId, repeated.id));
expect(reused!.sourceId).toBe(original.id);
});
});
server/src/__tests__/server-startup-feedback-export.test.ts
+6
View File
@@ -251,6 +251,12 @@ vi.mock("../app.js", () => ({
createApp: createAppMock,
}));
// This suite exercises startup coordination; source persistence and authority
// are covered by the database-backed source worker tests.
vi.mock("../services/skill-source-sync.js", () => ({
skillSourceSyncService: () => ({ tick: vi.fn(async () => ({ imported: 0, checked: 0 })), stop: vi.fn(async () => {}) }),
}));
vi.mock("../services/native-runtime/native-session-executor.js", () => ({
verifyStoppedNativeSessionForReplacement: vi.fn(async () => null),
}));
server/src/__tests__/skill-source-retry.test.ts
+19
View File
@@ -0,0 +1,19 @@
import { describe, expect, it } from 'vitest';
import { unprocessable } from '../errors.js';
import { skillSourceRetryAt } from '../services/skill-source-retry.js';
describe('skill source retry scheduling', () => {
const now = new Date('2026-10-12T12:00:00Z');
it('backs off repeated failures and caps ordinary retries at an hour', () => {
expect(skillSourceRetryAt(new Error('failed'), 0, now).getTime() - now.getTime()).toBe(300_000);
expect(skillSourceRetryAt(new Error('failed'), 2, now).getTime() - now.getTime()).toBe(1_200_000);
expect(skillSourceRetryAt(new Error('failed'), 20, now).getTime() - now.getTime()).toBe(3_600_000);
});
it('honors provider rate limits without accepting unbounded or invalid delays', () => {
const retry = (delay: number) => skillSourceRetryAt(unprocessable('Limited', { retryAfterMs: delay }), 0, now).getTime() - now.getTime();
expect(retry(7_200_000)).toBe(7_200_000);
expect(retry(100)).toBe(300_000);
expect(retry(Infinity)).toBe(300_000);
expect(retry(10 ** 12)).toBe(86_400_000);
});
});
server/src/__tests__/skill-sources.test.ts
+146 -3
View File
@@ -37,6 +37,88 @@ describe.skipIf(!support.supported)('skill source persistence', () => {
if (originalHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = originalHome;
await testDb?.cleanup(); if (home) await fs.rm(home, { recursive: true, force: true });
});
it('checks a new commit with an unchanged tree without downloading or replacing versions', async () => {
const originalFiles = files, originalCommit = commit;
try {
const service = skillSourceService(db);
files = { 'tree/SKILL.md': md('tree') }; commit = '3'.repeat(40);
const created = await service.create(companyId, { repositoryUrl: 'https://github.com/acme/skills', trackingRef: 'tree-only', commitSha: commit, selectedPaths: ['tree/SKILL.md'] }, context);
const skillId = created.source.entries[0]!.skillId!;
const previous = (await companySkillService(db).getById(companyId, skillId))!.currentVersionId;
await db.update(companySkillSources).set({ lastScanTreeSha: null }).where(eq(companySkillSources.id, created.source.id));
const currentReader = context.read(null), currentOpen = vi.spyOn(currentReader, 'openSnapshot');
await service.refresh(companyId, created.source.id, { ...context, read: () => currentReader }, undefined, undefined, { checkCommit: true });
expect(currentOpen).not.toHaveBeenCalled();
const [remembered] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, created.source.id));
expect(remembered!.lastScanTreeSha).toMatch(/^[a-f0-9]{40}$/);
commit = '4'.repeat(40);
const reader = context.read(null), open = vi.spyOn(reader, 'openSnapshot');
const result = await service.refresh(companyId, created.source.id, { ...context, read: () => reader }, undefined, undefined, { checkCommit: true });
expect(open).not.toHaveBeenCalled();
expect(result.source.lastScanCommit).toBe(commit);
expect((await companySkillService(db).getById(companyId, skillId))!.currentVersionId).toBe(previous);
files['tree/SKILL.md'] = md('tree changed'); commit = '5'.repeat(40);
const changed = context.read(null), changedOpen = vi.spyOn(changed, 'openSnapshot');
await service.refresh(companyId, created.source.id, { ...context, read: () => changed }, undefined, undefined, { checkCommit: true });
expect(changedOpen).toHaveBeenCalledTimes(1);
expect((await companySkillService(db).getById(companyId, skillId))!.currentVersionId).not.toBe(previous);
} finally { files = originalFiles; commit = originalCommit; }
});
it('persists automatic work limits, refills capacity, and allows explicit refresh', async () => {
const originalFiles = files, originalCommit = commit;
try {
files = { 'budget/SKILL.md': md('budget') }; commit = '6'.repeat(40);
let downloads = 0;
const budgetContext: SkillSourceContext = { ...context, read: () => {
const reader = context.read(null), open = reader.openSnapshot;
reader.openSnapshot = async (...args) => { downloads++; return { ...await open(...args), downloadBytes: 128 * 1024 * 1024 }; };
return reader;
} };
const service = skillSourceService(db);
const created = await service.create(companyId, { repositoryUrl: 'https://github.com/acme/skills', trackingRef: 'work-budget', commitSha: commit, selectedPaths: ['budget/SKILL.md'] }, budgetContext);
const start = new Date();
for (let index = 0; index < 4; index++) {
files['budget/SKILL.md'] = md(`budget revision ${index}`); commit = (7 + index).toString(16).repeat(40);
await service.refresh(companyId, created.source.id, budgetContext, undefined, undefined, { checkCommit: true, automatic: true, now: start });
}
const skillId = created.source.entries[0]!.skillId!;
const version = (await companySkillService(db).getById(companyId, skillId))!.currentVersionId;
files['budget/SKILL.md'] = md('budget next'); commit = 'b'.repeat(40);
const before = downloads;
await expect(skillSourceService(db).refresh(companyId, created.source.id, budgetContext, undefined, undefined,
{ checkCommit: true, automatic: true, now: start })).rejects.toThrow(/temporarily limited/);
expect(downloads).toBe(before);
expect((await companySkillService(db).getById(companyId, skillId))!.currentVersionId).toBe(version);
await service.refresh(companyId, created.source.id, budgetContext, undefined, undefined, { checkCommit: true });
expect(downloads).toBe(before + 1);
const [budgetRow] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, created.source.id));
expect(budgetRow!.automaticBudgetAt.getTime()).toBe(start.getTime());
expect(budgetRow!.automaticBudgetBytes).toBe(0);
files['budget/SKILL.md'] = md('budget refilled'); commit = 'c'.repeat(40);
await skillSourceService(db).refresh(companyId, created.source.id, budgetContext, undefined, undefined,
{ checkCommit: true, automatic: true, now: new Date(start.getTime() + 15 * 60_000) });
expect(downloads).toBe(before + 2);
expect((await companySkillService(db).getById(companyId, skillId))!.currentVersionId).not.toBe(version);
} finally { files = originalFiles; commit = originalCommit; }
});
it('refunds unused automatic capacity for small repositories', async () => {
const originalFiles = files, originalCommit = commit;
try {
files = { 'small/SKILL.md': md('small') }; commit = 'd'.repeat(40);
const service = skillSourceService(db);
const created = await service.create(companyId, { repositoryUrl: 'https://github.com/acme/skills', trackingRef: 'small-budget', commitSha: commit, selectedPaths: ['small/SKILL.md'] }, context);
const now = new Date();
for (let index = 0; index < 6; index++) {
files['small/SKILL.md'] = md(`small revision ${index}`); commit = String(index + 1).repeat(40);
await service.refresh(companyId, created.source.id, context, undefined, undefined, { checkCommit: true, automatic: true, now });
}
const [stored] = await db.select().from(companySkillSources).where(eq(companySkillSources.id, created.source.id));
expect(stored!.automaticBudgetBytes).toBeGreaterThan(500 * 1024 * 1024);
} finally { files = originalFiles; commit = originalCommit; }
});
it('imports duplicate names independently, stores complete snapshots and refreshes without duplicates', async () => {
const sources = skillSourceService(db), skills = companySkillService(db);
const result = await sources.create(companyId, { repositoryUrl: 'https://github.com/acme/skills', commitSha: sha, selectedPaths: ['deep/one/SKILL.md', 'elsewhere/one/SKILL.md'] }, context);
@@ -167,7 +249,12 @@ describe.skipIf(!support.supported)('skill source persistence', () => {
{ companyId: legacyCompany, key: 'legacy/custom', slug: 'original-slug', name: 'Original name', markdown: md('legacy'), sourceType: 'github', sourceRef: sha, metadata: { hostname: 'github.com', owner: 'acme', repo: 'skills', trackingRef: 'legacy', repoSkillDir: 'legacy' } },
{ companyId: legacyCompany, key: 'paperclipai/paperclip/bundled', slug: 'bundled', name: 'Bundled', markdown: md('bundled'), sourceType: 'github', metadata: { hostname: 'github.com', owner: 'acme', repo: 'skills', sourceKind: 'paperclip_bundled' } },
]).returning();
const migration = await fs.readFile(new URL('../../../packages/db/src/migrations/0291_conscious_secret_warriors.sql', import.meta.url), 'utf8');
// Production runs 0291 before 0341. This harness uses the latest schema,
// whose repository/ref uniqueness is partial; retain the same backfill
// assertions while specifying that index's inference predicate.
const migration = (await fs.readFile(new URL('../../../packages/db/src/migrations/0291_conscious_secret_warriors.sql', import.meta.url), 'utf8'))
.replace('ON CONFLICT (company_id, repository_url, tracking_ref) DO NOTHING',
'ON CONFLICT (company_id, repository_url, tracking_ref) WHERE removed_at IS NULL DO NOTHING');
await db.execute(sql.raw(migration.slice(migration.indexOf('DO $$', migration.indexOf('-- Adopt only')))));
await db.execute(sql.raw(migration.slice(migration.indexOf('DO $$', migration.indexOf('-- Adopt only')))));
const service = skillSourceService(db);
@@ -225,7 +312,12 @@ describe.skipIf(!support.supported)('skill source persistence', () => {
{ companyId: legacyCompany, key: 'legacy/missing-ref', slug: 'old', name: 'Old', markdown: md('old'), sourceType: 'github', sourceRef: sha, metadata: { owner: 'acme', repo: 'skills', ref: sha, repoSkillDir: 'old' } },
{ companyId: legacyCompany, key: 'legacy/pinned', slug: 'pinned', name: 'Pinned', markdown: md('pinned'), sourceType: 'github', sourceRef: sha, metadata: { owner: 'acme', repo: 'skills', ref: sha, trackingRef: sha, repoSkillDir: 'pinned' } },
]).returning();
const migration = await fs.readFile(new URL('../../../packages/db/src/migrations/0291_conscious_secret_warriors.sql', import.meta.url), 'utf8');
// Production runs 0291 before 0341. This harness uses the latest schema,
// whose repository/ref uniqueness is partial; retain the same backfill
// assertions while specifying that index's inference predicate.
const migration = (await fs.readFile(new URL('../../../packages/db/src/migrations/0291_conscious_secret_warriors.sql', import.meta.url), 'utf8'))
.replace('ON CONFLICT (company_id, repository_url, tracking_ref) DO NOTHING',
'ON CONFLICT (company_id, repository_url, tracking_ref) WHERE removed_at IS NULL DO NOTHING');
await db.execute(sql.raw(migration.slice(migration.indexOf('DO $$', migration.indexOf('-- Adopt only')))));
const service = skillSourceService(db);
const adopted = (await service.sourceForSkill(legacyCompany, legacy!.id))!;
@@ -250,7 +342,12 @@ describe.skipIf(!support.supported)('skill source persistence', () => {
{ companyId: legacyCompany, key: 'legacy/implicit', slug: 'implicit', name: 'Implicit', markdown: md('old'), sourceType: 'github', sourceRef: sha, metadata: { owner: 'acme', repo: 'skills', ref: sha, repoSkillDir: 'one' } },
{ companyId: legacyCompany, key: 'legacy/explicit', slug: 'explicit', name: 'Explicit', markdown: md('old'), sourceType: 'github', sourceRef: sha, metadata: { owner: 'acme', repo: 'skills', trackingRef: 'main', repoSkillDir: 'one' } },
]).returning();
const migration = await fs.readFile(new URL('../../../packages/db/src/migrations/0291_conscious_secret_warriors.sql', import.meta.url), 'utf8');
// Production runs 0291 before 0341. This harness uses the latest schema,
// whose repository/ref uniqueness is partial; retain the same backfill
// assertions while specifying that index's inference predicate.
const migration = (await fs.readFile(new URL('../../../packages/db/src/migrations/0291_conscious_secret_warriors.sql', import.meta.url), 'utf8'))
.replace('ON CONFLICT (company_id, repository_url, tracking_ref) DO NOTHING',
'ON CONFLICT (company_id, repository_url, tracking_ref) WHERE removed_at IS NULL DO NOTHING');
await db.execute(sql.raw(migration.slice(migration.indexOf('DO $$', migration.indexOf('-- Adopt only')))));
const service = skillSourceService(db);
files = { 'one/SKILL.md': md('updated') }; commit = 'e'.repeat(40);
@@ -424,6 +521,52 @@ describe.skipIf(!support.supported)('skill source persistence', () => {
await expect(fs.stat(path.join(runtime.source, '.paperclip-repository/shared/tools'))).rejects.toMatchObject({ code: 'ENOENT' });
});
it.each([false, true])('retains installed repository dependencies after an oversized refresh (scheduled: %s)', async scheduled => {
const temporaryCompany = randomUUID();
await db.insert(companies).values({ id: temporaryCompany, name: 'Repository fallback', issuePrefix: 'FBK' });
let repoFiles = {
'skills/main/SKILL.md': md('main') + '\nRead [shared](../../shared/guide.md) and [helper](../helper/SKILL.md).',
'skills/helper/SKILL.md': md('helper'),
'shared/guide.md': 'Working shared instructions',
};
let head = sha;
const localContext = { ...context, read: () => {
const fixture = githubFixture(repoFiles, {}, head);
const open = fixture.openSnapshot;
fixture.openSnapshot = async (...args) => {
const snapshot = await open(...args);
for (const entry of snapshot.entries) if (entry.path.startsWith('generated/')) entry.size = 1024 * 1024;
return snapshot;
};
return fixture;
} };
try {
const service = skillSourceService(db), library = companySkillService(db);
const installed = await service.create(temporaryCompany, { repositoryUrl: 'https://github.com/acme/skills', packageMode: 'repository',
selectedPaths: ['skills/main/SKILL.md', 'skills/helper/SKILL.md'] }, localContext);
const main = installed.imported.find(skill => skill.name === 'main')!;
const versions = await library.listVersions(temporaryCompany, main.id);
head = 'f'.repeat(40);
repoFiles = { ...repoFiles, 'shared/guide.md': 'New shared instructions',
...Object.fromEntries(Array.from({ length: 62 }, (_, i) => [`generated/${i}.bin`, 'x'])) };
await expect(service.refresh(temporaryCompany, installed.source.id, localContext, undefined, undefined, { checkCommit: scheduled }))
.rejects.toThrow('Installed repository skills and their dependencies were kept');
const retained = await service.detail(temporaryCompany, installed.source.id);
expect(retained.packageMode).toBe('repository');
expect(retained.lastScanCommit).toBe(sha);
expect(retained.lastError).toContain('dependencies were kept');
expect(await library.listVersions(temporaryCompany, main.id)).toEqual(versions);
expect((await library.readFile(temporaryCompany, main.id, '.paperclip-repository/shared/guide.md'))?.content).toBe('Working shared instructions');
expect((await library.readFile(temporaryCompany, main.id, '.paperclip-repository/skills/helper/SKILL.md'))?.content).toBe(md('helper'));
repoFiles = { 'skills/main/SKILL.md': repoFiles['skills/main/SKILL.md'], 'skills/helper/SKILL.md': md('helper'), 'shared/guide.md': 'New shared instructions' };
await service.refresh(temporaryCompany, installed.source.id, localContext);
expect((await service.detail(temporaryCompany, installed.source.id)).lastError).toBeNull();
expect((await library.readFile(temporaryCompany, main.id, '.paperclip-repository/shared/guide.md'))?.content).toBe('New shared instructions');
} finally {
await deleteCompany(db, temporaryCompany);
}
});
it('cascades repository snapshots and referencing versions when the owning company is deleted', async () => {
const temporaryCompany = randomUUID();
await db.insert(companies).values({ id: temporaryCompany, name: 'Temporary repository company', issuePrefix: 'TMP' });
server/src/app.ts
+2
View File
@@ -52,6 +52,7 @@ import {
import type { InspectDatabaseBackupHealthOptions } from "./services/database-backup-health.js";
import type { StorageService } from "./storage/types.js";
import { httpLogger, errorHandler } from "./middleware/index.js";
import { skillAccessRequestContext } from "./services/skill-access.js";
import { actorMiddleware } from "./middleware/auth.js";
import { boardMutationGuard } from "./middleware/board-mutation-guard.js";
import {
@@ -655,6 +656,7 @@ export async function createApp(
// REPLACES whatever actor the request otherwise resolved to, and only on
// the one endpoint it authorizes (see the middleware for the contract).
app.use(cloudControlMiddleware());
app.use(skillAccessRequestContext);
app.use("/api/auth", authRoutes(db));
if (opts.betterAuthHandler) {
app.all("/api/auth/{*authPath}", opts.betterAuthHandler);
server/src/index.ts
+14
View File
@@ -1,3 +1,4 @@
import { skillSourceSyncService } from "./services/skill-source-sync.js";
import { subscribeAllCompanyLiveEvents } from "./services/live-events.js";
import { DELIVERY_QUEUES } from "./services/delivery-work-notifications.js";
import type { createDeliveryWorkCoordinator } from "./services/delivery-work-coordinator.js";
@@ -1212,11 +1213,22 @@ async function startServerWithDatabaseTeardown(
}, config.heartbeatSchedulerIntervalMs);
heartbeatSchedulerInterval?.unref?.();
};
const skillSourceSync = skillSourceSyncService(db as any);
const sweepSkillSources = () => {
if (isIdleTaskDrainActive() || heartbeatSchedulerStopped || isWarmStandby()) return;
trackHeartbeatSchedulerWork(skillSourceSync.tick().catch(err => {
logger.error({ err }, "skill source synchronization failed");
}));
};
const skillSourceSyncInterval = setInterval(sweepSkillSources, 5000);
skillSourceSyncInterval.unref?.();
sweepSkillSources();
const externalObjects = externalObjectService(db as any, {
pluginWorkerManager,
enabled: async () => (await instanceSettingsService(db).getExperimental()).enableExternalObjects === true,
});
const scheduleExternalObjectRefreshSweep = (now = new Date()) => {
sweepSkillSources();
if (isIdleTaskDrainActive() || heartbeatSchedulerStopped) return;
trackHeartbeatSchedulerWork(externalObjects
.refreshDueObjectsForActiveCompanies(50, now)
@@ -1968,6 +1980,8 @@ async function startServerWithDatabaseTeardown(
) => {
await systemdNotify(["--stopping", `--status=Stopping after ${signal}`]);
heartbeatSchedulerStopped = true;
clearInterval(skillSourceSyncInterval);
await skillSourceSync.stop();
unsubscribeChatCompletions();
await deliveryWork.stop();
clearInterval(executionControlInterval);
server/src/realtime/live-events-ws.ts
+8
View File
@@ -1,3 +1,4 @@
import { skillAccessService } from "../services/skill-access.js";
import { createHash } from "node:crypto";
import type { IncomingMessage, Server as HttpServer } from "node:http";
import { createRequire } from "node:module";
@@ -303,6 +304,13 @@ export function setupLiveEventsWebSocketServer(
if (!issue || !(await access.decide({ actor: context!.actor, action: "issue:read",
resource: { type: "issue", companyId: event.companyId, issueId } })).allowed) return null;
}
if (["company_skill", "company_skill_source"].includes(String(payload.entityType)) && typeof payload.entityId === "string") {
try {
const skills = skillAccessService(db, context!.actor);
if (payload.entityType === "company_skill") await skills.assertSkill(event.companyId, payload.entityId);
else await skills.assertSource(event.companyId, payload.entityId);
} catch { return null; }
}
if (payload.entityType === "project" && typeof payload.entityId === "string") {
const [project] = await db.select({ id: projects.id }).from(projects)
.where(and(eq(projects.id, payload.entityId), eq(projects.companyId, event.companyId))).limit(1);
server/src/routes/company-skills.ts
+50 -4
View File
@@ -1,7 +1,10 @@
import { projectSkillSetupIdentity } from "../services/project-skill-source-bindings.js";
import { adoptProjectSkillRepository, projectSkillRepositories, skillSourceSyncService } from "../services/skill-source-sync.js";
import { skillAccessRequestContext, skillAccessService } from "../services/skill-access.js";
import { skillSourceService, type SkillSourceContext } from "../services/skill-sources.js";
import { skillSourceGitHubReader } from "../services/skill-source-github-access.js";
import { toolAccessService } from "../services/tool-access.js";
import { skillSourceDisconnectSchema, skillSourceCreateSchema, skillSourceDiscoverySchema, skillSourcePreviewSchema, skillSourceSelectionSchema } from "@paperclipai/shared";
import { skillSourceCreateSchema, skillSourceDiscoverySchema, skillSourcePreviewSchema, skillSourceSelectionSchema, skillSourceSettingsSchema, projectSkillSourceAdoptSchema, skillSourceDisconnectSchema } from "@paperclipai/shared";
import type { ActivityPublication } from "../services/activity-log.js";
import { once } from "node:events";
import type { SkillSourceDiscoveryEvent } from "@paperclipai/shared";
@@ -52,7 +55,7 @@ import {
listCatalogSkillsOrEmpty,
readCatalogSkillFile,
} from "../services/skills-catalog.js";
import { badRequest, conflict, forbidden, unauthorized, HttpError } from "../errors.js";
import { badRequest, conflict, forbidden, unauthorized, notFound, HttpError } from "../errors.js";
import { assertAuthenticated, assertCompanyAccess, getActorInfo } from "./authz.js";
import { getTelemetryClient } from "../telemetry.js";
import {
@@ -60,7 +63,7 @@ import {
normalizeSkillPolicySourceType,
type SkillPolicyPrincipal,
} from "../services/company-skill-policy.js";
import { authorizationDeniedDetails } from "../services/authorization.js";
import { authorizationDeniedDetails, type AuthorizationActor } from "../services/authorization.js";
import {
normalizeSkillPolicySourceLocator,
type SkillPolicyAction,
@@ -97,9 +100,23 @@ type SkillPolicyResourceInput =
export function companySkillRoutes(db: Db) {
const router = Router();
router.use(skillAccessRequestContext);
router.param("skillId", async (req, _res, next, id) => {
try {
if (req.actor.type === "none") throw unauthorized("Authentication required");
assertCompanyAccess(req, String(req.params.companyId));
if (!await svc.getById(String(req.params.companyId), id)) throw notFound("Skill not found");
next();
} catch (error) { next(error); }
});
router.param("sourceId", async (req, _res, next, id) => {
try { assertCompanyAccess(req, String(req.params.companyId)); await skillAccessService(db).assertSource(String(req.params.companyId), id); next(); }
catch (error) { next(error); }
});
const access = accessService(db);
const svc = companySkillService(db);
const sourceSvc = skillSourceService(db);
const sourceSync = skillSourceSyncService(db);
const issues = issueService(db);
const heartbeat = heartbeatService(db);
const skillPolicies = companySkillPolicyService(db);
@@ -300,8 +317,10 @@ export function companySkillRoutes(db: Db) {
assertCompanyAccess(req, companyId);
const actor = getActorInfo(req);
const publications: ActivityPublication[] = [];
const setup = await projectSkillSetupIdentity(db, companyId, req.actor);
const result = await operation({
actor: skillActor(req),
setup: { userId: setup.refreshUserId, agentId: setup.refreshAgentId, localTrusted: setup.refreshLocalTrusted },
read: connectionId => skillSourceGitHubReader(db, companyId, req.actor, connectionId),
authorize: (action, resource) => assertCanMutateCompanySkills(req, companyId, action, resource),
audit: async (tx, sourceId, action, details) => {
@@ -313,6 +332,21 @@ export function companySkillRoutes(db: Db) {
return result;
}
router.get("/companies/:companyId/skill-sources/project-repositories", async (req, res) => {
const companyId = String(req.params.companyId);
assertCompanyAccess(req, companyId);
res.json(await projectSkillRepositories(db, companyId, req.actor));
});
router.post("/companies/:companyId/skill-sources/adopt-project", validate(projectSkillSourceAdoptSchema), async (req, res) => {
const companyId = String(req.params.companyId);
const result = await sourceOperation(req, companyId, context => adoptProjectSkillRepository(db, companyId, req.body.workspaceId, context.authorize));
publishActivity(result.publication);
res.status(202).json({ bindingId: result.bindingId });
});
router.patch("/companies/:companyId/skill-sources/:sourceId/settings", validate(skillSourceSettingsSchema), async (req, res) => {
const companyId = String(req.params.companyId);
res.json(await sourceOperation(req, companyId, context => sourceSvc.settings(companyId, String(req.params.sourceId), req.body, context)));
});
router.get("/companies/:companyId/skill-sources/repositories", async (req, res) => {
const companyId = req.params.companyId as string;
assertCompanyAccess(req, companyId);
@@ -388,7 +422,7 @@ export function companySkillRoutes(db: Db) {
});
router.post("/companies/:companyId/skill-sources/:sourceId/refresh", async (req, res) => {
const companyId = req.params.companyId as string;
res.json(await sourceOperation(req, companyId, context => sourceSvc.refresh(companyId, req.params.sourceId as string, context)));
res.json(await sourceOperation(req, companyId, context => sourceSync.refreshSource(companyId, req.params.sourceId as string, context)));
});
router.delete("/companies/:companyId/skill-sources/:sourceId", (req, _res, next) => {
assertCompanyAccess(req, req.params.companyId as string);
@@ -702,10 +736,22 @@ export function companySkillRoutes(db: Db) {
assigneeAgentId: req.body.agentId,
});
const actor = getActorInfo(req);
const skill = await svc.getById(companyId, skillId);
let skillProjectId: string | undefined;
if (skill?.accessSourceId) {
const bindings = await skillAccessService(db).projectIds(companyId, skill.accessSourceId);
const testActor: AuthorizationActor = { type: "agent", agentId: req.body.agentId, companyId, source: "agent_key",
onBehalfOfUserId: req.actor.source === "local_implicit" ? null : req.actor.userId ?? req.actor.onBehalfOfUserId };
for (const binding of bindings) {
if ((await access.decide({ actor: testActor, action: "project:read", resource: { type: "project", companyId, projectId: binding.projectId } })).allowed) { skillProjectId = binding.projectId; break; }
}
if (!skillProjectId) throw forbidden("Choose an agent with access to this skill's project.");
}
const result = await svc.createTestRun(companyId, skillId, req.body, skillActor(req), {
createHarnessIssue: async (harnessIssue) => {
const created = await issues.create(companyId, {
...harnessIssue,
...(skillProjectId ? { projectId: skillProjectId, visibility: "private" as const } : {}),
priority: "medium",
createdByAgentId: actor.agentId,
createdByUserId: actor.actorType === "user" ? actor.actorId : null,
server/src/routes/openapi.ts
+9 -2
View File
@@ -172,6 +172,8 @@ import {
skillSourcePreviewSchema,
skillSourceCreateSchema,
skillSourceSelectionSchema,
skillSourceSettingsSchema,
projectSkillSourceAdoptSchema,
companySkillCreateSchema,
companySkillFileDeleteSchema,
companySkillFileUpdateSchema,
@@ -12377,6 +12379,9 @@ registerCurrentRoute({
for (const [method, path, summary] of [
["get", "/api/companies/{companyId}/skill-sources", "List GitHub skill sources"],
["get", "/api/companies/{companyId}/skill-sources/project-repositories", "List accessible project repositories and skill import status"],
["post", "/api/companies/{companyId}/skill-sources/adopt-project", "Import all skills from an accessible project repository"],
["patch", "/api/companies/{companyId}/skill-sources/{sourceId}/settings", "Set automatic skill updates and authorized refresh connection"],
["get", "/api/companies/{companyId}/skill-sources/repositories", "Browse authorized GitHub repositories for skills"],
["get", "/api/companies/{companyId}/skill-sources/{sourceId}", "Get a skill source and entries"],
["post", "/api/companies/{companyId}/skill-sources/discover", "Discover and validate repository skills"],
@@ -12387,12 +12392,14 @@ for (const [method, path, summary] of [
["delete", "/api/companies/{companyId}/skill-sources/{sourceId}", "Disconnect a skill source and retain installed skills"],
] as const) registerCurrentRoute({
method, path, tags: ["skills"], summary,
...(method === "patch" ? { body: skillSourceSelectionSchema }
...(method === "patch" && path.endsWith("/settings") ? { body: skillSourceSettingsSchema }
: method === "post" && path.endsWith("/adopt-project") ? { body: projectSkillSourceAdoptSchema }
: method === "patch" ? { body: skillSourceSelectionSchema }
: method === "post" && path.endsWith("/discover") ? { body: skillSourceDiscoverySchema }
: method === "post" && path.endsWith("/preview") ? { body: skillSourcePreviewSchema }
: method === "post" && path.endsWith("/skill-sources") ? { body: skillSourceCreateSchema } : {}),
responses: {
[method === "post" && path.endsWith("/skill-sources") ? 201 : 200]: path.endsWith("/discover") ? {
[path.endsWith("/adopt-project") ? 202 : method === "post" && path.endsWith("/skill-sources") ? 201 : 200]: path.endsWith("/discover") ? {
description: "JSON discovery by default. Accept: application/x-ndjson streams progress (phase, measured Git download percentages, and package/file counts), candidate metadata, then complete with discovery. An error event terminates a failed scan; partial candidates cannot be imported. Disconnecting cancels further provider reads.",
content: { "application/json": { schema: z.unknown() }, "application/x-ndjson": { schema: z.string() } },
} : r.ok(),
server/src/routes/projects.ts
+17
View File
@@ -1,3 +1,4 @@
import { skillAccessRequestContext } from "../services/skill-access.js";
import { createHash } from "node:crypto";
import { and, eq, sql } from "drizzle-orm";
import { activityLog } from "@paperclipai/db";
@@ -58,6 +59,7 @@ const SHARED_WORKSPACE_STOP_AND_RESTART_ACTIONS = new Set(["stop", "restart"]);
export function projectRoutes(db: Db) {
const router = Router();
router.use(skillAccessRequestContext);
const svc = projectService(db);
async function repositoryViewer(req: Request) {
@@ -167,6 +169,16 @@ export function projectRoutes(db: Db) {
return false;
}
async function assertProjectWorkspaceWriteAllowed(req: Request, res: Response, project: { id: string; companyId: string }) {
const assigneeAgentId = req.actor.type === "agent" ? req.actor.agentId ?? null : null;
const decision = await access.decide({ actor: req.actor, action: "project:write_workspace",
resource: { type: "issue", companyId: project.companyId, projectId: project.id, assigneeAgentId },
scope: { projectId: project.id, assigneeAgentId } });
if (decision.allowed) return true;
res.status(403).json({ error: decision.explanation });
return false;
}
async function assertCanManageProjectPrivacy(req: Request, res: Response, project: {
companyId: string; privacyOwnerUserId?: string | null; personalOwnerUserId?: string | null;
}) {
@@ -286,6 +298,8 @@ export function projectRoutes(db: Db) {
assertBoard(req);
const project = await getAccessibleResource(req, res, svc.getById(req.params.id as string), "Project not found");
if (!project) return;
if (!(await assertProjectReadAllowed(req, res, project))) return;
if (!(await assertProjectWorkspaceWriteAllowed(req, res, project))) return;
const repositories = await selectedRepositories(req, project.companyId, req.body.repositoryIds, project.workspaces);
const updated = await svc.replaceRepositories(project.id, repositories);
const actor = getActorInfo(req);
@@ -600,6 +614,7 @@ export function projectRoutes(db: Db) {
const existing = await getAccessibleResource(req, res, svc.getById(id), "Project not found");
if (!existing) return;
if (!(await assertProjectReadAllowed(req, res, existing))) return;
if (!(await assertProjectWorkspaceWriteAllowed(req, res, existing))) return;
assertNoAgentHostWorkspaceCommandMutation(
req,
collectProjectWorkspaceCommandPaths(req.body),
@@ -640,6 +655,7 @@ export function projectRoutes(db: Db) {
const existing = await getAccessibleResource(req, res, svc.getById(id), "Project not found");
if (!existing) return;
if (!(await assertProjectReadAllowed(req, res, existing))) return;
if (!(await assertProjectWorkspaceWriteAllowed(req, res, existing))) return;
assertNoAgentHostWorkspaceCommandMutation(
req,
collectProjectWorkspaceCommandPaths(req.body),
@@ -962,6 +978,7 @@ export function projectRoutes(db: Db) {
const existing = await getAccessibleResource(req, res, svc.getById(id), "Project not found");
if (!existing) return;
if (!(await assertProjectReadAllowed(req, res, existing))) return;
if (!(await assertProjectWorkspaceWriteAllowed(req, res, existing))) return;
const workspace = await svc.removeWorkspace(id, workspaceId);
if (!workspace) {
res.status(404).json({ error: "Project workspace not found" });
server/src/services/activity.ts
+3 -2
View File
@@ -1,3 +1,4 @@
import { skillAccessService } from "./skill-access.js";
import { credentialAccessConnectionNameSql } from "./credential-access-visibility.js";
import { executionProjectionsForRuns } from "./execution-projection.js";
import { and, asc, desc, eq, inArray, isNull, or, sql, type SQL } from "drizzle-orm";
@@ -354,8 +355,8 @@ export function activityService(db: Db) {
}
return {
list: (filters: ActivityFilters) => {
const conditions = [eq(activityLog.companyId, filters.companyId)];
list: async (filters: ActivityFilters) => {
const conditions = [eq(activityLog.companyId, filters.companyId), await skillAccessService(db).activityCondition(filters.companyId)];
const limit = normalizeActivityLimit(filters.limit);
if (filters.agentId) {
server/src/services/company-portability.ts
+3
View File
@@ -3903,6 +3903,9 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
const allAgentRows = include.agents ? await agents.list(companyId, { includeTerminated: true }) : [];
const agentSelection = resolvePortableExportAgentSelection(allAgentRows, input.agents, include.agents);
const companySkillRowsRaw = include.skills ? await companySkills.listFull(companyId) : [];
if (companySkillRowsRaw.some(skill => skill.accessSourceId)) {
throw unprocessable("Project-restricted skills cannot be exported until the package can preserve their project access.");
}
const managedSkillRows = companySkillRowsRaw.filter((skill) => managedSkillIds.has(skill.id));
const companySkillRows = companySkillRowsRaw.filter((skill) => !managedSkillIds.has(skill.id));
warnings.push(...agentSelection.warnings);
server/src/services/company-skills.ts
+93 -18
View File
@@ -1,3 +1,5 @@
import { retainProjectSkillAccess, skillAccessService } from "./skill-access.js";
import type { AuthorizationActor } from "./authorization.js";
import { updateAgentConfigurationInTransaction } from "./agent-configuration-transaction.js";
import { logger } from "../middleware/logger.js";
import { removeRuntimeSkillCache, resolveRuntimeSkillCache, runtimeSkillCacheSpec } from "./runtime-skill-cache.js";
@@ -5,9 +7,9 @@ import { createHash, randomUUID } from "node:crypto";
import { promises as fs } from "node:fs";
import path from "node:path";
import os from "node:os";
import { skillFileBytes, snapshotFile, assertSkillSnapshotPath } from "./skill-snapshot.js";
import { skillFileBytes, snapshotFile, assertSkillSnapshotPath, skillSnapshotHash } from "./skill-snapshot.js";
import { fileURLToPath } from "node:url";
import { and, asc, desc, eq, inArray, isNull, lt, sql } from "drizzle-orm";
import { and, asc, desc, eq, inArray, isNull, isNotNull, lt, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
agents as agentsTable,
@@ -29,7 +31,8 @@ import {
issueThreadInteractions,
issueWorkProducts,
} from "@paperclipai/db";
import { readPaperclipSkillSyncPreference, writePaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils";
import { readPaperclipSkillSyncPreference, writePaperclipSkillSyncPreference, resolvePaperclipDesiredSkillNames,
resolveLegacyPaperclipDesiredSkillNames } from "@paperclipai/adapter-utils/server-utils";
import { REPOSITORY_CONTENT_DIR, repositorySkillProjection } from "./skill-repository-package.js";
import type { PaperclipDesiredSkillEntry, PaperclipSkillEntry } from "@paperclipai/adapter-utils/server-utils";
import type {
@@ -240,6 +243,7 @@ type ImportedSkillPersistValues = Pick<
| "name"
| "description"
| "markdown"
| "accessSourceId"
| "sourceType"
| "sourceLocator"
| "sourceRef"
@@ -377,6 +381,10 @@ export type ProjectSkillScanTarget = {
type RuntimeSkillEntryOptions = {
materializeMissing?: boolean;
versionSelections?: Map<string, string | null>;
selectionConfig?: Record<string, unknown>;
includeLegacy?: boolean;
onlyKeys?: Set<string>;
snapshotLatest?: boolean;
};
type SkillActor = {
@@ -440,6 +448,7 @@ function selectCompanySkillColumns() {
authorName: companySkills.authorName,
homepageUrl: companySkills.homepageUrl,
categories: companySkills.categories,
accessSourceId: companySkills.accessSourceId,
sharingScope: companySkills.sharingScope,
publicShareToken: companySkills.publicShareToken,
forkedFromSkillId: companySkills.forkedFromSkillId,
@@ -2953,7 +2962,8 @@ async function listLastEditorsBySkillId(
return editors;
}
export function companySkillService(db: Db) {
export function companySkillService(db: Db, options: { actor?: AuthorizationActor } = {}) {
const visibility = (database: DbOrTransaction = db) => skillAccessService(database as Db, options.actor);
const folderSvc = folderService(db);
const agents = agentService(db);
const projects = projectService(db);
@@ -2986,6 +2996,25 @@ export function companySkillService(db: Db) {
remediation: "Import from a configured Paperclip workspace or the company managed-skill directory.",
});
}
// A filesystem import cannot discard the provenance of installed private
// snapshots or their materialized copies, even for an authorized reader.
const restricted = await db.select({ id: companySkills.id, key: companySkills.key, slug: companySkills.slug,
sourceType: companySkills.sourceType, sourceLocator: companySkills.sourceLocator }).from(companySkills).where(and(
eq(companySkills.companyId, companyId), isNotNull(companySkills.accessSourceId)));
for (const skill of restricted) {
const localRoot = skill.sourceType === "local_path" || skill.sourceType === "catalog" ? normalizeSourceLocatorDirectory(skill.sourceLocator) : null;
const protectedRoots = [path.join(resolveManagedSkillsRoot(companyId), "__versions__", skill.id),
resolveRuntimeSkillMaterializedPath(companyId, skill), localRoot].filter((root): root is string => Boolean(root));
for (const root of protectedRoots) {
const canonical = await fs.realpath(root).catch(() => null);
if (canonical && (sourceRealPath === canonical || sourceRealPath.startsWith(`${canonical}${path.sep}`)
|| canonical.startsWith(`${sourceRealPath}${path.sep}`))) {
await visibility().assertSkill(companyId, skill.id);
throw unprocessable("Project-derived skill files cannot be imported as a company-wide local source. Fork the installed skill to retain project access.",
{ code: "skill_project_provenance_required" });
}
}
}
}
async function ensureBundledSkills(companyId: string) {
@@ -3211,6 +3240,8 @@ export function companySkillService(db: Db) {
}
async function ensureSkillInventoryCurrent(companyId: string) {
// Denied reads must not perform the lazy bundled-skill reconciliation.
if (!visibility().canReadCompany(companyId)) return;
const existingRefresh = skillInventoryRefreshPromises.get(companyId);
if (existingRefresh) {
await existingRefresh;
@@ -3243,6 +3274,7 @@ export function companySkillService(db: Db) {
}
async function list(companyId: string, query: CompanySkillListQuery = {}): Promise<CompanySkillListItem[]> {
if (!visibility().canReadCompany(companyId)) return [];
await ensureSkillInventoryCurrent(companyId);
const [dbRows, folderListing] = await Promise.all([db
.select({
@@ -3265,6 +3297,7 @@ export function companySkillService(db: Db) {
authorName: companySkills.authorName,
homepageUrl: companySkills.homepageUrl,
categories: companySkills.categories,
accessSourceId: companySkills.accessSourceId,
sharingScope: companySkills.sharingScope,
publicShareToken: companySkills.publicShareToken,
forkedFromSkillId: companySkills.forkedFromSkillId,
@@ -3278,7 +3311,7 @@ export function companySkillService(db: Db) {
updatedAt: companySkills.updatedAt,
})
.from(companySkills)
.where(eq(companySkills.companyId, companyId))
.where(and(eq(companySkills.companyId, companyId), await visibility().condition(companyId)))
.orderBy(asc(companySkills.name), asc(companySkills.key))
.then((entries) => entries.map((entry) => toCompanySkillListRow(entry as CompanySkillListDbRow))),
folderSvc.list(companyId, "skill"),
@@ -3348,11 +3381,12 @@ export function companySkillService(db: Db) {
}
async function listFull(companyId: string): Promise<CompanySkill[]> {
if (!visibility().canReadCompany(companyId)) return [];
await ensureSkillInventoryCurrent(companyId);
const rows = await db
.select(selectCompanySkillColumns())
.from(companySkills)
.where(eq(companySkills.companyId, companyId))
.where(and(eq(companySkills.companyId, companyId), await visibility().condition(companyId)))
.orderBy(asc(companySkills.name), asc(companySkills.key));
return rows.map((row) => toCompanySkill(row));
}
@@ -3378,7 +3412,7 @@ export function companySkillService(db: Db) {
slug: companySkills.slug,
})
.from(companySkills)
.where(eq(companySkills.companyId, companyId));
.where(and(eq(companySkills.companyId, companyId), await visibility().condition(companyId)));
return rows as CompanySkillReferenceRow[];
}
@@ -3386,7 +3420,7 @@ export function companySkillService(db: Db) {
const row = await database
.select(selectCompanySkillColumns())
.from(companySkills)
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.id, id)))
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.id, id), await visibility(database).condition(companyId)))
.then((rows) => rows[0] ?? null);
return row ? enrichFolderPath(companyId, toCompanySkill(row)) : null;
}
@@ -3395,7 +3429,7 @@ export function companySkillService(db: Db) {
const row = await database
.select(selectCompanySkillColumns())
.from(companySkills)
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.key, key)))
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.key, key), await visibility(database).condition(companyId)))
.then((rows) => rows[0] ?? null);
return row ? enrichFolderPath(companyId, toCompanySkill(row)) : null;
}
@@ -3404,7 +3438,7 @@ export function companySkillService(db: Db) {
const rows = await db
.select(selectCompanySkillColumns())
.from(companySkills)
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.slug, slug)));
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.slug, slug), await visibility().condition(companyId)));
return rows.length === 1 ? enrichFolderPath(companyId, toCompanySkill(rows[0]!)) : null;
}
@@ -3421,6 +3455,7 @@ export function companySkillService(db: Db) {
}
async function getVersion(companyId: string, skillId: string, versionId: string): Promise<CompanySkillVersion | null> {
if (!await getById(companyId, skillId)) return null;
const row = await db
.select()
.from(companySkillVersions)
@@ -3993,16 +4028,17 @@ export function companySkillService(db: Db) {
await ensureSkillInventoryCurrent(companyId);
const source = await getById(companyId, skillId);
if (!source) throw notFound("Skill not found");
const existing = await listFull(companyId);
if (source.accessSourceId && input.sharingScope === "public_link") throw forbidden("Project skills cannot be published through a public link.");
const existing = await db.select({ slug: companySkills.slug }).from(companySkills)
.where(eq(companySkills.companyId, companyId));
const usedSlugs = new Set(existing.map((skill) => normalizeSkillSlug(skill.slug) ?? skill.slug));
const forkSlug = uniqueSkillSlug(normalizeSkillSlug(input.slug ?? `${source.slug}-fork`) ?? `${source.slug}-fork`, usedSlugs);
const forkName = input.name?.trim() || `${source.name} Fork`;
const forkKey = `company/${companyId}/${forkSlug}`;
const plannedReassignments = await planSkillReassignments(companyId, source, forkKey, input.reassignAgentIds);
const managedRoot = resolveManagedSkillsRoot(companyId);
const forkDir = path.resolve(managedRoot, forkSlug);
await fs.rm(forkDir, { recursive: true, force: true });
await fs.mkdir(forkDir, { recursive: true });
await fs.mkdir(managedRoot, { recursive: true });
const forkDir = await fs.mkdtemp(path.join(managedRoot, `${forkSlug}-fork-`));
const sourceVersion = await getCurrentVersion(source);
const repositoryFiles = sourceVersion?.repositorySnapshotId ? await runtimeVersionInventory(companyId, sourceVersion) : null;
for (const entry of repositoryFiles ?? source.fileInventory) {
@@ -4019,7 +4055,9 @@ export function companySkillService(db: Db) {
}
const inventory = await collectLocalSkillInventory(forkDir);
const parsed = parseFrontmatterMarkdown(markdown);
const accessSourceId = await retainProjectSkillAccess(db, companyId, source.accessSourceId);
const metadata = {
accessSourceId,
sourceKind: "managed_local",
forkedFromSkillId: source.id,
forkedFromCompanyId: source.companyId,
@@ -4044,6 +4082,7 @@ export function companySkillService(db: Db) {
await db
.update(companySkills)
.set({
accessSourceId,
iconUrl: source.iconUrl,
color: source.color,
tagline: source.tagline,
@@ -4437,6 +4476,7 @@ export function companySkillService(db: Db) {
const skill = await getById(companyId, skillId);
if (!skill) throw notFound("Skill not found");
if (skill.accessSourceId && input.sharingScope === "public_link") throw forbidden("Project skills cannot be published through a public link.");
const sharingScope = Object.prototype.hasOwnProperty.call(input, "sharingScope")
? normalizeMutableSharingScope(input.sharingScope)
: null;
@@ -4495,6 +4535,9 @@ export function companySkillService(db: Db) {
if (input.forkedFromSkillId && !forkSource) {
throw notFound("Fork source skill not found");
}
if (forkSource?.accessSourceId && input.sharingScope === "public_link") {
throw forbidden("Project skills cannot be published through a public link.");
}
const forkVersion = forkSource ? await getCurrentVersion(forkSource) : null;
const repositoryFiles = forkVersion?.repositorySnapshotId ? await runtimeVersionInventory(companyId, forkVersion) : null;
const sharingScope = normalizeMutableSharingScope(input.sharingScope) ?? "company";
@@ -4576,8 +4619,10 @@ export function companySkillService(db: Db) {
? await collectLocalSkillInventory(skillDir)
: [{ path: "SKILL.md", kind: "skill" as const }];
const parsed = parseFrontmatterMarkdown(markdown);
const accessSourceId = await retainProjectSkillAccess(db, companyId, forkSource?.accessSourceId);
const metadata = {
sourceKind: "managed_local",
...(accessSourceId ? { accessSourceId } : {}),
...(forkSource ? {
forkedFromSkillId: forkSource.id,
forkedFromCompanyId: forkSource.companyId,
@@ -6015,7 +6060,7 @@ export function companySkillService(db: Db) {
options: RuntimeSkillEntryOptions,
): Promise<RuntimeSkillSourceResolution | null> {
const selectedVersionId = options.versionSelections?.get(skill.key)
?? (skill.metadata?.skillSourceId && skill.metadata?.snapshotHash ? skill.currentVersionId : null);
?? (options.snapshotLatest || (skill.metadata?.skillSourceId && skill.metadata?.snapshotHash) || skill.accessSourceId ? skill.currentVersionId : null);
if (selectedVersionId) {
const versionPath = path.resolve(resolveManagedSkillsRoot(companyId), "__versions__", skill.id, selectedVersionId);
const version = await getVersion(companyId, skill.id, selectedVersionId);
@@ -6099,17 +6144,46 @@ export function companySkillService(db: Db) {
options: RuntimeSkillEntryOptions = {},
): Promise<PaperclipSkillEntry[]> {
const skills = await listFull(companyId);
const selectedKeys = options.onlyKeys ?? (options.selectionConfig ? new Set(
(options.includeLegacy ? resolveLegacyPaperclipDesiredSkillNames : resolvePaperclipDesiredSkillNames)(
options.selectionConfig, skills.map(skill => ({ key: skill.key, runtimeName: buildSkillRuntimeName(skill.key, skill.slug) }))),
) : null);
const out: PaperclipSkillEntry[] = [];
for (const skill of skills) {
const sourceResolution = await resolveRuntimeSkillSource(companyId, skill, options);
for (let skill of skills) {
if (selectedKeys && !selectedKeys.has(skill.key)) continue;
let installedResolution: RuntimeSkillSourceResolution | null = null;
const unpinned = !options.versionSelections?.get(skill.key);
const installedDirectory = options.snapshotLatest && unpinned && (skill.sourceType === "local_path" || skill.sourceType === "catalog")
? await resolveExistingSkillDirectory(normalizeSkillDirectory(skill)) : null;
if (options.snapshotLatest && unpinned && (!skill.currentVersionId || installedDirectory)) {
// Capture installed bytes only. Older remote imports can have no
// version yet; execution must not fetch their missing files upstream.
const installed: RuntimeSkillSourceResolution | null = installedDirectory ? { status: "available", source: installedDirectory }
: await resolveRuntimeSkillSource(companyId, skill, { ...options, snapshotLatest: false, materializeMissing: false });
const inventory = installed?.status === "available"
? await collectVersionFileInventoryFromDirectory(installed.source)
: skill.fileInventory.length === 1 && skill.fileInventory[0]?.path === "SKILL.md" && skill.markdown
? [{ path: "SKILL.md", kind: "skill" as const, content: skill.markdown }] : null;
if (inventory) {
const previous = skill.currentVersionId ? await getVersion(companyId, skill.id, skill.currentVersionId) : null;
if (!previous || skillSnapshotHash(previous.fileInventory) !== skillSnapshotHash(inventory)) {
const version = await createVersion(companyId, skill.id, { label: "Execution snapshot" }, { type: "system" },
{ skipInventoryRefresh: true, skill, fileInventory: inventory });
skill = { ...skill, currentVersionId: version.id };
}
} else installedResolution = installed;
}
const sourceResolution = installedResolution ?? await resolveRuntimeSkillSource(companyId, skill, {
...options, ...(options.snapshotLatest ? { materializeMissing: false } : {}),
});
if (!sourceResolution) continue;
out.push({
key: skill.key,
runtimeName: buildSkillRuntimeName(skill.key, skill.slug),
source: sourceResolution.source,
versionId: options.versionSelections?.get(skill.key) ?? (skill.metadata?.skillSourceId && skill.metadata?.snapshotHash ? skill.currentVersionId : null),
versionId: options.versionSelections?.get(skill.key) ?? (options.snapshotLatest || (skill.metadata?.skillSourceId && skill.metadata?.snapshotHash) || skill.accessSourceId ? skill.currentVersionId : null),
currentVersionId: skill.currentVersionId,
sourceStatus: sourceResolution.status,
missingDetail: sourceResolution.status === "missing" ? sourceResolution.detail : null,
@@ -6308,6 +6382,7 @@ export function companySkillService(db: Db) {
name: skill.name,
description: skill.description,
markdown: skill.markdown,
accessSourceId: typeof skill.metadata?.accessSourceId === "string" ? skill.metadata.accessSourceId : existing?.accessSourceId ?? null,
sourceType: skill.sourceType,
sourceLocator: skill.sourceLocator,
sourceRef: skill.sourceRef,
server/src/services/github-skill-source.ts
+27 -2
View File
@@ -14,9 +14,10 @@ export interface GitHubRead {
withScan?: <T>(operation: () => Promise<T>) => Promise<T>;
openSnapshot: (input: { repositoryUrl: string; ref: string; commitSha?: string }, options?: GitSkillSnapshotOptions) => Promise<GitSkillSnapshot>;
readonly connectionId?: string | null;
readonly grantId?: string | null;
}
export type DiscoveredSkill = SkillSourceCandidate & { files: CompanySkillVersionFileInventoryEntry[] };
export type ScannedSkillSource = SkillSourceDiscovery & { skills: DiscoveredSkill[]; defaultBranch: string; repositoryFiles?: CompanySkillVersionFileInventoryEntry[] };
export type ScannedSkillSource = SkillSourceDiscovery & { skills: DiscoveredSkill[]; defaultBranch: string; treeSha?: string; downloadBytes?: number; scanBytes?: number; repositoryFiles?: CompanySkillVersionFileInventoryEntry[] };
const MAX_FILE_BYTES = 1024 * 1024;
const MAX_SCAN_BYTES = 100 * 1024 * 1024;
const MAX_SCAN_PACKAGES = 1_000;
@@ -42,6 +43,28 @@ export function scanGitHubSkills(input: SkillScanInput, providerRead: GitHubRead
return providerRead.withScan ? providerRead.withScan(scan) : scan();
}
/** Oversized repositories may still contain valid, bounded skill packages. */
export async function scanGitHubSkillsWithPackageFallback(input: SkillScanInput, providerRead: GitHubRead, options: SkillScanOptions = {}): Promise<ScannedSkillSource> {
try { return await scanGitHubSkills(input, providerRead, options); }
catch (error) {
if (input.packageMode !== 'repository' || !(error instanceof Error) || !/^Repository package exceeds/.test(error.message)) throw error;
options.signal?.throwIfAborted();
const scan = await scanGitHubSkills({ ...input, packageMode: 'skills' }, providerRead, options);
rejectIncompleteSkillPackages(scan);
scan.warnings.push('Only skill packages were imported because the repository exceeds the supported package size.');
return scan;
}
}
export function rejectIncompleteSkillPackages(scan: ScannedSkillSource): void {
for (const skill of scan.skills) {
if (!skill.error && skill.inspection?.references.length) {
skill.error = 'This skill depends on files that its imported package cannot provide. Its dependencies could not be preserved. Import a supported repository package or include the dependencies in this skill package.';
}
}
scan.candidates = scan.skills.map(({ files: _files, ...candidate }) => candidate);
}
async function scanRepository(input: SkillScanInput, providerRead: GitHubRead, options: SkillScanOptions): Promise<ScannedSkillSource> {
const read = async (apiPath: string) => {
options.signal?.throwIfAborted();
@@ -249,7 +272,9 @@ async function scanRepository(input: SkillScanInput, providerRead: GitHubRead, o
if (skill.inspection) skill.inspection.references = skill.inspection.references.filter(ref => !included.has(ref.resolvedPath));
}
}
return { ...(input.packageMode ? { packageMode: input.packageMode } : {}), ...(repositoryPackage ? { repositoryPackage } : {}),
return { treeSha: snapshot.treeSha, downloadBytes: snapshot.downloadBytes,
scanBytes: Math.min(MAX_SCAN_BYTES, snapshot.entries.reduce((sum, entry) => sum + (entry.type === 'blob' ? entry.size ?? MAX_SCAN_BYTES : 0), 0)),
...(input.packageMode ? { packageMode: input.packageMode } : {}), ...(repositoryPackage ? { repositoryPackage } : {}),
...(repositoryFiles && options.retainFiles !== false ? { repositoryFiles } : {}),
connectionId: providerRead.connectionId ?? null, repositoryId: String(repo.id), repositoryUrl: `https://github.com/${repo.full_name.toLowerCase()}`, fullName: repo.full_name, trackingRef, commitSha: commit.sha,
defaultBranch: repo.default_branch, candidates: skills.map(({ files: _files, ...candidate }) => candidate), warnings, skills };
server/src/services/heartbeat/execution-configuration.ts
+50 -12
View File
@@ -1,7 +1,10 @@
import { withSkillActor } from "../skill-access.js";
import type { AuthorizationActor } from "../authorization.js";
import { eq, sql } from "drizzle-orm";
import { heartbeatRuns, type Db, type agents, type projects } from "@paperclipai/db";
import type { AiConnectionBinding } from "@paperclipai/shared";
import { readPaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils";
import { readPaperclipSkillSyncPreference, resolvePaperclipDesiredSkillNames, resolveLegacyPaperclipDesiredSkillNames,
writePaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils";
import { parseObject } from "../../adapters/utils.js";
import { logger } from "../../middleware/logger.js";
import { AI_CONNECTION_BUSY_RETRY_REASON, extractWakeCommentIds, isNonAssigneeWorkspaceBusyRetry } from "../../modules/run-dispatch/index.js";
@@ -88,12 +91,15 @@ export async function prepareHeartbeatExecutionConfiguration(db: Db, input: Hear
);
const executionRunConfig =
stripWorkspaceRuntimeFromExecutionRunConfig(mergedConfig);
const skillActor: AuthorizationActor = { type: "agent", agentId: agent.id, companyId: agent.companyId,
runId: run.id, source: "agent_jwt", onBehalfOfUserId: responsibleUserId === "local-board" ? null : responsibleUserId };
const runScopedMentionedSkillKeys =
await resolveRunScopedMentionedSkillKeys({
db,
companyId: agent.companyId,
issueId,
commentIds: extractWakeCommentIds(context),
actor: skillActor,
});
const runScopedSkillKeys =
acceptedPlanContinuationWake &&
@@ -199,18 +205,50 @@ export async function prepareHeartbeatExecutionConfiguration(db: Db, input: Hear
);
const nativeRunnerPreparationSpans: NativeRunHistoricalSpan[] = [];
const skillsPrepareStartedAtMs = Date.now();
let skillDeliveryConfig = effectiveResolvedConfig;
const runtimeSkillEntries = await (async () => {
try {
return await companySkills.listRuntimeSkillEntries(agent.companyId, {
versionSelections: skillVersionSelectionMap(
runtimeSkillPreference.desiredSkillEntries,
{
versionPinsEnabled:
resolvedInstanceSettings.experimental.enableBetaSkills ===
true,
},
),
});
const storedVersions = parseObject(run.contextSnapshot?.skillVersions);
const hasStoredVersions = Object.prototype.hasOwnProperty.call(run.contextSnapshot ?? {}, "skillVersions");
const versionSelections = hasStoredVersions
? new Map(Object.entries(storedVersions).filter((entry): entry is [string, string | null] => entry[1] === null || typeof entry[1] === "string"))
: skillVersionSelectionMap(runtimeSkillPreference.desiredSkillEntries, {
versionPinsEnabled: resolvedInstanceSettings.experimental.enableBetaSkills === true,
});
const entries = await withSkillActor(skillActor, () => companySkills.listRuntimeSkillEntries(agent.companyId, {
versionSelections, snapshotLatest: true,
...(hasStoredVersions && run.contextSnapshot?.skillSelectionSnapshot === true
? { onlyKeys: new Set(Object.keys(storedVersions)) }
: { selectionConfig: effectiveResolvedConfig, includeLegacy: agent.adapterType !== "paperclip_runner" }),
}));
const available = hasStoredVersions ? entries.filter(entry => Object.prototype.hasOwnProperty.call(storedVersions, entry.key)) : entries;
const availableKeys = new Set(available.map(entry => entry.key));
const desiredKeys = hasStoredVersions && run.contextSnapshot?.skillSelectionSnapshot === true
? Object.keys(storedVersions)
: agent.adapterType === "paperclip_runner"
? resolvePaperclipDesiredSkillNames(effectiveResolvedConfig, available)
: resolveLegacyPaperclipDesiredSkillNames(effectiveResolvedConfig, available);
const selectedKeys = new Set(desiredKeys.filter(key => availableKeys.has(key)));
const resolved = available.filter(entry => selectedKeys.has(entry.key));
// Optional assignments can outlive project membership. Drop inaccessible
// assignments from delivery; explicit task mentions remain errors below.
skillDeliveryConfig = writePaperclipSkillSyncPreference(effectiveResolvedConfig,
resolved.map(entry => ({ key: entry.key, versionId: entry.versionId ?? null })));
for (const key of runScopedMentionedSkillKeys) {
const entry = resolved.find(candidate => candidate.key === key);
if (!entry || entry.sourceStatus === "missing") {
throw new ConfigurationIncompleteFailure("A referenced skill is unavailable. Restore its installed version or update the task's skill reference.", {
configurationIncomplete: { reason: "skill_unavailable", skillKey: key, actionUrl: "/skills" },
});
}
}
if (!hasStoredVersions) {
const skillVersions = Object.fromEntries(resolved.map(entry => [entry.key, entry.versionId ?? entry.currentVersionId ?? null]));
context.skillVersions = skillVersions;
context.skillSelectionSnapshot = true;
await db.update(heartbeatRuns).set({ contextSnapshot: sql`coalesce(${heartbeatRuns.contextSnapshot}, '{}'::jsonb) || ${JSON.stringify({ skillVersions, skillSelectionSnapshot: true })}::jsonb` }).where(eq(heartbeatRuns.id, run.id));
}
return resolved;
} catch (error) {
if (agent.adapterType === "paperclip_runner") {
await recordFailedSkillPreparation({
@@ -231,7 +269,7 @@ export async function prepareHeartbeatExecutionConfiguration(db: Db, input: Hear
endedAtMs: Date.now(),
});
const connectorAssignments = await resolveConnectorAssignments(db, { companyId: agent.companyId, agentId: agent.id, runId: run.id, issueId: typeof context.issueId === "string" ? context.issueId : undefined });
const connectorSkillConfig = await applyConnectorSkills(effectiveResolvedConfig, runtimeSkillEntries, connectorAssignments);
const connectorSkillConfig = await applyConnectorSkills(skillDeliveryConfig, runtimeSkillEntries, connectorAssignments);
// Both CLI adapters and native context materialization use the same resolved set.
runtimeSkillEntries.splice(0, runtimeSkillEntries.length, ...connectorSkillConfig.paperclipRuntimeSkills);
const connectorDelivery = await prepareConnectorSkillDelivery(connectorSkillConfig, agent.adapterType);
server/src/services/heartbeat/run-preparation.ts
+11
View File
@@ -1,3 +1,5 @@
import { skillAccessService } from "../skill-access.js";
import type { AuthorizationActor } from "../authorization.js";
import { spekoToolsForSession } from "../voice/speko-agent-tools.js";
import { githubConfiguredInstructionSources } from "../chat-github-review-policy.js";
import { CONFIGURED_ENVIRONMENT_KEYS } from "../../vendor/paperclip-runner/index.js";
@@ -727,6 +729,7 @@ export async function resolveRunScopedMentionedSkillKeys(input: {
companyId: string;
issueId: string | null;
commentIds?: string[];
actor?: AuthorizationActor;
}): Promise<string[]> {
if (!input.issueId) return [];
@@ -801,6 +804,14 @@ export async function resolveRunScopedMentionedSkillKeys(input: {
inArray(companySkillsTable.id, mentionedSkillIds),
),
);
for (const skillId of mentionedSkillIds) {
try { await skillAccessService(input.db, input.actor).assertSkill(input.companyId, skillId); }
catch {
throw new ConfigurationIncompleteFailure("A referenced skill is unavailable or you do not have project access. Update the task's skill reference or project access.", {
configurationIncomplete: { reason: "skill_unavailable", companyId: input.companyId, actionUrl: "/skills" },
});
}
}
const skillKeyById = new Map(skillRows.map((row) => [row.id, row.key]));
return mentionedSkillIds
.map((skillId) => skillKeyById.get(skillId) ?? null)
server/src/services/heartbeat/runtime-selection.test.ts
+22
View File
@@ -157,6 +157,28 @@ describe.skipIf(!support.supported)("heartbeat runtime selection boundary", () =
expect((await read(run.id)).runnerProfileJson).toEqual(saved.runnerProfileJson);
});
it("rejects resume after skill access revocation without rewriting the admitted revision", async () => {
const { input, run } = await fixture();
const key = "company/test/private-skill", versionId = randomUUID();
await db.update(heartbeatRuns).set({ contextSnapshot: { skillVersions: { [key]: versionId } } }).where(eq(heartbeatRuns.id, run.id));
input.run = await read(run.id);
vi.mocked(buildNativeRuntimeContext).mockImplementationOnce(async () => {
const context = nativeRuntimeContextFixture();
context.instructions.bundle.rootPath = instructionRoot;
context.skills = [{ key, runtimeName: "private-skill", versionId, bundle: { ...context.instructions.bundle } }];
return { ...context, aggregateDigest: canonicalNativeRuntimeContextDigest(context) };
});
const selected = await selectHeartbeatRuntime(db, input);
if (!selected.selected || !selected.nativeExecution) throw new Error("Expected native selection");
const saved = await read(run.id);
input.run = saved;
input.session.persistedNativeExecutionInput = selected.nativeExecution;
input.config.runtimeSkillEntries = [];
await expect(selectHeartbeatRuntime(db, input)).rejects.toThrow("Project skill access changed");
expect((await read(run.id)).runnerProfileJson).toEqual(saved.runnerProfileJson);
expect((await read(run.id)).contextSnapshot?.skillVersions).toEqual({ [key]: versionId });
});
it.each(["cancelled", "startup_cancellation"])("does not create a native coordinator when %s wins preparation", async winner => {
const { input, run } = await fixture();
input.beforeNativeRuntimeSelection = async () => {
server/src/services/heartbeat/runtime-selection.ts
+10
View File
@@ -536,6 +536,16 @@ export async function selectHeartbeatRuntime(db: Db, input: HeartbeatRuntimeSele
throw new Error(
"native_execution_input_persisted_binding_mismatch",
);
if ("runtimeContext" in nativeExecution) {
const savedVersions = parseObject(run.contextSnapshot?.skillVersions);
const accessibleKeys = new Set(runtimeSkillEntries.filter(entry => entry.sourceStatus !== "missing").map(entry => entry.key));
if (nativeExecution.runtimeContext.skills.some(skill =>
Object.prototype.hasOwnProperty.call(savedVersions, skill.key) && !accessibleKeys.has(skill.key))) {
throw new ConfigurationIncompleteFailure("Project skill access changed while this execution was suspended. Restore access or start a new execution.", {
configurationIncomplete: { reason: "skill_access_changed", actionUrl: "/skills" },
});
}
}
// Recover only the originally admitted request. A stored idle
// checkpoint can precede an already-started provider turn, so even
// apparent idleness is not authority to rewrite its contract.
server/src/services/local-ai-browser-login.test.ts
+2
View File
@@ -63,6 +63,7 @@ function resolveFixtureCodexCommand(home: string): string {
describe.skipIf(process.platform === "win32")("local browser subscription login", () => {
it.each(["native", "arm32", "musl", "glibc-2.38", "glibc-3.0", "unknown"])("uses the selected Codex command with its credential home and ambient HOME/cwd (%s)", async platform => {
const legacy = platform === "arm32";
if (platform === "native") targetPlatform("linux", "x64");
if (platform === "arm32") targetPlatform("linux", "arm");
if (platform === "musl") targetPlatform("linux", "arm64", false);
if (platform === "glibc-2.38") targetPlatform("linux", "arm64", true, "2.38");
@@ -163,6 +164,7 @@ describe.skipIf(process.platform === "win32")("local browser subscription login"
});
it("reports a missing terminal prerequisite without exposing spawn details", async () => {
targetPlatform("linux", "x64");
const home = await fakeCli("codex", "exit 0");
process.env.PATH = path.join(root!, "no-python");
const login = startLocalBrowserLogin("openai", home);
server/src/services/native-runtime/paperclip-runner-tool-authority.ts
+15 -2
View File
@@ -1,3 +1,4 @@
import { skillAccessService } from "../skill-access.js";
import { callTaskWorkspaceTool } from "../task-workspace-tools.js";
import { TaskAttachmentReadCache, TASK_ATTACHMENT_DEFINITIONS, TASK_ATTACHMENT_SCHEMAS, listTaskAttachments, readTaskAttachment } from "./runner-task-attachments.js";
import { RUNNER_BRIDGE_SCHEMAS, runnerBridgeDefinitions, readAssignedSkill, executeWorkspaceTool, readWorkspaceUploadFile, settleRunnerBridgeRead, assertRunnerWorkspacePathAllowed } from "./runner-bridge-tools.js";
@@ -55,6 +56,7 @@ import type { Db } from "@paperclipai/db";
import {
authUsers,
companyMemberships,
companySkills,
activityLog,
assets,
runnerApiResponseReservations,
@@ -350,8 +352,19 @@ export class PaperclipRunnerToolAuthority {
.innerJoin(authUsers, eq(authUsers.id, companyMemberships.principalId))
.where(and(eq(companyMemberships.companyId, this.binding.companyId), eq(companyMemberships.principalType, "user"), eq(companyMemberships.status, "active"), input.after ? sql`${authUsers.id} > ${input.after}` : undefined))
.orderBy(authUsers.id).limit(input.limit);
if (call.tool === "list_assigned_skills") return this.binding.runtimeContext!.skills.map(skill => ({ key: skill.key, name: skill.runtimeName, versionId: skill.versionId, digest: skill.bundle.digest }));
if (call.tool === "read_assigned_skill") return settleRunnerBridgeRead(() => readAssignedSkill(this.binding.runtimeContext!, input));
if (call.tool === "list_assigned_skills" || call.tool === "read_assigned_skill") {
const current = await this.#boundContext();
const visibility = await skillAccessService(this.db, this.#privacyActor(current.run)).condition(this.binding.companyId);
const known = await this.db.select({ key: companySkills.key }).from(companySkills).where(eq(companySkills.companyId, this.binding.companyId));
const permitted = await this.db.select({ key: companySkills.key }).from(companySkills).where(and(eq(companySkills.companyId, this.binding.companyId), visibility));
const hidden = new Set(known.filter(skill => !permitted.some(allowed => allowed.key === skill.key)).map(skill => skill.key));
const savedVersions = current.run.contextSnapshot?.skillVersions;
const permittedKeys = new Set(permitted.map(skill => skill.key));
const runtimeContext = { ...this.binding.runtimeContext!, skills: this.binding.runtimeContext!.skills.filter(skill =>
!hidden.has(skill.key) && !(savedVersions && Object.prototype.hasOwnProperty.call(savedVersions, skill.key) && !permittedKeys.has(skill.key))) };
if (call.tool === "list_assigned_skills") return runtimeContext.skills.map(skill => ({ key: skill.key, name: skill.runtimeName, versionId: skill.versionId, digest: skill.bundle.digest }));
return settleRunnerBridgeRead(() => readAssignedSkill(runtimeContext, input));
}
const paths: Record<string, [string, string]> = { get_task: ["GET", "/api/issues/{id}"], comment_on_task: ["POST", "/api/issues/{id}/comments"],
list_task_documents: ["GET", "/api/issues/{id}/documents"], read_task_document: ["GET", "/api/issues/{id}/documents/{key}"], write_task_document: ["PUT", "/api/issues/{id}/documents/{key}"] };
const [method, path] = paths[call.tool]!;
server/src/services/project-skill-source-bindings.ts
+64
View File
@@ -0,0 +1,64 @@
import { lockSkillImportQueue, MAX_COMPANY_PENDING_SKILL_IMPORTS, MAX_PROJECT_PENDING_SKILL_IMPORTS, SKILL_IMPORT_CAPACITY_WAIT } from './skill-source-import-queue.js';
import { and, eq, ne } from 'drizzle-orm';
import { projectSkillSources, type Db, projectWorkspaces } from '@paperclipai/db';
import { parseGitHubSkillRepositoryUrl } from '@paperclipai/shared';
import { currentSkillActor } from './skill-access.js';
import { captureRunIdentity } from './run-identity.js';
import { forbidden } from '../errors.js';
export async function projectSkillSetupIdentity(db: Db, companyId: string, actor = currentSkillActor()) {
let userId = actor?.type === 'board' ? actor.userId ?? null : null;
let connectionId: string | null = null, refreshGrantId: string | null = null;
if (actor?.type === 'agent' && actor.source === 'agent_jwt') {
if (!actor.runId || !actor.agentId || actor.companyId !== companyId) throw forbidden('Project skill setup requires an authenticated agent run.');
const identity = await captureRunIdentity(db, { companyId, agentId: actor.agentId, runId: actor.runId });
userId = identity.run.responsibleUserId;
// Only reuse the connection already authorized by managed GitHub delivery
// for this active run. Responsibility alone cannot discover another grant.
if (identity.context?.github?.status === 'available') {
connectionId = identity.context.github.connectionId ?? null;
refreshGrantId = identity.context.github.grantId ?? null;
}
}
// An API key can attach repositories, but cannot supply a setup person.
// Keep the durable job so an authorized person can adopt it from Sources.
return { refreshUserId: userId, refreshAgentId: actor?.type === 'agent' ? actor.agentId ?? null : null,
refreshLocalTrusted: actor?.source === 'local_implicit' || userId === 'local-board', connectionId, refreshGrantId };
}
export function projectSkillRepository(workspace: Pick<typeof projectWorkspaces.$inferSelect, 'repoUrl' | 'defaultRef' | 'repoRef'>) {
const url = workspace.repoUrl?.replace(/^git@github\.com:/i, 'https://github.com/');
const parsed = url ? parseGitHubSkillRepositoryUrl(url) : null;
if (!parsed) return null;
return { repositoryUrl: parsed.repositoryUrl.toLowerCase(), trackingRef: workspace.defaultRef ?? workspace.repoRef ?? parsed.trackingRef ?? 'HEAD' };
}
/** Called inside the workspace mutation transaction; no provider calls here. */
export async function enqueueProjectSkillSource(db: Db, workspace: typeof projectWorkspaces.$inferSelect, setupIdentity?: Awaited<ReturnType<typeof projectSkillSetupIdentity>>) {
return db.transaction(async tx => {
const db = tx as unknown as Db;
const repository = projectSkillRepository(workspace);
const setup = repository ? setupIdentity ?? await projectSkillSetupIdentity(db, workspace.companyId) : undefined;
const pending = await lockSkillImportQueue(db, workspace.companyId);
await db.update(projectSkillSources).set({ active: false, lastError: null })
.where(and(eq(projectSkillSources.companyId, workspace.companyId), eq(projectSkillSources.workspaceId, workspace.id),
...(repository ? [ne(projectSkillSources.trackingRef, repository.trackingRef)] : [])));
if (!repository) return null;
await db.update(projectSkillSources).set({ active: false, lastError: null })
.where(and(eq(projectSkillSources.companyId, workspace.companyId), eq(projectSkillSources.workspaceId, workspace.id), ne(projectSkillSources.repositoryUrl, repository.repositoryUrl)));
const [existing] = await db.select().from(projectSkillSources).where(and(eq(projectSkillSources.workspaceId, workspace.id),
eq(projectSkillSources.repositoryUrl, repository.repositoryUrl), eq(projectSkillSources.trackingRef, repository.trackingRef)));
const available = Boolean(existing?.sourceId || pending.some(row => row.id === existing?.id))
|| (pending.length < MAX_COMPANY_PENDING_SKILL_IMPORTS && pending.filter(row => row.projectId === workspace.projectId).length < MAX_PROJECT_PENDING_SKILL_IMPORTS);
const [binding] = await db.insert(projectSkillSources).values({
companyId: workspace.companyId, projectId: workspace.projectId, workspaceId: workspace.id, ...repository, nextSyncAt: new Date(),
...setup, active: available, lastError: available ? null : SKILL_IMPORT_CAPACITY_WAIT,
}).onConflictDoUpdate({ target: [projectSkillSources.workspaceId, projectSkillSources.repositoryUrl, projectSkillSources.trackingRef],
set: { active: available, lastError: available ? existing?.lastError === SKILL_IMPORT_CAPACITY_WAIT ? null : existing?.lastError : SKILL_IMPORT_CAPACITY_WAIT } }).returning();
return binding!;
});
}
export async function detachProjectSkillSource(db: Db, companyId: string, workspaceId: string) {
await db.update(projectSkillSources).set({ active: false, lastError: null }).where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.workspaceId, workspaceId)));
}
server/src/services/projects.ts
+7 -1
View File
@@ -1,3 +1,4 @@
import { enqueueProjectSkillSource, detachProjectSkillSource } from "./project-skill-source-bindings.js";
import { and, asc, desc, eq, gt, inArray, isNull, or, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
@@ -41,7 +42,7 @@ import { lockExecutionWorkspaceRootProvenance, preserveConfiguredProjectWorkspac
type ProjectTransaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
async function lockProjectRootMutation(tx: ProjectTransaction, projectId: string) {
export async function lockProjectRootMutation(tx: ProjectTransaction, projectId: string) {
const [identity] = await tx.select({ companyId: projects.companyId }).from(projects).where(eq(projects.id, projectId));
if (!identity) return null;
await lockExecutionWorkspaceRootProvenance(tx, identity.companyId);
@@ -1174,6 +1175,7 @@ export function projectService(db: Db, options: { captureWorkspaceUpdates?: bool
})
.returning()
.then((rows) => rows[0] ?? null);
if (row) await enqueueProjectSkillSource(tx as unknown as Db, row);
return row;
});
@@ -1285,6 +1287,9 @@ export function projectService(db: Db, options: { captureWorkspaceUpdates?: bool
.returning()
.then((rows) => rows[0] ?? null);
if (!row) return null;
if (row.repoUrl !== existing.repoUrl || row.defaultRef !== existing.defaultRef || row.repoRef !== existing.repoRef) {
await enqueueProjectSkillSource(tx as unknown as Db, row);
}
if (row.isPrimary) return row;
@@ -1364,6 +1369,7 @@ export function projectService(db: Db, options: { captureWorkspaceUpdates?: bool
await preserveConfiguredProjectWorkspaceRootProvenance(tx, existing);
const removed = await db.transaction(async (tx) => {
await detachProjectSkillSource(tx as unknown as Db, existing.companyId, workspaceId);
const row = await tx
.delete(projectWorkspaces)
.where(eq(projectWorkspaces.id, workspaceId))
server/src/services/skill-access.ts
+99
View File
@@ -0,0 +1,99 @@
import { randomUUID } from 'node:crypto';
import { AsyncLocalStorage } from 'node:async_hooks';
import type { RequestHandler } from 'express';
import { and, eq, inArray, isNotNull, isNull, or, sql } from 'drizzle-orm';
import { activityLog, companySkills, companySkillSources, projectSkillSources, projects, type Db } from '@paperclipai/db';
import { authorizationService, projectReadSqlCondition, type AuthorizationActor } from './authorization.js';
import { notFound } from '../errors.js';
// Carries the authenticated identity through shared services, including native API dispatch.
// Background operations must install their stored, revalidated setup identity explicitly.
const actors = new AsyncLocalStorage<AuthorizationActor>();
export const currentSkillActor = () => actors.getStore();
export function withSkillActor<T>(actor: AuthorizationActor, operation: () => T): T {
return actors.run(actor, operation);
}
export const skillAccessRequestContext: RequestHandler = (req, _res, next) => withSkillActor(req.actor, next);
export function skillAccessService(db: Db, actor = currentSkillActor()) {
function canReadCompany(companyId: string) {
return actor?.type !== 'none' && !(actor?.type === 'agent' && actor.companyId !== companyId)
&& !(actor?.type === 'board' && actor.companyIds && !actor.companyIds.includes(companyId)
&& actor.source !== 'local_implicit' && !actor.isInstanceAdmin);
}
async function projectIds(companyId: string, sourceId?: string) {
const bindings = await db.select({ sourceId: projectSkillSources.sourceId, projectId: projectSkillSources.projectId })
.from(projectSkillSources).where(and(eq(projectSkillSources.companyId, companyId), isNotNull(projectSkillSources.sourceId),
...(sourceId ? [eq(projectSkillSources.sourceId, sourceId)] : [])));
if (!actor) return bindings;
const access = authorizationService(db);
const decisions = new Map<string, boolean>();
for (const id of new Set(bindings.map(binding => binding.projectId))) {
decisions.set(id, (await access.decide({ actor, action: 'project:read',
resource: { type: 'project', companyId, projectId: id } })).allowed);
}
return bindings.filter(binding => decisions.get(binding.projectId));
}
async function visibleSourceIds(companyId: string) {
if (!canReadCompany(companyId)) return [];
if (!actor) return null;
const sources = await db.select({ id: companySkillSources.id, scope: companySkillSources.accessScope })
.from(companySkillSources).where(eq(companySkillSources.companyId, companyId));
const permitted = new Set((await projectIds(companyId)).map(binding => binding.sourceId));
return sources.filter(source => source.scope === 'company' || permitted.has(source.id)).map(source => source.id);
}
async function condition(companyId: string) {
if (!canReadCompany(companyId)) return sql`false`;
const ids = await visibleSourceIds(companyId);
if (ids === null) return sql`true`;
return or(isNull(companySkills.accessSourceId), ids.length ? inArray(companySkills.accessSourceId, ids) : sql`false`)!;
}
async function assertSkill(companyId: string, skillId: string) {
const [row] = await db.select({ id: companySkills.id }).from(companySkills)
.where(and(eq(companySkills.companyId, companyId), eq(companySkills.id, skillId), await condition(companyId)));
if (!row) throw notFound('Skill not found');
}
async function assertSource(companyId: string, sourceId: string) {
const ids = await visibleSourceIds(companyId);
if (ids !== null && !ids.includes(sourceId)) throw notFound('Skill source not found');
}
async function activityCondition(companyId: string) {
if (!actor) return sql<boolean>`true`;
const ids = await visibleSourceIds(companyId);
const skillCondition = await condition(companyId);
const projectCondition = await projectReadSqlCondition(db, actor);
return sql<boolean>`(
(${activityLog.entityType} != 'company_skill' or exists (
select 1 from ${companySkills} where ${companySkills.companyId} = ${companyId}
and ${companySkills.id}::text = ${activityLog.entityId} and ${skillCondition}))
and (${activityLog.entityType} != 'company_skill_source' or ${ids?.length ? inArray(activityLog.entityId, ids) : sql`false`})
and (${activityLog.entityType} != 'project' or exists (
select 1 from ${projects} where ${projects.companyId} = ${companyId}
and ${projects.id}::text = ${activityLog.entityId} and ${projectCondition})))`;
}
return { canReadCompany, condition, projectIds, visibleSourceIds, assertSkill, assertSource, activityCondition };
}
/** Independent editable forks retain the originating project set at creation.
* Membership/visibility remain dynamic; later repository attachments grant no access.
*/
export async function retainProjectSkillAccess(db: Db, companyId: string, sourceId: string | null | undefined) {
if (!sourceId) return null;
return db.transaction(async tx => {
const [source] = await tx.select().from(companySkillSources).where(and(
eq(companySkillSources.companyId, companyId), eq(companySkillSources.id, sourceId)));
if (!source || source.removedAt || source.accessScope !== 'projects') return sourceId;
const origins = await tx.select({ projectId: projectSkillSources.projectId }).from(projectSkillSources)
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.sourceId, sourceId)));
const id = randomUUID();
await tx.insert(companySkillSources).values({ id, companyId, repositoryId: source.repositoryId,
repositoryUrl: source.repositoryUrl, fullName: source.fullName, trackingRef: source.trackingRef,
accessScope: 'projects', removedAt: new Date(), enabled: false, automaticUpdates: false });
const projectIds = [...new Set(origins.map(origin => origin.projectId))];
if (projectIds.length) await tx.insert(projectSkillSources).values(projectIds.map(projectId => ({
companyId, projectId, sourceId: id, repositoryUrl: source.repositoryUrl, trackingRef: source.trackingRef,
active: false, status: 'ready' as const,
})));
return id;
});
}
server/src/services/skill-source-git-snapshot.ts
+12 -7
View File
@@ -11,6 +11,8 @@ import { MAX_REPOSITORY_BYTES, MAX_REPOSITORY_FILES } from './skill-repository-p
export type GitSkillTreeEntry = { path: string; type: string; mode: string; sha: string; size?: number };
export interface GitSkillSnapshot {
commitSha: string;
treeSha?: string;
downloadBytes?: number;
entries: GitSkillTreeEntry[];
readBlob: (entry: GitSkillTreeEntry, signal?: AbortSignal) => Promise<Buffer>;
readBlobs?: (entries: GitSkillTreeEntry[], signal?: AbortSignal) => Promise<Map<string, Buffer>>;
@@ -26,7 +28,7 @@ const MAX_DOWNLOAD_BYTES = 128 * 1024 * 1024;
const MAX_TREE_BYTES = 32 * 1024 * 1024;
const CACHE_TTL_MS = 10 * 60_000;
const MAX_CACHED_SNAPSHOTS = 4;
const snapshots = new Map<string, { directory: string; commitSha: string; entries: GitSkillTreeEntry[]; users: number; expires: number; timer?: NodeJS.Timeout }>();
const snapshots = new Map<string, { directory: string; commitSha: string; treeSha: string; entries: GitSkillTreeEntry[]; users: number; expires: number; timer?: NodeJS.Timeout }>();
let activeDownloads = 0;
/** Only structured numeric progress crosses the process boundary; never forward stderr. */
@@ -180,11 +182,11 @@ async function discard(key: string) {
await rm(cached.directory, { recursive: true, force: true });
}
function acquire(cached: NonNullable<ReturnType<typeof snapshots.get>>, key: string): GitSkillSnapshot {
function acquire(cached: NonNullable<ReturnType<typeof snapshots.get>>, key: string, downloadBytes = 0): GitSkillSnapshot {
cached.users++;
clearTimeout(cached.timer);
let released = false;
return { commitSha: cached.commitSha, entries: cached.entries,
return { commitSha: cached.commitSha, treeSha: cached.treeSha, downloadBytes, entries: cached.entries,
readBlob: async (entry, signal) => {
if (released || !/^[a-f0-9]{40}$/.test(entry.sha) || entry.type !== 'blob' || (entry.size ?? Infinity) > 1024 * 1024) throw unprocessable('Invalid snapshot file.');
return runGit(cached.directory, ['cat-file', 'blob', entry.sha], '', { signal, maxOutput: 1024 * 1024 });
@@ -253,17 +255,20 @@ export async function openGitSkillSnapshot(input: { repositoryUrl: string; ref:
if (!input.commitSha && !/^[a-f0-9]{40}$/i.test(input.ref)) await runGit(directory, ['check-ref-format', '--branch', input.ref], '', options);
await runGit(directory, ['-c', 'fetch.unpackLimit=1', 'fetch', '--depth=1', '--no-tags', '--no-recurse-submodules', '--no-auto-maintenance', '--progress',
'--', `${parsed.repositoryUrl}.git`, input.commitSha ?? input.ref], input.token, { ...options, download: true });
if (await directoryBytes(directory) > MAX_DOWNLOAD_BYTES) throw unprocessable('Repository download exceeds the 128 MB limit. Use a smaller skills repository.');
const downloadBytes = await directoryBytes(directory);
if (downloadBytes > MAX_DOWNLOAD_BYTES) throw unprocessable('Repository download exceeds the 128 MB limit. Use a smaller skills repository.');
const commitSha = (await runGit(directory, ['rev-parse', '--verify', 'FETCH_HEAD^{commit}'], '', options)).toString('utf8').trim();
if (!/^[a-f0-9]{40}$/.test(commitSha) || (input.commitSha && commitSha !== input.commitSha.toLowerCase())) throw unprocessable('The downloaded commit did not match the requested snapshot.');
const treeSha = (await runGit(directory, ['rev-parse', '--verify', `${commitSha}^{tree}`], '', options)).toString('utf8').trim();
if (!/^[a-f0-9]{40}$/.test(treeSha)) throw unprocessable('Git did not return an immutable tree.');
const entries = parseTree(await runGit(directory, ['ls-tree', '-r', '-l', '-z', '--full-tree', commitSha], '', options));
const key = `${prefix}${commitSha}`;
const duplicate = snapshots.get(key);
if (duplicate) return acquire(duplicate, key);
const cached = { directory, commitSha, entries, users: 0, expires: Date.now() + CACHE_TTL_MS };
if (duplicate) return acquire(duplicate, key, downloadBytes);
const cached = { directory, commitSha, treeSha, entries, users: 0, expires: Date.now() + CACHE_TTL_MS };
snapshots.set(key, cached);
directory = undefined;
return acquire(cached, key);
return acquire(cached, key, downloadBytes);
} finally {
activeDownloads--;
if (directory) await rm(directory, { recursive: true, force: true });
server/src/services/skill-source-github-access.ts
+13 -3
View File
@@ -15,7 +15,7 @@ function providerStatus(error: unknown): number | undefined {
}
/** Automatic access considers only this caller's grants, never connection management visibility. */
export function skillSourceGitHubReader(db: Db, companyId: string, actor: Request['actor'], connectionId: string | null): GitHubRead {
export function skillSourceGitHubReader(db: Db, companyId: string, actor: Request['actor'], connectionId: string | null, options: { grantId?: string | null } = {}): GitHubRead {
let chosen: Authorization | undefined;
const access = toolAccessService(db);
const localTrusted = actor.type === 'board' && actor.source === 'local_implicit';
@@ -44,7 +44,10 @@ export function skillSourceGitHubReader(db: Db, companyId: string, actor: Reques
const ids = connectionId ? [connectionId] : await access.githubReadConnectionIds(companyId, actor.userId ?? null, localTrusted);
const result: Authorization[] = [];
for (const id of ids) {
for (const grantId of await access.githubReadGrantIds(companyId, id, actor.userId ?? null, localTrusted)) result.push({ connectionId: id, grantId });
for (const grantId of await access.githubReadGrantIds(companyId, id, actor.userId ?? null, localTrusted)) {
if (options.grantId && options.grantId !== grantId) continue;
result.push({ connectionId: id, grantId });
}
}
result.sort((a, b) => Number(b.connectionId === chosen?.connectionId && b.grantId === chosen?.grantId) - Number(a.connectionId === chosen?.connectionId && a.grantId === chosen?.grantId));
if (!connectionId) result.push({ connectionId: null });
@@ -100,7 +103,13 @@ export function skillSourceGitHubReader(db: Db, companyId: string, actor: Reques
: response.status === 404 ? 'Repository, branch, or file is unavailable. Check the URL and GitHub repository access.'
: [401, 403].includes(response.status) ? 'GitHub denied repository access. Reconnect GitHub or choose an authorized connection.'
: 'GitHub could not complete the read. Try again.';
throw unprocessable(message, { code: limited ? 'skill_source_github_rate_limited' : 'skill_source_github_read_failed', status: response.status });
const retryAfter = response.headers.get('retry-after');
const resetAt = Number(response.headers.get('x-ratelimit-reset')) * 1_000;
const retryAfterMs = retryAfter && /^\d+$/.test(retryAfter)
? Number(retryAfter) * 1_000
: retryAfter ? Date.parse(retryAfter) - Date.now() : resetAt - Date.now();
throw unprocessable(message, { code: limited ? 'skill_source_github_rate_limited' : 'skill_source_github_read_failed', status: response.status,
...(limited && Number.isFinite(retryAfterMs) && retryAfterMs > 0 ? { retryAfterMs } : {}) });
}
return response.json();
}, signal);
@@ -129,6 +138,7 @@ export function skillSourceGitHubReader(db: Db, companyId: string, actor: Reques
throw error;
}
};
Object.defineProperty(read, 'grantId', { get: () => chosen?.grantId ?? null });
Object.defineProperty(read, 'connectionId', { get: () => chosen?.connectionId ?? null });
return read;
}
server/src/services/skill-source-import-budget.ts
+32
View File
@@ -0,0 +1,32 @@
import { and, eq, isNull, sql } from 'drizzle-orm';
import { companySkillImportBudgets, projectSkillSources, type Db } from '@paperclipai/db';
import { conflict, tooManyRequests } from '../errors.js';
import { automaticSkillWorkAvailable, automaticSkillWorkRefund, AUTOMATIC_SKILL_WORK_CAPACITY_BYTES,
AUTOMATIC_SKILL_WORK_RESERVATION_BYTES, REFILL_WINDOW_MS } from './skill-source-refresh-budget.js';
/** Reserve before provider work; failed attempts retain their reservation. */
export async function reserveInitialSkillWork(db: Db, binding: typeof projectSkillSources.$inferSelect, token: string, now: Date) {
await db.transaction(async tx => {
const [current] = await tx.select({ id: projectSkillSources.id }).from(projectSkillSources).where(and(
eq(projectSkillSources.companyId, binding.companyId), eq(projectSkillSources.id, binding.id),
eq(projectSkillSources.leaseToken, token), eq(projectSkillSources.active, true), isNull(projectSkillSources.sourceId))).for('no key update');
if (!current) throw conflict('Repository import changed. Retry from Skills.');
await tx.insert(companySkillImportBudgets).values({ companyId: binding.companyId, updatedAt: now }).onConflictDoNothing();
const [budget] = await tx.select().from(companySkillImportBudgets).where(eq(companySkillImportBudgets.companyId, binding.companyId)).for('update');
const effectiveNow = new Date(Math.max(now.getTime(), budget!.updatedAt.getTime()));
const available = automaticSkillWorkAvailable(budget!.availableBytes, budget!.updatedAt, effectiveNow);
if (available < AUTOMATIC_SKILL_WORK_RESERVATION_BYTES) throw tooManyRequests('Automatic repository import is temporarily limited. It will resume when work credits refill.', {
code: 'skill_source_initial_work_limited',
retryAfterMs: Math.ceil((AUTOMATIC_SKILL_WORK_RESERVATION_BYTES - available) * REFILL_WINDOW_MS / AUTOMATIC_SKILL_WORK_CAPACITY_BYTES),
});
await tx.update(companySkillImportBudgets).set({ availableBytes: available - AUTOMATIC_SKILL_WORK_RESERVATION_BYTES, updatedAt: effectiveNow })
.where(eq(companySkillImportBudgets.companyId, binding.companyId));
});
}
/** Called once, inside the transaction that successfully links the binding. */
export async function settleInitialSkillWork(db: Db, companyId: string, work: { downloadBytes?: number; scanBytes?: number }) {
const refund = automaticSkillWorkRefund(work);
if (refund) await db.update(companySkillImportBudgets).set({ availableBytes: sql`least(${AUTOMATIC_SKILL_WORK_CAPACITY_BYTES}, ${companySkillImportBudgets.availableBytes} + ${refund})` })
.where(eq(companySkillImportBudgets.companyId, companyId));
}
server/src/services/skill-source-import-queue.ts
+44
View File
@@ -0,0 +1,44 @@
import { and, asc, eq, isNull, isNotNull, notInArray } from 'drizzle-orm';
import { companies, projectSkillSources, projectWorkspaces, type Db } from '@paperclipai/db';
export const MAX_COMPANY_PENDING_SKILL_IMPORTS = 100;
export const MAX_PROJECT_PENDING_SKILL_IMPORTS = 20;
export const SKILL_IMPORT_CAPACITY_WAIT = 'Waiting for earlier repository imports to finish.';
/** Call inside the attachment transaction, before locking any binding rows. */
export async function lockSkillImportQueue(db: Db, companyId: string) {
await db.select({ id: companies.id }).from(companies).where(eq(companies.id, companyId)).for('no key update');
return db.select({ projectId: projectSkillSources.projectId, id: projectSkillSources.id }).from(projectSkillSources)
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.active, true), isNull(projectSkillSources.sourceId)))
.limit(MAX_COMPANY_PENDING_SKILL_IMPORTS);
}
/** Deferred provenance stays durable; free slots admit it without user ceremony. */
export async function admitDeferredSkillImports(db: Db, companyId: string, now: Date) {
return db.transaction(async tx => {
const pending = await lockSkillImportQueue(tx as unknown as Db, companyId);
let count = pending.length;
const perProject = new Map<string, number>();
for (const row of pending) perProject.set(row.projectId, (perProject.get(row.projectId) ?? 0) + 1);
if (count >= MAX_COMPANY_PENDING_SKILL_IMPORTS) return;
const fullProjects = [...perProject].filter(([, count]) => count >= MAX_PROJECT_PENDING_SKILL_IMPORTS).map(([id]) => id);
const waiting = await tx.select({ id: projectSkillSources.id, projectId: projectSkillSources.projectId })
.from(projectSkillSources).innerJoin(projectWorkspaces, eq(projectWorkspaces.id, projectSkillSources.workspaceId))
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.active, false),
isNull(projectSkillSources.sourceId), eq(projectSkillSources.status, 'pending'),
eq(projectSkillSources.lastError, SKILL_IMPORT_CAPACITY_WAIT),
...(fullProjects.length ? [notInArray(projectSkillSources.projectId, fullProjects)] : [])))
.orderBy(asc(projectSkillSources.createdAt)).limit(MAX_COMPANY_PENDING_SKILL_IMPORTS);
for (const row of waiting) {
if (count >= MAX_COMPANY_PENDING_SKILL_IMPORTS) break;
if ((perProject.get(row.projectId) ?? 0) >= MAX_PROJECT_PENDING_SKILL_IMPORTS) continue;
await tx.update(projectSkillSources).set({ active: true, lastError: null, nextSyncAt: now })
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.id, row.id),
isNotNull(projectSkillSources.workspaceId), eq(projectSkillSources.active, false),
isNull(projectSkillSources.sourceId), eq(projectSkillSources.status, 'pending'),
eq(projectSkillSources.lastError, SKILL_IMPORT_CAPACITY_WAIT)));
count++;
perProject.set(row.projectId, (perProject.get(row.projectId) ?? 0) + 1);
}
});
}
server/src/services/skill-source-refresh-budget.ts
+43
View File
@@ -0,0 +1,43 @@
import { and, eq, sql } from 'drizzle-orm';
import { companySkillSources, type Db } from '@paperclipai/db';
import { conflict, tooManyRequests } from '../errors.js';
export const AUTOMATIC_SKILL_WORK_CAPACITY_BYTES = 512 * 1024 * 1024;
export const AUTOMATIC_SKILL_WORK_RESERVATION_BYTES = 128 * 1024 * 1024;
export const REFILL_WINDOW_MS = 60 * 60_000;
export function automaticSkillWorkAvailable(bytes: number, updatedAt: Date, now: Date) {
const elapsed = Math.max(0, now.getTime() - updatedAt.getTime());
return Math.min(AUTOMATIC_SKILL_WORK_CAPACITY_BYTES,
bytes + Math.floor(Math.min(elapsed, REFILL_WINDOW_MS) * AUTOMATIC_SKILL_WORK_CAPACITY_BYTES / REFILL_WINDOW_MS));
}
export function automaticSkillWorkRefund(work: { downloadBytes?: number; scanBytes?: number }) {
return AUTOMATIC_SKILL_WORK_RESERVATION_BYTES - Math.ceil(Math.min(AUTOMATIC_SKILL_WORK_RESERVATION_BYTES,
Math.max(work.downloadBytes ?? AUTOMATIC_SKILL_WORK_RESERVATION_BYTES, work.scanBytes ?? AUTOMATIC_SKILL_WORK_RESERVATION_BYTES, 0)));
}
/** The refresh lease serializes this durable token bucket across workers/restarts. */
export async function reserveAutomaticSkillWork(db: Db, source: typeof companySkillSources.$inferSelect, token: string, now: Date) {
const effectiveNow = new Date(Math.max(now.getTime(), source.automaticBudgetAt.getTime()));
const available = automaticSkillWorkAvailable(source.automaticBudgetBytes, source.automaticBudgetAt, effectiveNow);
if (available < AUTOMATIC_SKILL_WORK_RESERVATION_BYTES) {
throw tooManyRequests('Automatic refresh is temporarily limited. Use Refresh now, or wait for automatic updates to resume.', {
code: 'skill_source_automatic_work_limited',
retryAfterSeconds: Math.ceil((AUTOMATIC_SKILL_WORK_RESERVATION_BYTES - available) * REFILL_WINDOW_MS / AUTOMATIC_SKILL_WORK_CAPACITY_BYTES / 1000),
});
}
const changed = await db.update(companySkillSources).set({ automaticBudgetAt: effectiveNow,
automaticBudgetBytes: available - AUTOMATIC_SKILL_WORK_RESERVATION_BYTES })
.where(and(eq(companySkillSources.companyId, source.companyId), eq(companySkillSources.id, source.id),
eq(companySkillSources.leaseToken, token))).returning({ id: companySkillSources.id });
if (!changed.length) throw conflict('Source refresh changed. Try again.');
}
/** Small repositories refund unused capacity; failures keep the bounded reservation. */
export async function settleAutomaticSkillWork(db: Db, companyId: string, sourceId: string, token: string,
work: { downloadBytes?: number; scanBytes?: number }) {
const refund = automaticSkillWorkRefund(work);
if (!refund) return;
await db.update(companySkillSources).set({ automaticBudgetBytes: sql`least(${AUTOMATIC_SKILL_WORK_CAPACITY_BYTES}, ${companySkillSources.automaticBudgetBytes} + ${refund})` })
.where(and(eq(companySkillSources.companyId, companyId), eq(companySkillSources.id, sourceId), eq(companySkillSources.leaseToken, token)));
}
server/src/services/skill-source-retry.ts
+9
View File
@@ -0,0 +1,9 @@
/** Share retry scheduling between initial imports and subsequent refreshes. */
export function skillSourceRetryAt(error: unknown, failureCount: number, now = new Date()) {
const backoff = Math.min(60 * 60_000, 5 * 60_000 * 2 ** Math.min(failureCount, 4));
const providerDelay = error instanceof Error && 'details' in error
? (error.details as { retryAfterMs?: unknown } | null)?.retryAfterMs : undefined;
const delay = typeof providerDelay === 'number' && Number.isFinite(providerDelay)
? Math.max(backoff, Math.min(24 * 60 * 60_000, providerDelay)) : backoff;
return new Date(now.getTime() + delay);
}
server/src/services/skill-source-sync.ts
+314
View File
@@ -0,0 +1,314 @@
import { lockProjectRootMutation } from './projects.js';
import { reserveInitialSkillWork, settleInitialSkillWork } from './skill-source-import-budget.js';
import { admitDeferredSkillImports, lockSkillImportQueue, SKILL_IMPORT_CAPACITY_WAIT } from './skill-source-import-queue.js';
import { randomUUID } from 'node:crypto';
import { skillSourceRetryAt } from './skill-source-retry.js';
import { and, asc, eq, inArray, isNotNull, isNull, lte, or, sql } from 'drizzle-orm';
import { activityLog, agents, authUsers, heartbeatRuns, companies, companyMemberships, companySkillSources, companySkillSourceEntries,
projectSkillSources, projectWorkspaces, projects, type Db } from '@paperclipai/db';
import type { ProjectSkillRepository } from '@paperclipai/shared';
import { authorizationService, type AuthorizationActor } from './authorization.js';
import { skillSourceGitHubReader } from './skill-source-github-access.js';
import { skillSourceService, type SkillSourceContext } from './skill-sources.js';
import { scanGitHubSkillsWithPackageFallback } from './github-skill-source.js';
import { companySkillPolicyService } from './company-skill-policy.js';
import { persistActivity, publishActivity } from './activity-log.js';
import type { ActivityPublication } from './activity-log.js';
import { currentSkillActor, skillAccessService } from './skill-access.js';
import { enqueueProjectSkillSource, projectSkillRepository, projectSkillSetupIdentity } from './project-skill-source-bindings.js';
import { conflict, forbidden, notFound } from '../errors.js';
import { loadConfig } from '../config.js';
export const SKILL_SOURCE_REFRESH_MS = 5 * 60_000;
type Setup = { companyId: string; refreshUserId: string | null; refreshAgentId: string | null;
refreshLocalTrusted: boolean; refreshGrantId: string | null; connectionId: string | null };
export async function projectSkillRepositories(db: Db, companyId: string, actor: AuthorizationActor): Promise<ProjectSkillRepository[]> {
const rows = await db.select({ workspace: projectWorkspaces, project: projects }).from(projectWorkspaces)
.innerJoin(projects, and(eq(projects.id, projectWorkspaces.projectId), eq(projects.companyId, companyId)))
.where(eq(projectWorkspaces.companyId, companyId)).orderBy(asc(projects.name), asc(projectWorkspaces.name));
const bindings = await db.select().from(projectSkillSources).where(eq(projectSkillSources.companyId, companyId));
const removed = new Set((await db.select({ id: companySkillSources.id }).from(companySkillSources)
.where(and(eq(companySkillSources.companyId, companyId), sql`${companySkillSources.removedAt} is not null`))).map(source => source.id));
const allowed = new Map<string, boolean>();
const access = authorizationService(db);
const result: ProjectSkillRepository[] = [];
for (const { workspace, project } of rows) {
const repo = projectSkillRepository(workspace);
if (!repo) continue;
if (!allowed.has(project.id)) allowed.set(project.id, (await access.decide({ actor, action: 'project:read',
resource: { type: 'project', companyId, projectId: project.id } })).allowed);
if (!allowed.get(project.id)) continue;
const binding = bindings.find(item => (item.active || (!item.sourceId && item.lastError !== null)) && item.workspaceId === workspace.id && item.repositoryUrl === repo.repositoryUrl && item.trackingRef === repo.trackingRef);
result.push({ projectId: project.id, projectName: project.name, projectVisibility: project.visibility as 'open' | 'private',
workspaceId: workspace.id, ...repo, bindingId: binding?.id ?? null, sourceId: binding?.sourceId && !removed.has(binding.sourceId) ? binding.sourceId : null,
status: binding?.sourceId && removed.has(binding.sourceId) ? 'available' : binding?.status ?? 'available', lastError: binding?.lastError ?? null });
}
return result;
}
export async function adoptProjectSkillRepository(db: Db, companyId: string, workspaceId: string, authorize: SkillSourceContext['authorize']) {
const actor = currentSkillActor();
if (!actor) throw forbidden('Authentication required');
const [workspace] = await db.select().from(projectWorkspaces).where(and(eq(projectWorkspaces.companyId, companyId), eq(projectWorkspaces.id, workspaceId)));
if (!workspace || !(await authorizationService(db).decide({ actor, action: 'project:read', resource: { type: 'project', companyId, projectId: workspace.projectId } })).allowed) throw notFound('Project repository not found');
const repository = projectSkillRepository(workspace);
if (!repository) throw notFound('Project repository not found');
await authorize('skills.import', { sourceType: 'git', sourceLocator: repository.repositoryUrl });
return db.transaction(async tx => {
const database = tx as unknown as Db;
// Follow the editor: root provenance, project, workspace, then queue/binding.
const project = await lockProjectRootMutation(tx, workspace.projectId);
if (!project || project.companyId !== companyId) throw notFound('Project repository not found');
const [currentWorkspace] = await tx.select().from(projectWorkspaces).where(and(
eq(projectWorkspaces.companyId, companyId), eq(projectWorkspaces.projectId, project.id), eq(projectWorkspaces.id, workspaceId))).for('update');
if (!currentWorkspace) throw notFound('Project repository not found');
const currentRepository = projectSkillRepository(currentWorkspace);
if (!currentRepository || currentRepository.repositoryUrl !== repository.repositoryUrl || currentRepository.trackingRef !== repository.trackingRef) {
throw conflict('Project repository changed. Reload Sources and try again.');
}
if (!(await authorizationService(database).decide({ actor, action: 'project:read', resource: { type: 'project', companyId, projectId: project.id } })).allowed) throw notFound('Project repository not found');
// Capture the setup before the company queue, consistently with attachment.
const setup = await projectSkillSetupIdentity(database, companyId);
await lockSkillImportQueue(database, companyId);
const [previous] = await tx.select({ bindingId: projectSkillSources.id, removedAt: companySkillSources.removedAt })
.from(projectSkillSources).innerJoin(companySkillSources, eq(companySkillSources.id, projectSkillSources.sourceId))
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.workspaceId, workspace.id),
eq(projectSkillSources.repositoryUrl, repository.repositoryUrl), eq(projectSkillSources.trackingRef, repository.trackingRef)))
.for('update');
if (previous?.removedAt) {
// Retain the original project's access provenance, but free this workspace
// for a new tracking identity. Never attach new readers to retained copies.
await tx.update(projectSkillSources).set({ workspaceId: null, active: false })
.where(eq(projectSkillSources.id, previous.bindingId));
}
const binding = await enqueueProjectSkillSource(database, currentWorkspace, setup);
if (!binding) throw notFound('Project repository not found');
if (!binding.sourceId) await tx.update(projectSkillSources).set({ status: 'pending', nextSyncAt: new Date(), failureCount: 0, lastError: binding.lastError === SKILL_IMPORT_CAPACITY_WAIT ? SKILL_IMPORT_CAPACITY_WAIT : null,
...setup,
}).where(eq(projectSkillSources.id, binding.id));
const { publication } = await persistActivity(database, { companyId, actorType: actor.type === 'agent' ? 'agent' : 'user',
actorId: actor.agentId ?? actor.userId ?? 'board', agentId: actor.agentId, runId: actor.runId,
action: 'project.skills_import_requested', entityType: 'project', entityId: workspace.projectId, details: { workspaceId, bindingId: binding.id } });
// Publish only after the transaction completes (caller's route refreshes query state).
return { bindingId: binding.id, publication };
});
}
function setupActor(setup: Setup): AuthorizationActor {
return setup.refreshAgentId ? { type: 'agent', agentId: setup.refreshAgentId, companyId: setup.companyId,
onBehalfOfUserId: setup.refreshUserId === 'local-board' ? null : setup.refreshUserId, source: 'agent_key' }
: { type: 'board', userId: setup.refreshUserId, source: setup.refreshLocalTrusted ? 'local_implicit' : 'session', companyIds: [setup.companyId] };
}
async function setupContext(db: Db, setup: Setup): Promise<SkillSourceContext> {
if (!setup.refreshUserId && !setup.refreshLocalTrusted) throw forbidden('Choose a GitHub connection to enable automatic updates.');
if (!setup.refreshLocalTrusted) {
const [member] = await db.select({ id: companyMemberships.id }).from(companyMemberships)
.innerJoin(authUsers, eq(authUsers.id, companyMemberships.principalId)).where(and(
eq(companyMemberships.companyId, setup.companyId), eq(companyMemberships.principalType, 'user'),
eq(companyMemberships.principalId, setup.refreshUserId!), eq(companyMemberships.status, 'active')));
if (!member) throw forbidden('The person who connected this source no longer has company access. Choose another GitHub connection.');
}
if (setup.refreshAgentId) {
const [agent] = await db.select({ status: agents.status }).from(agents).where(and(eq(agents.companyId, setup.companyId), eq(agents.id, setup.refreshAgentId)));
if (!agent || agent.status === 'terminated') throw forbidden('The agent who connected this source is unavailable. Choose a GitHub connection.');
}
if (setup.refreshAgentId && (!setup.connectionId || !setup.refreshGrantId)) throw forbidden('Choose a GitHub connection to authorize this agent-created repository.');
const readerActor: AuthorizationActor = { type: 'board', userId: setup.refreshUserId,
source: setup.refreshLocalTrusted ? 'local_implicit' : 'session', companyIds: [setup.companyId] };
const actor = setupActor(setup);
const access = authorizationService(db), policies = companySkillPolicyService(db);
const principal = setup.refreshAgentId ? await policies.resolveAgentPrincipal(setup.companyId, setup.refreshAgentId)
: { type: 'board' as const, id: setup.refreshUserId ?? 'board', role: 'board' };
const readers = new Map<string, ReturnType<typeof skillSourceGitHubReader>>();
const publications: ActivityPublication[] = [];
return {
actor: { type: 'system', userId: setup.refreshUserId, agentId: setup.refreshAgentId },
setup: { userId: setup.refreshUserId, agentId: setup.refreshAgentId, localTrusted: setup.refreshLocalTrusted, grantId: setup.refreshGrantId },
read: connectionId => {
const key = connectionId ?? '';
for (const reader of readers.values()) if (reader.connectionId && reader.connectionId === connectionId) return reader;
if (!readers.has(key)) readers.set(key, skillSourceGitHubReader(db, setup.companyId, readerActor as Parameters<typeof skillSourceGitHubReader>[2], connectionId, { grantId: setup.refreshGrantId }));
return readers.get(key)!;
},
authorize: async (action, resource) => {
const platform = await access.decide({ actor, action: 'skill_config:update', resource: { type: 'company', companyId: setup.companyId } });
if (!platform.allowed && !['deny_no_grant', 'deny_missing_consent', 'deny_missing_grant'].includes(platform.reason)) throw forbidden('Automatic skill updates are unavailable for the setup identity.');
if (resource.skillId) await skillAccessService(db, actor).assertSkill(setup.companyId, resource.skillId);
const decision = await policies.evaluate({ companyId: setup.companyId, principal, action, resource });
if (!decision.allowed) throw forbidden('Automatic skill updates are denied by company policy.');
},
audit: async (tx, sourceId, action, details) => {
const { publication } = await persistActivity(tx as unknown as Db, { companyId: setup.companyId, actorType: 'system', actorId: 'skill-source-sync',
agentId: setup.refreshAgentId, action, entityType: 'company_skill_source', entityId: sourceId,
details: { ...details, refreshUserId: setup.refreshUserId } });
publications.push(publication);
},
publishCommittedActivity: () => { for (const publication of publications.splice(0)) publishActivity(publication); },
};
}
export function skillSourceSyncService(db: Db, options: { context?: (setup: Setup) => Promise<SkillSourceContext> } = {}) {
const contextFor = options.context ?? (setup => setupContext(db, setup));
const sources = skillSourceService(db);
let running: Promise<{ imported: number; checked: number }> | null = null;
const shutdown = new AbortController();
async function recoverSetup(source: typeof companySkillSources.$inferSelect) {
if (source.refreshUserId || source.refreshLocalTrusted || source.refreshAgentId) return source;
const installed = await db.select({ id: companySkillSourceEntries.skillId }).from(companySkillSourceEntries)
.where(and(eq(companySkillSourceEntries.companyId, source.companyId), eq(companySkillSourceEntries.sourceId, source.id)));
const skillIds = installed.map(entry => entry.id).filter((id): id is string => Boolean(id));
const [event] = await db.select().from(activityLog).where(and(eq(activityLog.companyId, source.companyId),
or(eq(activityLog.actorType, 'user'), and(eq(activityLog.actorType, 'agent'), isNotNull(activityLog.runId))),
inArray(activityLog.action, ['company.skill_source_refreshed', 'company.skill_imported', 'company.skills_imported']),
or(eq(activityLog.entityId, source.id), ...(skillIds.length ? [inArray(activityLog.entityId, skillIds)] : []))))
.orderBy(asc(activityLog.createdAt)).limit(1);
if (!event || !event.actorId || event.actorId === 'board') return source;
let userId = event.actorId;
if (event.actorType === 'agent') {
const [run] = await db.select({ userId: heartbeatRuns.responsibleUserId }).from(heartbeatRuns).where(and(
eq(heartbeatRuns.companyId, source.companyId), eq(heartbeatRuns.id, event.runId!),
eq(heartbeatRuns.agentId, event.agentId ?? event.actorId)));
if (!run?.userId) return source;
userId = run.userId;
}
if (!userId) return source;
const localTrusted = userId === 'local-board' && loadConfig().deploymentMode === 'local_trusted';
const [updated] = await db.update(companySkillSources).set({ refreshUserId: userId,
refreshAgentId: event.actorType === 'agent' ? event.agentId ?? event.actorId : null,
refreshLocalTrusted: localTrusted,
}).where(and(eq(companySkillSources.id, source.id), isNull(companySkillSources.refreshUserId))).returning();
return updated ?? (await db.select().from(companySkillSources).where(eq(companySkillSources.id, source.id)))[0]!;
}
async function importBinding(binding: typeof projectSkillSources.$inferSelect, now: Date) {
const token = randomUUID();
const [claimed] = await db.update(projectSkillSources).set({ leaseToken: token, leaseExpiresAt: new Date(now.getTime() + 10 * 60_000) })
.where(and(eq(projectSkillSources.id, binding.id), eq(projectSkillSources.active, true), isNull(projectSkillSources.sourceId),
or(isNull(projectSkillSources.leaseToken), lte(projectSkillSources.leaseExpiresAt, now)))).returning();
if (!claimed) return false;
try {
if (!options.context) {
const actor = setupActor(binding);
if (!(await authorizationService(db).decide({ actor, action: 'project:read', resource: { type: 'project', companyId: binding.companyId, projectId: binding.projectId } })).allowed) throw forbidden('Project access changed. Choose an authorized GitHub connection to retry.');
}
const context = await contextFor(binding);
await context.authorize('skills.import', { sourceType: 'git', sourceLocator: binding.repositoryUrl });
await reserveInitialSkillWork(db, binding, token, now);
const reader = context.read(binding.connectionId);
const scan = await scanGitHubSkillsWithPackageFallback({ repositoryUrl: binding.repositoryUrl, trackingRef: binding.trackingRef, packageMode: 'repository' }, reader, { signal: shutdown.signal });
const [existing] = await db.select().from(companySkillSources).where(and(eq(companySkillSources.companyId, binding.companyId),
eq(companySkillSources.repositoryUrl, scan.repositoryUrl), eq(companySkillSources.trackingRef, binding.trackingRef), isNull(companySkillSources.removedAt)));
let sourceId: string;
if (existing) {
await context.authorize('skills.import', { sourceType: 'git', sourceLocator: existing.repositoryUrl });
sourceId = existing.id;
} else {
try {
const result = await sources.create(binding.companyId, { repositoryUrl: scan.repositoryUrl, trackingRef: binding.trackingRef,
commitSha: scan.commitSha, packageMode: scan.packageMode, selectedPaths: scan.candidates.map(candidate => candidate.path),
connectionId: scan.connectionId ?? binding.connectionId }, { ...context, projectBindingId: binding.id, projectBindingLeaseToken: token }, scan);
sourceId = result.source.id;
} catch (error) {
if (!(error instanceof Error && 'status' in error && error.status === 409)) throw error;
const [concurrent] = await db.select().from(companySkillSources).where(and(eq(companySkillSources.companyId, binding.companyId), eq(companySkillSources.repositoryUrl, scan.repositoryUrl), eq(companySkillSources.trackingRef, binding.trackingRef), isNull(companySkillSources.removedAt)));
if (!concurrent) throw error;
sourceId = concurrent.id;
}
}
return await db.transaction(async tx => {
// Removal locks the source before its bindings. Follow the same order
// and recheck provenance while that lock prevents concurrent removal.
const [source] = await tx.select().from(companySkillSources).where(and(
eq(companySkillSources.companyId, binding.companyId), eq(companySkillSources.id, sourceId))).for('update');
if (!source || source.removedAt) throw conflict('Source was removed during import. Retry to start independent tracking.');
const linked = await tx.update(projectSkillSources).set({ sourceId, status: 'ready', lastError: null, failureCount: 0, leaseToken: null, leaseExpiresAt: null })
.where(and(eq(projectSkillSources.id, binding.id), eq(projectSkillSources.companyId, binding.companyId),
eq(projectSkillSources.active, true), eq(projectSkillSources.leaseToken, token))).returning({ id: projectSkillSources.id });
if (linked.length) await settleInitialSkillWork(tx as unknown as Db, binding.companyId, scan.packageMode === 'skills' ? {} : scan);
return linked.length > 0;
});
} catch (error) {
const message = shutdown.signal.aborted ? binding.lastError : error instanceof Error && 'status' in error ? error.message : 'Skill import failed. Retry from Skills.';
await db.update(projectSkillSources).set({ status: shutdown.signal.aborted ? 'pending' : 'error', lastError: message, leaseToken: null, leaseExpiresAt: null,
...(error instanceof Error && 'status' in error && error.status === 403 ? { active: false } : {}),
failureCount: shutdown.signal.aborted ? binding.failureCount : binding.failureCount + 1, nextSyncAt: shutdown.signal.aborted ? now : skillSourceRetryAt(error, binding.failureCount, now) })
.where(and(eq(projectSkillSources.id, binding.id), eq(projectSkillSources.leaseToken, token)));
return false;
}
}
async function sweep(now: Date) {
const activeCompanies = await db.select({ id: companies.id }).from(companies).where(eq(companies.status, 'active'));
const ids = activeCompanies.map(company => company.id);
if (!ids.length) return { imported: 0, checked: 0 };
for (const id of ids) await admitDeferredSkillImports(db, id, now);
const pending = await db.select().from(projectSkillSources).where(and(inArray(projectSkillSources.companyId, ids),
eq(projectSkillSources.active, true), isNull(projectSkillSources.sourceId), lte(projectSkillSources.nextSyncAt, now),
or(isNull(projectSkillSources.leaseToken), lte(projectSkillSources.leaseExpiresAt, now))))
.orderBy(asc(projectSkillSources.nextSyncAt)).limit(10);
let imported = 0, checked = 0;
for (const binding of pending) { if (shutdown.signal.aborted) break; if (await importBinding(binding, now)) imported++; }
const due = await db.select().from(companySkillSources).where(and(inArray(companySkillSources.companyId, ids),
eq(companySkillSources.enabled, true), eq(companySkillSources.automaticUpdates, true), lte(companySkillSources.nextSyncAt, now),
or(isNull(companySkillSources.leaseToken), lte(companySkillSources.leaseExpiresAt, now)),
or(eq(companySkillSources.accessScope, 'company'), sql`exists (select 1 from ${projectSkillSources} b where b.source_id = ${companySkillSources.id} and b.company_id = ${companySkillSources.companyId} and b.active)`)))
.orderBy(asc(companySkillSources.nextSyncAt)).limit(10);
for (const entry of due) {
if (shutdown.signal.aborted) break;
const source = await recoverSetup(entry);
try {
const context = await contextFor(source);
if (!options.context) {
try { await skillAccessService(db, setupActor(source)).assertSource(source.companyId, source.id); }
catch (error) {
if (!(error instanceof Error && 'status' in error && error.status === 404)) throw error;
throw forbidden('The person who connected this source no longer has project access. Choose another GitHub connection.');
}
}
await sources.refresh(source.companyId, source.id, context, undefined, undefined, { checkCommit: true, automatic: true, now, signal: shutdown.signal });
checked++;
} catch (error) {
if (error instanceof Error && 'status' in error && error.status === 409) continue;
// Failures in context or policy authorization precede the refresh lease.
// Compare the attempt and revision so another worker's result wins.
if (!shutdown.signal.aborted) {
await db.update(companySkillSources).set({ lastError: error instanceof Error && 'status' in error ? error.message : 'Automatic update failed. The installed skills were kept.',
failureCount: source.failureCount + 1, nextSyncAt: skillSourceRetryAt(error, source.failureCount, now), lastAttemptAt: now })
.where(and(eq(companySkillSources.id, source.id), eq(companySkillSources.revision, source.revision),
eq(companySkillSources.failureCount, source.failureCount),
source.lastAttemptAt ? eq(companySkillSources.lastAttemptAt, source.lastAttemptAt) : isNull(companySkillSources.lastAttemptAt),
or(isNull(companySkillSources.leaseToken), lte(companySkillSources.leaseExpiresAt, now))));
}
}
}
return { imported, checked };
}
async function refreshSource(companyId: string, sourceId: string, initiator: SkillSourceContext) {
const [source] = await db.select().from(companySkillSources).where(and(eq(companySkillSources.companyId, companyId), eq(companySkillSources.id, sourceId)));
if (!source) throw notFound('Skill source not found');
await initiator.authorize('skills.update', { sourceType: 'git', sourceLocator: source.repositoryUrl });
if (!source.refreshUserId && !source.refreshLocalTrusted) return sources.refresh(companyId, sourceId, initiator);
const setup = await contextFor(source);
if (!options.context) {
try { await skillAccessService(db, setupActor(source)).assertSource(companyId, sourceId); }
catch (error) {
if (!(error instanceof Error && 'status' in error && error.status === 404)) throw error;
throw forbidden('The person who connected this source no longer has project access. Choose another GitHub connection.');
}
}
return sources.refresh(companyId, sourceId, { ...setup, actor: initiator.actor,
publishCommittedActivity: initiator.publishCommittedActivity,
authorize: async (action, resource) => { await initiator.authorize(action, resource); await setup.authorize(action, resource); },
audit: async (tx, id, action, details) => initiator.audit(tx, id, action, { ...details, refreshUserId: source.refreshUserId }),
});
}
function tick(now = new Date()) {
if (shutdown.signal.aborted) return Promise.resolve({ imported: 0, checked: 0 });
if (running) return running;
running = sweep(now).finally(() => { running = null; });
return running;
}
return { tick, refreshSource, stop: async () => { shutdown.abort(); await running; } };
}
server/src/services/skill-sources.ts
+145 -28
View File
@@ -1,23 +1,30 @@
import { reserveAutomaticSkillWork, settleAutomaticSkillWork } from './skill-source-refresh-budget.js';
import { createHash, randomUUID } from 'node:crypto';
import { skillSourceRetryAt } from './skill-source-retry.js';
import path from 'node:path';
import { promises as fs } from 'node:fs';
import { folderService } from './folders.js';
import { and, eq, isNull, lt, or, sql } from 'drizzle-orm';
import { companySkillSources as sources, companySkillSourceEntries as entries, companySkills, companySkillRepositorySnapshots, authUsers, type Db } from '@paperclipai/db';
import type { CompanySkill, SkillSourceDisconnectRequest, SkillSourceDisconnectResult, SkillSource, SkillSourceCreateRequest, SkillSourceDiscoveryRequest, SkillSourceSelectionRequest, SkillSourceRefreshResult, SkillSourcePreviewRequest } from '@paperclipai/shared';
import { companySkillSources as sources, companySkillSourceEntries as entries, companySkills, companySkillRepositorySnapshots, projectSkillSources, projects, type Db, authUsers } from '@paperclipai/db';
import type { CompanySkill, SkillSource, SkillSourceCreateRequest, SkillSourceDiscoveryRequest, SkillSourceSelectionRequest, SkillSourceRefreshResult, SkillSourcePreviewRequest, SkillSourceSettingsRequest, SkillSourceDisconnectRequest, SkillSourceDisconnectResult } from '@paperclipai/shared';
import { normalizeAgentUrlKey } from '@paperclipai/shared';
import { conflict, notFound, unprocessable } from '../errors.js';
import { companySkillService, parseSkillImportSourceInput } from './company-skills.js';
import { scanGitHubSkills, previewGitHubSkillFile, type GitHubRead, type ScannedSkillSource, type SkillScanOptions } from './github-skill-source.js';
import { scanGitHubSkillsWithPackageFallback as scanGitHubSkills, rejectIncompleteSkillPackages, previewGitHubSkillFile, type GitHubRead, type ScannedSkillSource, type SkillScanOptions } from './github-skill-source.js';
import { skillAccessService } from './skill-access.js';
import { skillSnapshotHash } from './skill-snapshot.js';
type Tx = Parameters<Parameters<Db['transaction']>[0]>[0];
type SourceRow = typeof sources.$inferSelect;
export type SkillSourceContext = {
actor: { type: 'user' | 'agent' | 'system'; userId?: string | null; agentId?: string | null };
setup?: { userId: string | null; agentId?: string | null; localTrusted: boolean; grantId?: string | null };
projectBindingId?: string;
projectBindingLeaseToken?: string;
read: (connectionId: string | null) => GitHubRead;
authorize: (action: 'skills.import' | 'skills.update' | 'skills.edit' | 'skills.remove' | 'skills.create', resource: { skillId?: string; skillKey?: string; sourceType: 'git'; sourceLocator: string }) => Promise<void>;
audit: (tx: Tx, sourceId: string, action: string, details: Record<string, unknown>) => Promise<void>;
publishCommittedActivity?: () => void;
};
const scope = (companyId: string, id: string) => and(eq(sources.companyId, companyId), eq(sources.id, id));
@@ -25,17 +32,32 @@ export function skillSourceService(db: Db) {
const skills = companySkillService(db);
async function row(companyId: string, id: string, database: Db | Tx = db) {
const [source] = await database.select().from(sources).where(scope(companyId, id));
if (!source) throw notFound('Skill source not found.');
if (!source || source.removedAt) throw notFound('Skill source not found.');
return source;
}
async function detail(companyId: string, id: string, database: Db | Tx = db): Promise<SkillSource> {
const { leaseToken: _token, leaseExpiresAt: _expires, createdAt: _created, ...source } = await row(companyId, id, database);
await skillAccessService(database as Db).assertSource(companyId, id);
const { leaseToken, leaseExpiresAt, createdAt: _created, removedAt: _removed, refreshUserId: _user, refreshAgentId: _agent, refreshLocalTrusted: _local, refreshGrantId: _grant, failureCount: _failures, nextSyncAt: _next, lastScanTreeSha: _tree, automaticBudgetBytes: _bytes, automaticBudgetAt: _budgetAt, ...source } = await row(companyId, id, database);
const permitted = await skillAccessService(database as Db).projectIds(companyId, id);
const associated = await database.select({ id: projects.id, name: projects.name, visibility: projects.visibility, active: projectSkillSources.active })
.from(projectSkillSources).innerJoin(projects, eq(projects.id, projectSkillSources.projectId))
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.sourceId, id)));
const visibleProjects = new Map<string, typeof associated[number]>();
for (const project of associated) {
if (!permitted.some(binding => binding.projectId === project.id)) continue;
const previous = visibleProjects.get(project.id);
visibleProjects.set(project.id, { ...project, active: project.active || Boolean(previous?.active) });
}
const refreshing = leaseToken && leaseExpiresAt && leaseExpiresAt > new Date();
const trackingActive = source.accessScope === 'company' || associated.some(project => project.active);
const selected = await database.select().from(entries).where(and(eq(entries.companyId, companyId), eq(entries.sourceId, id))).orderBy(entries.path);
return { ...source, entries: selected };
return { ...source, trackingActive, entries: selected, projects: [...visibleProjects.values()].map(project => ({ ...project, visibility: project.visibility as 'open' | 'private' })),
syncState: !source.enabled || !source.automaticUpdates || !trackingActive ? 'paused' : refreshing ? 'refreshing' : source.lastError ? 'error' : source.lastSuccessAt ? 'ready' : 'pending' };
}
async function list(companyId: string) {
const rows = await db.select({ id: sources.id }).from(sources).where(eq(sources.companyId, companyId)).orderBy(sources.fullName);
return Promise.all(rows.map(source => detail(companyId, source.id)));
const rows = await db.select({ id: sources.id }).from(sources).where(and(eq(sources.companyId, companyId), isNull(sources.removedAt))).orderBy(sources.fullName);
const visible = await skillAccessService(db).visibleSourceIds(companyId);
return Promise.all(rows.filter(source => visible === null || visible.includes(source.id)).map(source => detail(companyId, source.id)));
}
async function sourceForSkill(companyId: string, skillId: string) {
const [entry] = await db.select().from(entries).where(and(eq(entries.companyId, companyId), eq(entries.skillId, skillId)));
@@ -43,7 +65,7 @@ export function skillSourceService(db: Db) {
}
async function discover(input: SkillSourceDiscoveryRequest, context: SkillSourceContext, options?: SkillScanOptions) {
await context.authorize('skills.import', { sourceType: 'git', sourceLocator: input.repositoryUrl });
const { skills: _files, repositoryFiles: _repositoryFiles, defaultBranch: _defaultBranch, ...result } = await scanGitHubSkills(input, context.read(input.connectionId ?? null), { ...options, retainFiles: false });
const { skills: _files, repositoryFiles: _repositoryFiles, defaultBranch: _defaultBranch, treeSha: _tree, downloadBytes: _download, scanBytes: _scan, ...result } = await scanGitHubSkills(input, context.read(input.connectionId ?? null), { ...options, retainFiles: false });
return result;
}
async function preview(input: SkillSourcePreviewRequest, context: SkillSourceContext) {
@@ -56,7 +78,7 @@ export function skillSourceService(db: Db) {
const old = previous.find(entry => entry.path === candidate.path);
const skill = old?.skillId ? await skills.getById(source.companyId, old.skillId) : null;
await context.authorize(skill ? 'skills.update' : 'skills.import', {
sourceType: 'git', sourceLocator: source.repositoryUrl, ...(skill ? { skillId: skill.id, skillKey: skill.key } : { skillKey: newSkillKey(scan, candidate.path, candidate.name, source.id) }),
sourceType: 'git', sourceLocator: source.repositoryUrl, ...(skill ? { skillId: skill.id, skillKey: skill.key } : { skillKey: newSkillKey(scan, candidate.path, candidate.name, source.id, source.revision) }),
});
}
}
@@ -67,17 +89,19 @@ export function skillSourceService(db: Db) {
if (skill) await context.authorize('skills.edit', { sourceType: 'git', sourceLocator: source.repositoryUrl, skillId: skill.id, skillKey: skill.key });
}
}
function newSkillKey(scan: ScannedSkillSource, skillPath: string, name: string, sourceId: string) {
const identity = createHash('sha256').update(`${scan.repositoryId}:${sourceId}:${skillPath}`).digest('hex').slice(0, 16);
function newSkillKey(scan: ScannedSkillSource, skillPath: string, name: string, sourceId: string, revision = 0) {
// New imports after removal must not overwrite retained editable copies.
// Existing entries keep their installed identity across later revisions.
const identity = createHash('sha256').update(`${scan.repositoryId}:${sourceId}${revision ? `:${revision}` : ''}:${skillPath}`).digest('hex').slice(0, 16);
return `github/${scan.repositoryId}/${identity}/${normalizeAgentUrlKey(name) || 'skill'}`;
}
async function publish(tx: Tx, source: SourceRow, scan: ScannedSkillSource, selectedPaths: string[], excludedFolders: string[], context: SkillSourceContext, selectionReviewed: boolean): Promise<Omit<SkillSourceRefreshResult, 'source'>> {
const mode = scan.packageMode ?? 'skills';
if (mode === 'repository' && (scan.repositoryPackage?.error || !scan.repositoryFiles?.length)) {
if (mode === 'repository' && scan.skills.length > 0 && (scan.repositoryPackage?.error || !scan.repositoryFiles?.length)) {
throw unprocessable(scan.repositoryPackage?.error ?? 'Repository package has no files. Scan it again before importing.');
}
let repositorySnapshotId: string | undefined;
const repositoryHash = mode === 'repository' ? skillSnapshotHash(scan.repositoryFiles!) : null;
const repositoryHash = mode === 'repository' && scan.repositoryFiles?.length && !scan.repositoryPackage?.error ? skillSnapshotHash(scan.repositoryFiles) : null;
if (repositoryHash) {
await tx.insert(companySkillRepositorySnapshots).values({ companyId: source.companyId, contentHash: repositoryHash, fileInventory: scan.repositoryFiles! })
.onConflictDoNothing();
@@ -103,7 +127,7 @@ export function skillSourceService(db: Db) {
if (selection === 'selected' && !candidate.error) {
const hash = repositoryHash ? createHash('sha256').update(`${repositoryHash}:${candidate.path}`).digest('hex') : skillSnapshotHash(candidate.files);
const slug = normalizeAgentUrlKey(candidate.name) || 'skill';
const key = current?.key ?? newSkillKey(scan, candidate.path, candidate.name, source.id);
const key = current?.key ?? newSkillKey(scan, candidate.path, candidate.name, source.id, source.revision);
const ownerRepo = scan.fullName.split('/');
const metadata = { ...(current?.metadata ?? {}), sourceKind: 'github', hostname: 'github.com',
owner: ownerRepo[0], repo: ownerRepo[1], ref: current?.metadata?.snapshotHash === hash && current.currentVersionId ? current.sourceRef : scan.commitSha, trackingRef: scan.trackingRef,
@@ -111,6 +135,7 @@ export function skillSourceService(db: Db) {
skillSourceId: source.id, skillSourcePath: candidate.path, skillSourceState: 'synced', snapshotHash: hash, packageMode: mode };
const trustFiles = mode === 'repository' ? scan.repositoryFiles! : candidate.files;
const values = { name: candidate.name, description: candidate.description, markdown: candidate.files.find(file => file.path === 'SKILL.md')!.content,
accessSourceId: source.accessScope === 'projects' ? source.id : current?.accessSourceId ?? null,
sourceType: 'github', sourceLocator: scan.repositoryUrl, sourceRef: current?.metadata?.snapshotHash === hash && current.currentVersionId ? current.sourceRef : scan.commitSha,
fileInventory: candidate.files.map(({ path, kind }) => ({ path, kind })),
trustLevel: trustFiles.some(file => file.kind === 'script' || file.executable) ? 'scripts_executables' : trustFiles.some(file => file.kind === 'asset' || file.kind === 'other') ? 'assets' : 'markdown_only',
@@ -160,17 +185,20 @@ export function skillSourceService(db: Db) {
eq(sources.companyId, source.companyId), eq(sources.trackingRef, scan.trackingRef),
or(eq(sources.repositoryId, scan.repositoryId), eq(sources.repositoryUrl, scan.repositoryUrl)),
)) : [];
const [defaultBinding] = source.trackingRef === 'HEAD' ? await tx.select({ id: projectSkillSources.id }).from(projectSkillSources)
.where(and(eq(projectSkillSources.companyId, source.companyId), eq(projectSkillSources.sourceId, source.id), eq(projectSkillSources.trackingRef, 'HEAD'))).limit(1) : [];
await tx.update(sources).set({ repositoryId: scan.repositoryId, repositoryUrl: scan.repositoryUrl, fullName: scan.fullName,
packageMode: mode, repositoryPackage: scan.repositoryPackage ?? null,
trackingRef: existingBranch ? 'HEAD' : scan.trackingRef, excludedFolders, lastSuccessAt: new Date(), lastScanCommit: scan.commitSha, lastError: null,
revision: source.revision + 1, leaseToken: null, leaseExpiresAt: null }).where(scope(source.companyId, source.id));
trackingRef: existingBranch || defaultBinding || (source.accessScope === 'projects' && source.trackingRef === 'HEAD') ? 'HEAD' : scan.trackingRef, excludedFolders, lastSuccessAt: new Date(), lastScanCommit: scan.commitSha, lastScanTreeSha: scan.treeSha ?? null, lastError: null,
revision: source.revision + 1, leaseToken: null, leaseExpiresAt: null, failureCount: 0, nextSyncAt: new Date(Date.now() + 5 * 60_000) }).where(scope(source.companyId, source.id));
await context.audit(tx, source.id, 'company.skill_source_refreshed', { commit: scan.commitSha, importedCount: imported.length, updatedCount: updated.length, unchanged, warningCount: warnings.length });
return { imported, updated, unchanged, warnings: [...new Set(warnings)] };
}
async function create(companyId: string, input: SkillSourceCreateRequest, context: SkillSourceContext, staged?: ScannedSkillSource) {
await context.authorize('skills.import', { sourceType: 'git', sourceLocator: input.repositoryUrl });
const scan = staged ?? await scanGitHubSkills(input, context.read(input.connectionId ?? null));
if (scan.trackingRef === scan.defaultBranch && (await list(companyId)).some(source => source.repositoryUrl.toLowerCase() === scan.repositoryUrl.toLowerCase() && source.trackingRef === 'HEAD')) {
const reader = context.read(input.connectionId ?? null);
const scan = staged ?? await scanGitHubSkills(input, reader);
if (!context.projectBindingId && scan.trackingRef === scan.defaultBranch && (await list(companyId)).some(source => source.repositoryUrl.toLowerCase() === scan.repositoryUrl.toLowerCase() && source.trackingRef === 'HEAD')) {
throw conflict('This repository is already in Sources. Manage its skills there.');
}
const id = randomUUID();
@@ -179,15 +207,24 @@ export function skillSourceService(db: Db) {
await context.authorize('skills.import', { sourceType: 'git', sourceLocator: scan.repositoryUrl,
skillKey: newSkillKey(scan, candidate.path, candidate.name, id) });
}
return db.transaction(async tx => {
const published = await db.transaction(async tx => {
const [source] = await tx.insert(sources).values({ id, companyId, repositoryId: scan.repositoryId, repositoryUrl: scan.repositoryUrl, fullName: scan.fullName,
packageMode: scan.packageMode ?? 'skills', trackingRef: scan.trackingRef, connectionId: scan.connectionId ?? input.connectionId ?? null, lastAttemptAt: new Date() }).onConflictDoNothing().returning();
packageMode: scan.packageMode ?? 'skills', trackingRef: context.projectBindingId && input.trackingRef === 'HEAD' ? 'HEAD' : scan.trackingRef, connectionId: scan.connectionId ?? input.connectionId ?? null, lastAttemptAt: new Date(),
accessScope: context.projectBindingId ? 'projects' : 'company', importNewSkills: Boolean(context.projectBindingId),
refreshUserId: context.setup?.userId, refreshAgentId: context.setup?.agentId, refreshLocalTrusted: context.setup?.localTrusted ?? false, refreshGrantId: reader.grantId ?? context.setup?.grantId }).onConflictDoNothing().returning();
if (!source) throw conflict('This repository and branch are already in Sources. Manage its skills there.');
if (context.projectBindingId) {
const [binding] = await tx.select().from(projectSkillSources).where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.id, context.projectBindingId))).for('update');
if (!binding?.active || (context.projectBindingLeaseToken && binding.leaseToken !== context.projectBindingLeaseToken)) throw conflict('Project repository changed during import.');
await tx.update(projectSkillSources).set({ sourceId: source.id, status: 'ready', lastError: null }).where(eq(projectSkillSources.id, binding.id));
}
const result = await publish(tx, source, scan, input.selectedPaths, input.excludedFolders ?? [], context, true);
return { ...result, source: await detail(companyId, id, tx) };
});
context.publishCommittedActivity?.();
return published;
}
async function refresh(companyId: string, id: string, context: SkillSourceContext, selection?: SkillSourceSelectionRequest, staged?: ScannedSkillSource) {
async function refresh(companyId: string, id: string, context: SkillSourceContext, selection?: SkillSourceSelectionRequest, staged?: ScannedSkillSource, options: { checkCommit?: boolean; automatic?: boolean; now?: Date; signal?: AbortSignal } = {}) {
let source = await row(companyId, id);
if (!source.enabled && !selection) throw conflict('This source is disconnected. Save its selection to reconnect.');
await context.authorize(selection ? 'skills.edit' : 'skills.update', { sourceType: 'git', sourceLocator: source.repositoryUrl });
@@ -200,25 +237,94 @@ export function skillSourceService(db: Db) {
source = leased;
try {
const connectionId = selection?.connectionId !== undefined ? selection.connectionId : source.connectionId;
const scan = staged ?? await scanGitHubSkills({ repositoryUrl: source.repositoryUrl, trackingRef: source.trackingRef, packageMode: selection?.packageMode ?? source.packageMode }, context.read(connectionId));
const reader = context.read(connectionId);
if (options.checkCommit && source.repositoryId && source.lastScanCommit) {
const repo = await reader(`/repos/${source.fullName}`, options.signal) as { id: number; default_branch: string };
if (String(repo.id) !== source.repositoryId) throw conflict('The repository at this URL has changed identity. Add it as a new source.');
const ref = source.trackingRef === 'HEAD' ? repo.default_branch : source.trackingRef;
const head = await reader(`/repos/${source.fullName}/commits/${encodeURIComponent(ref)}`, options.signal) as { sha: string; commit?: { tree?: { sha?: string } } };
const rawTreeSha = head.commit?.tree?.sha;
const treeSha = rawTreeSha && /^[a-f0-9]{40}$/i.test(rawTreeSha) ? rawTreeSha.toLowerCase() : null;
if (head.sha === source.lastScanCommit || (treeSha && treeSha === source.lastScanTreeSha)) {
await db.transaction(async tx => {
const [locked] = await tx.select().from(sources).where(scope(companyId, id)).for('update');
if (!locked || locked.leaseToken !== token || locked.revision !== source.revision) throw conflict('Source refresh changed. Try again.');
await context.audit(tx, id, 'company.skill_source_checked', { commit: head.sha, unchangedTree: head.sha !== source.lastScanCommit });
await tx.update(sources).set({ leaseToken: null, leaseExpiresAt: null, lastError: null, lastSuccessAt: new Date(), failureCount: 0, lastScanCommit: head.sha, lastScanTreeSha: treeSha ?? source.lastScanTreeSha,
connectionId: reader.connectionId ?? source.connectionId, refreshGrantId: reader.grantId ?? source.refreshGrantId,
nextSyncAt: new Date(Date.now() + 5 * 60_000) }).where(scope(companyId, id));
});
context.publishCommittedActivity?.();
return { source: await detail(companyId, id), imported: [], updated: [], unchanged: (await detail(companyId, id)).entries.filter(entry => entry.selection === 'selected').length, warnings: [] };
}
}
const packageMode = selection?.packageMode ?? (source.accessScope === 'projects' ? 'repository' : source.packageMode);
if (options.automatic) await reserveAutomaticSkillWork(db, source, token, options.now ?? new Date());
const scan = staged ?? await scanGitHubSkills({ repositoryUrl: source.repositoryUrl, trackingRef: source.trackingRef, packageMode }, reader, { signal: options.signal });
if (options.automatic) await settleAutomaticSkillWork(db, companyId, id, token,
packageMode === 'repository' && scan.packageMode === 'skills' ? {} : scan);
// A project source may already be in package-only mode after initial fallback.
if (source.accessScope === 'projects' && scan.packageMode === 'skills') rejectIncompleteSkillPackages(scan);
if (source.repositoryId && source.repositoryId !== scan.repositoryId) throw conflict('The repository at this URL has changed identity. Add it as a new source.');
const current = await detail(companyId, id);
const omittedInstalledFiles = current.entries.some(entry => entry.skillId && current.repositoryPackage?.files.some(file => {
const root = path.posix.dirname(entry.path);
return root !== '.' && !file.path.startsWith(`${root}/`);
}));
if (source.packageMode === 'repository' && scan.packageMode === 'skills'
&& selection?.packageMode !== 'skills' && current.entries.some(entry => entry.skillId)
&& (omittedInstalledFiles || scan.skills.some(skill => skill.inspection?.references.length))) {
throw unprocessable('The repository exceeds the supported package size. Installed repository skills and their dependencies were kept. Reduce the repository package size before refreshing.');
}
const selectedPaths = selection?.selectedPaths ?? current.entries.filter(entry => entry.selection === 'selected').map(entry => entry.path);
if (!selection && source.importNewSkills) {
for (const candidate of scan.skills) {
if (current.entries.some(entry => entry.path === candidate.path) || source.excludedFolders.some(folder => !folder || candidate.path.startsWith(`${folder}/`))) continue;
selectedPaths.push(candidate.path);
}
}
await authorizeScan(source, scan, selectedPaths, context);
return await db.transaction(async tx => {
const published = await db.transaction(async tx => {
const [locked] = await tx.select().from(sources).where(scope(companyId, id)).for('update');
if (!locked || locked.leaseToken !== token || locked.revision !== source.revision || locked.leaseExpiresAt! < new Date()) throw conflict('Source refresh expired or changed. Try again.');
const result = await publish(tx, locked, scan, selectedPaths, selection?.excludedFolders ?? source.excludedFolders, context, Boolean(selection));
if (selection || scan.connectionId !== undefined) await tx.update(sources).set({ ...(selection ? { enabled: true } : {}), connectionId: scan.connectionId ?? connectionId }).where(scope(companyId, id));
if (selection || scan.connectionId !== undefined) await tx.update(sources).set({ ...(selection ? { enabled: true } : {}), connectionId: scan.connectionId ?? connectionId,
...(source.refreshGrantId === null && reader.grantId ? { refreshGrantId: reader.grantId } : {}),
...(selection?.connectionId !== undefined || (!source.refreshUserId && !source.refreshLocalTrusted && context.setup) ? { refreshUserId: context.setup?.userId, refreshAgentId: context.setup?.agentId ?? null, refreshLocalTrusted: context.setup?.localTrusted ?? false, refreshGrantId: reader.grantId ?? null } : {}) }).where(scope(companyId, id));
return { ...result, source: await detail(companyId, id, tx) };
});
context.publishCommittedActivity?.();
return published;
} catch (error) {
// Provider errors have already been sanitized. Never persist arbitrary exception text.
const message = error instanceof Error && 'status' in error ? error.message : 'Refresh failed. The installed skills were kept. Try again.';
await db.update(sources).set({ leaseToken: null, leaseExpiresAt: null, lastError: message }).where(and(scope(companyId, id), eq(sources.leaseToken, token)));
await db.update(sources).set({ leaseToken: null, leaseExpiresAt: null, lastError: options.signal?.aborted ? source.lastError : message, failureCount: options.signal?.aborted ? source.failureCount : source.failureCount + 1, nextSyncAt: options.signal?.aborted ? new Date() : skillSourceRetryAt(error, source.failureCount) }).where(and(scope(companyId, id), eq(sources.leaseToken, token)));
throw error;
}
}
async function settings(companyId: string, id: string, input: SkillSourceSettingsRequest, context: SkillSourceContext) {
const source = await row(companyId, id);
await context.authorize('skills.edit', { sourceType: 'git', sourceLocator: source.repositoryUrl });
// Validate a replacement connection before binding unattended work to it.
let reader: GitHubRead | undefined;
if (input.connectionId !== undefined) {
reader = context.read(input.connectionId);
await reader(`/repos/${source.fullName}`);
}
return db.transaction(async tx => {
const [updated] = await tx.update(sources).set({
...(input.automaticUpdates !== undefined ? { automaticUpdates: input.automaticUpdates } : {}),
...(input.importNewSkills !== undefined ? { importNewSkills: input.importNewSkills } : {}),
...(input.connectionId !== undefined ? { connectionId: reader?.connectionId ?? input.connectionId,
refreshUserId: context.setup?.userId, refreshAgentId: context.setup?.agentId ?? null,
refreshLocalTrusted: context.setup?.localTrusted ?? false, refreshGrantId: reader?.grantId ?? null, lastError: null } : {}),
revision: sql`${sources.revision} + 1`, nextSyncAt: new Date(),
}).where(and(scope(companyId, id), eq(sources.revision, input.revision), or(isNull(sources.leaseToken), lt(sources.leaseExpiresAt, new Date())))).returning();
if (!updated) throw conflict('This source is refreshing or changed. Reload before saving.');
await context.audit(tx, id, 'company.skill_source_settings_updated', { automaticUpdates: updated.automaticUpdates, importNewSkills: updated.importNewSkills, connectionChanged: input.connectionId !== undefined });
return detail(companyId, id, tx);
});
}
async function disconnect(companyId: string, id: string, context: SkillSourceContext, input: SkillSourceDisconnectRequest): Promise<SkillSourceDisconnectResult> {
const source = await detail(companyId, id);
if (source.revision !== input.revision) throw conflict('This source changed. Reload before disconnecting.');
@@ -236,7 +342,18 @@ export function skillSourceService(db: Db) {
if (!locked || locked.revision !== input.revision) throw conflict('This source changed. Reload before disconnecting.');
}
async function remove(tx: Tx, copiedCount: number, deletedCount: number) {
await tx.delete(sources).where(scope(companyId, id));
// Stop both pending attachments and an in-flight refresh before removing tracking.
await tx.update(projectSkillSources).set({ active: false, leaseToken: null, leaseExpiresAt: null })
.where(and(eq(projectSkillSources.companyId, companyId), eq(projectSkillSources.sourceId, id)));
if (source.accessScope === 'projects') {
// Copies and forks keep this source identity as their dynamic access boundary.
await tx.delete(entries).where(and(eq(entries.companyId, companyId), eq(entries.sourceId, id)));
await tx.update(sources).set({ removedAt: new Date(), enabled: false, automaticUpdates: false,
revision: sql`${sources.revision} + 1`, leaseToken: null, leaseExpiresAt: null })
.where(scope(companyId, id));
} else {
await tx.delete(sources).where(scope(companyId, id));
}
await context.audit(tx, id, 'company.skill_source_disconnected', { skills: input.skills, skillIds, copiedCount, deletedCount });
}
if (input.skills === 'delete') {
@@ -305,5 +422,5 @@ export function skillSourceService(db: Db) {
const imported = await Promise.all(result.source.entries.filter(entry => selectedPaths.includes(entry.path) && entry.skillId && !entry.error).map(entry => skills.getById(companyId, entry.skillId!)));
return { imported: imported.filter((skill): skill is CompanySkill => Boolean(skill)), warnings: result.warnings };
}
return { list, detail, sourceForSkill, discover, preview, create, refresh, disconnect, importFromUrl };
return { list, detail, sourceForSkill, discover, preview, create, refresh, settings, disconnect, importFromUrl };
}
tests/e2e/signoff-policy.spec.ts
+7 -3
View File
@@ -275,6 +275,7 @@ async function waitForFixtureRun(
agentId: string,
issueId: string,
runId: string,
allowCancelledLock = false,
) {
// Negative authorization cases deliberately invoke a non-participant. Leave
// their API rejection intact instead of waiting for a run they cannot own.
@@ -286,7 +287,8 @@ async function waitForFixtureRun(
const run = await fetchVerifiedRun(board, runId, agentId, issueId);
expect(run, "Fixture run must belong to the acting agent and issue").not.toBeNull();
status = run!.status;
if (status === "succeeded") return;
if (status === "succeeded") return true;
if (status === "cancelled" && allowCancelledLock) return false;
expect(["queued", "running"], `Fixture run ${runId} ended as ${status}`).toContain(status);
await new Promise((resolve) => setTimeout(resolve, WAKEUP_POLL_INTERVAL_MS));
} while (Date.now() < deadline);
@@ -311,7 +313,9 @@ async function retryAgentPatchWithCurrentLockOnConflict(
const lockedRunId = issueRunLock.checkoutRunId ?? issueRunLock.executionRunId ?? fallbackRunId;
// A new lock can belong to a newly promoted fixture run, not the completed
// run we first invoked. Let that exact verified run settle before PATCH.
await waitForFixtureRun(board, agent.agentId, issueId, lockedRunId);
// Recovery can cancel the selected lock while this wait is in flight.
// Read the current lock again; never PATCH under that cancelled run.
if (!(await waitForFixtureRun(board, agent.agentId, issueId, lockedRunId, true))) continue;
res = await agent.request.patch(`${BASE_URL}/api/issues/${issueId}`, {
headers: { "X-Paperclip-Run-Id": lockedRunId },
data: patchData,
@@ -368,7 +372,7 @@ async function agentPatch(
// run-lock race — leave it for the caller to assert on.
if (issueRunLock.assigneeAgentId !== agent.agentId) break;
const retryRunId = issueRunLock.checkoutRunId ?? issueRunLock.executionRunId ?? runId;
await waitForFixtureRun(board, agent.agentId, issueId, retryRunId);
if (!(await waitForFixtureRun(board, agent.agentId, issueId, retryRunId, true))) continue;
res = await patchWith(retryRunId);
}
return res;
tests/storybook-visual/playwright.config.ts
+2 -2
View File
@@ -47,9 +47,9 @@ export default defineConfig({
// JS-driven tickers/timers key off prefers-reduced-motion for
// deterministic captures (CSS animations are already disabled).
reducedMotion: "reduce",
baseURL: "http://localhost:6106",
baseURL: process.env.STORYBOOK_TEST_BASE_URL ?? "http://localhost:6106",
},
webServer: {
webServer: process.env.STORYBOOK_TEST_BASE_URL ? undefined : {
command: "node ../../scripts/serve-storybook-static.mjs --port 6106",
url: "http://localhost:6106/index.json",
reuseExistingServer: true,
tests/storybook-visual/project-repository-skills.spec.ts
+202
View File
@@ -0,0 +1,202 @@
import { expect, test } from '@playwright/test';
const story = (name: string) => `/iframe.html?id=skills-project-repository-skills-02-journeys--${name}&viewMode=story&globals=theme:dark`;
test('creating a project imports repository skills without another setup flow', async ({ page }) => {
await page.goto(story('create-project'));
await page.getByRole('button', { name: 'Add Project', exact: true }).first().click();
const dialog = page.getByRole('dialog', { name: 'Create project', exact: true });
await dialog.getByRole('textbox', { name: 'Project name', exact: true }).fill('Repository project');
await dialog.getByRole('button', { name: 'Add GitHub repo', exact: true }).click();
await page.getByRole('option', { name: 'acme/team-skills My GitHub account · Acme engineering', exact: true }).click();
await dialog.getByRole('button', { name: 'Create project', exact: true }).click();
await expect(dialog).toHaveCount(0);
await page.getByRole('link', { name: 'Skills', exact: true }).click();
await page.getByLabel('Skills', { exact: true }).getByRole('link', { name: 'Sources', exact: true }).click();
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
await expect(source.getByText('Code review', { exact: true })).toBeVisible();
await expect(page.getByRole('button', { name: 'Import skills', exact: true })).toHaveCount(0);
});
test('attaching another repository imports skills and keeps the original source', async ({ page }) => {
await page.goto(story('attach-repository'));
await page.getByRole('button', { name: 'Add another repo', exact: true }).click();
await page.getByRole('option', { name: 'maya/research-skills My GitHub account', exact: true }).click();
await page.getByRole('button', { name: 'Save changes', exact: true }).click();
await expect(page.getByText('Changes saved', { exact: true })).toBeVisible();
await page.getByRole('link', { name: 'Skills', exact: true }).click();
await page.getByLabel('Skills', { exact: true }).getByRole('link', { name: 'Sources', exact: true }).click();
const original = page.getByRole('region', { name: 'acme/team-skills', exact: true });
const added = page.getByRole('region', { name: 'maya/research-skills', exact: true });
await expect(original.getByText('Code review', { exact: true })).toBeVisible();
await expect(added.getByText('Code review', { exact: true })).toBeVisible();
expect(await original.getByRole('link', { name: /^Code review / }).getAttribute('href'))
.not.toBe(await added.getByRole('link', { name: /^Code review / }).getAttribute('href'));
});
test('empty repositories explain automatic discovery of future skills', async ({ page }) => {
await page.goto(story('empty-repository'));
await expect(page.getByText('No skills found yet. New skills will be imported automatically.', { exact: true })).toBeVisible();
});
test('routine instructions stay stable while new executions follow updated skill content', async ({ page }) => {
await page.goto(story('routine-reference'));
const description = page.getByRole('region', { name: 'Description', exact: true });
await expect(description.getByRole('link', { name: '/code-review', exact: true })).toBeVisible();
const instructions = await description.innerText();
await page.getByRole('button', { name: 'Run now', exact: true }).click();
await page.getByRole('button', { name: 'Run routine', exact: true }).click();
await page.getByRole('link', { name: 'Open PAP-124: Review project changes', exact: true }).click();
const outcome = page.getByText(/^Completed using code review version /);
await expect(outcome).toContainText('version-entry-1 at commit a17d36');
await expect(outcome).toBeVisible();
const first = await outcome.textContent() ?? '';
expect(first).toContain('version-entry-1 at commit a17d36');
await expect(page.getByRole('alert')).toHaveCount(0);
await page.getByRole('link', { name: 'Skills', exact: true }).click();
await page.getByLabel('Skills', { exact: true }).getByRole('link', { name: 'Sources', exact: true }).click();
await page.getByRole('region', { name: 'acme/team-skills', exact: true }).getByRole('button', { name: 'Refresh', exact: true }).click();
await expect(page.getByText('Security review', { exact: true })).toBeVisible();
await page.getByRole('link', { name: 'Routines', exact: true }).click();
await page.getByRole('main').getByText('Review project changes', { exact: true }).click();
await page.getByLabel('Routine navigation', { exact: true }).getByRole('link', { name: 'Overview', exact: true }).click();
await expect(description).toHaveText(instructions, { useInnerText: true });
await page.getByRole('button', { name: 'Run now', exact: true }).click();
await page.getByRole('button', { name: 'Run routine', exact: true }).click();
await page.getByRole('link', { name: 'Open PAP-125: Review project changes', exact: true }).click();
await expect(outcome).toContainText('version-entry-1-updated at commit bbbbbbbb');
await page.getByRole('link', { name: 'Routine', exact: true }).click();
await page.getByLabel('Routine navigation', { exact: true }).getByRole('link', { name: 'Runs', exact: true }).click();
await page.getByRole('link', { name: 'Open PAP-124: Review project changes', exact: true }).click();
await expect(outcome).toHaveText(first);
});
test('legacy adoption imports directly and survives navigation within the journey', async ({ page }, info) => {
await page.goto(story('legacy-adoption'));
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
const action = page.getByRole('button', { name: 'Import skills', exact: true });
await action.click();
await expect(source).toBeVisible();
await expect(source.getByText('Code review', { exact: true })).toBeVisible();
await expect(action).toHaveCount(0);
await page.getByRole('link', { name: 'Installed skills', exact: true }).click();
await expect(source).toHaveCount(0);
await expect(page.getByText('Code review', { exact: true }).first()).toBeVisible();
await page.getByLabel('Skills', { exact: true }).getByRole('link', { name: 'Sources', exact: true }).click();
await expect(source).toBeVisible();
await page.screenshot({ path: info.outputPath('legacy-adopted.png') });
});
test('background refresh imports an addition without clicking refresh', async ({ page }, info) => {
await page.goto(story('automatic-update'));
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
await expect(source.getByText('Security review', { exact: true })).toBeVisible({ timeout: 15000 });
await expect(source.getByRole('button', { name: 'Refresh', exact: true })).toBeEnabled();
await page.screenshot({ path: info.outputPath('automatic-update.png') });
});
test('refresh failure retains content and retry recovers', async ({ page }, info) => {
await page.goto(story('failed-refresh'));
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
await expect(source.getByText('Code review', { exact: true })).toBeVisible();
await expect(source.getByRole('alert')).toContainText('installed skills were kept');
await source.getByRole('button', { name: 'Refresh', exact: true }).click();
await expect(source.getByRole('button', { name: 'Refresh', exact: true })).toBeEnabled();
await source.getByRole('button', { name: 'Refresh', exact: true }).click();
await expect(source.getByText('Security review', { exact: true })).toBeVisible();
await expect(source.getByRole('alert')).toHaveCount(0);
await page.screenshot({ path: info.outputPath('refresh-recovered.png') });
});
test('automatic update preference is accessible and changes in place', async ({ page }) => {
await page.goto(story('synced'));
await page.getByRole('button', { name: 'More actions for acme/team-skills', exact: true }).click();
const toggle = page.getByRole('menuitemcheckbox', { name: 'Automatic updates', exact: true });
await expect(toggle).toBeChecked();
await toggle.click();
await expect(page.getByText('Automatic updates paused', { exact: true })).toBeVisible();
await page.getByRole('button', { name: 'More actions for acme/team-skills', exact: true }).click();
await expect(toggle).not.toBeChecked();
await page.keyboard.press('Escape');
await expect(page.getByRole('button', { name: 'More actions for acme/team-skills', exact: true })).toBeFocused();
});
test('mobile touch actions stay usable without horizontal overflow', async ({ browser }, info) => {
const context = await browser.newContext({ baseURL: info.project.use.baseURL, viewport: { width: 390, height: 844 }, hasTouch: true, reducedMotion: 'reduce' });
const page = await context.newPage();
try {
await page.goto(story('mobile'));
await expect(page.getByRole('heading', { name: 'Skill sources', exact: true })).toBeVisible();
await expect(page.getByRole('button', { name: 'Refresh', exact: true })).toBeVisible();
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
await page.getByRole('button', { name: 'More actions for acme/team-skills', exact: true }).tap();
await page.getByRole('menuitemcheckbox', { name: 'Automatic updates', exact: true }).tap();
await expect(page.getByText('Automatic updates paused', { exact: true })).toBeVisible();
await page.screenshot({ path: info.outputPath('sources-mobile.png') });
} finally { await context.close(); }
});
test('connection repair keeps the installed skills and selections', async ({ page }) => {
await page.goto(story('connection-repair'));
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
await source.getByRole('button', { name: 'Choose connection', exact: true }).click();
const dialog = page.getByRole('dialog');
await expect(dialog.getByRole('button', { name: 'Save connection', exact: true })).toBeDisabled();
await dialog.getByRole('radio', { name: 'My GitHub account', exact: true }).click();
await dialog.getByRole('button', { name: 'Save connection', exact: true }).click();
await expect(dialog).toHaveCount(0);
await expect(source.getByRole('button', { name: 'More actions for acme/team-skills', exact: true })).toBeFocused();
await expect(source.getByRole('alert')).toHaveCount(0);
await expect(source.getByText('Code review', { exact: true })).toBeVisible();
});
test('connection repair recovers from a concurrent update without losing the selected account', async ({ page }) => {
await page.goto(story('repair-conflict'));
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
await source.getByRole('button', { name: 'Choose connection', exact: true }).click();
const dialog = page.getByRole('dialog');
const account = dialog.getByRole('radio', { name: 'My GitHub account', exact: true });
await account.click();
await dialog.getByRole('button', { name: 'Save connection', exact: true }).click();
await expect(dialog.getByRole('alert')).toContainText('This source was updated');
await expect(account).toBeChecked();
await dialog.getByRole('button', { name: 'Save connection', exact: true }).click();
await expect(dialog).toHaveCount(0);
await expect(source.getByRole('alert')).toHaveCount(0);
await expect(source.getByText('Code review', { exact: true })).toBeVisible();
});
test('keeping project skill copies explains retained access and removes tracking', async ({ page }) => {
await page.goto(story('private-project'));
await page.getByRole('button', { name: 'More actions for acme/team-skills', exact: true }).click();
await page.getByRole('menuitem', { name: 'Disconnect source', exact: true }).click();
const dialog = page.getByRole('dialog', { name: 'Disconnect acme/team-skills?', exact: true });
await expect(dialog.getByRole('button', { name: 'Disconnect source', exact: true })).toBeDisabled();
await expect(dialog.getByText('Move them to My Skills, keeping project access restrictions. Assigned agents keep using them.', { exact: true })).toBeVisible();
await dialog.getByRole('radio', { name: /Keep editable copies/ }).click();
await dialog.getByRole('button', { name: 'Disconnect source', exact: true }).click();
await expect(dialog).toHaveCount(0);
await expect(page.getByRole('region', { name: 'acme/team-skills', exact: true })).toHaveCount(0);
});
test('cancelling source removal keeps tracking and allows refresh', async ({ page }) => {
await page.goto(story('synced'));
await page.getByRole('button', { name: 'More actions for acme/team-skills', exact: true }).click();
await page.getByRole('menuitem', { name: 'Disconnect source', exact: true }).click();
const dialog = page.getByRole('dialog', { name: 'Disconnect acme/team-skills?', exact: true });
await dialog.getByRole('radio', { name: /Delete skills/ }).click();
await dialog.getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(dialog).toHaveCount(0);
const source = page.getByRole('region', { name: 'acme/team-skills', exact: true });
await source.getByRole('button', { name: 'Refresh', exact: true }).click();
await expect(source.getByText('Security review', { exact: true })).toBeVisible();
});
test('withdrawn setup requests remain terminal on a narrow light layout', async ({ page }) => {
await page.setViewportSize({ width: 390, height: 844 });
await page.goto('/iframe.html?id=connections-in-task-connections--ai-withdrawn&viewMode=story&globals=theme:light');
await expect(page.getByTestId('connection-intent-terminal')).toBeVisible();
await expect(page.getByText('Connection request withdrawn', { exact: true })).toBeVisible();
await expect(page.getByRole('button', { name: 'Fix connection', exact: true })).toHaveCount(0);
});
ui/src/api/skillSources.ts
+4 -1
View File
@@ -1,8 +1,11 @@
import type { ProjectRepositoryOptions, SkillSourceDisconnectRequest, SkillSourceDisconnectResult, SkillSourceFilePreview, SkillSourcePreviewRequest, SkillSource, SkillSourceCreateRequest, SkillSourceDiscovery, SkillSourceDiscoveryRequest, SkillSourceRefreshResult, SkillSourceSelectionRequest, SkillSourceScanUpdate, SkillSourceDiscoveryEvent } from '@paperclipai/shared';
import type { ProjectRepositoryOptions, ProjectSkillRepository, SkillSourceSettingsRequest, SkillSourceFilePreview, SkillSourcePreviewRequest, SkillSource, SkillSourceCreateRequest, SkillSourceDiscovery, SkillSourceDiscoveryRequest, SkillSourceRefreshResult, SkillSourceSelectionRequest, SkillSourceScanUpdate, SkillSourceDiscoveryEvent, SkillSourceDisconnectRequest, SkillSourceDisconnectResult } from '@paperclipai/shared';
import { api, ApiError, requestResponse } from './client';
const base = (companyId: string) => `/companies/${companyId}/skill-sources`;
export const skillSourcesApi = {
list: (companyId: string) => api.get<SkillSource[]>(base(companyId)),
projectRepositories: (companyId: string) => api.get<ProjectSkillRepository[]>(`${base(companyId)}/project-repositories`),
adoptProject: (companyId: string, workspaceId: string) => api.post<{ bindingId: string }>(`${base(companyId)}/adopt-project`, { workspaceId }),
settings: (companyId: string, id: string, input: SkillSourceSettingsRequest) => api.patch<SkillSource>(`${base(companyId)}/${id}/settings`, input),
repositories: (companyId: string) => api.get<ProjectRepositoryOptions>(`${base(companyId)}/repositories`),
discover: (companyId: string, input: SkillSourceDiscoveryRequest) => api.post<SkillSourceDiscovery>(`${base(companyId)}/discover`, input),
discoverStream: async (companyId: string, input: SkillSourceDiscoveryRequest, onProgress: (event: SkillSourceScanUpdate) => void, signal?: AbortSignal): Promise<SkillSourceDiscovery> => {
ui/src/components/skill-studio/ProjectSkillRepositoryRow.tsx
+27
View File
@@ -0,0 +1,27 @@
import type { ProjectSkillRepository } from '@paperclipai/shared';
import { GithubIcon } from '@/components/icons/github-icon';
import { Button } from '@/components/ui/button';
import { Link } from '@/lib/router';
export function ProjectSkillRepositoryRow({ repository, pending, onImport }: {
repository: ProjectSkillRepository;
pending?: boolean;
onImport: () => void;
}) {
const importing = pending || repository.status === 'pending';
return <section className="flex flex-col gap-2 py-3" aria-label={`Skills from ${repository.projectName}`}>
<div className="flex items-center gap-3">
<GithubIcon className="size-5 shrink-0 text-muted-foreground" />
<div className="min-w-0 flex-1">
<a href={repository.repositoryUrl} target="_blank" rel="noreferrer" className="block truncate text-sm font-medium hover:underline">
{repository.repositoryUrl.replace('https://github.com/', '')}
</a>
<p className="truncate text-xs text-muted-foreground">{repository.projectName} · {repository.trackingRef === 'HEAD' ? 'Default branch' : repository.trackingRef}{repository.projectVisibility === 'private' ? ' · Project access' : ''}</p>
</div>
<Button size="sm" variant="outline" disabled={importing} onClick={onImport}>
{importing ? 'Importing…' : repository.status === 'error' ? 'Retry import' : 'Import skills'}
</Button>
</div>
{repository.lastError && <p role="alert" className="text-sm text-destructive">{repository.lastError} <Link to="/apps" className="underline">Manage GitHub connection</Link></p>}
</section>;
}
ui/src/components/skill-studio/SkillSourceConnectionRepair.tsx
+50
View File
@@ -0,0 +1,50 @@
import { useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import type { SkillSource } from '@paperclipai/shared';
import { skillSourcesApi } from '@/api/skillSources';
import { ApiError } from '@/api/client';
import { queryKeys } from '@/lib/queryKeys';
import { Link } from '@/lib/router';
import { Button } from '@/components/ui/button';
import { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle } from '@/components/ui/dialog';
import { RadioCardGroup } from '@/components/ui/radio-card';
export function SkillSourceConnectionRepair({ companyId, source, onClose, onSaved, onReturnFocus }: {
companyId: string; source: SkillSource; onClose: () => void; onSaved: () => Promise<void>;
onReturnFocus?: () => void;
}) {
const [connectionId, setConnectionId] = useState('');
const client = useQueryClient();
const query = useQuery({ queryKey: queryKeys.skillSources.repositories(companyId), queryFn: () => skillSourcesApi.repositories(companyId), refetchOnMount: 'always', retry: false });
const save = useMutation({ mutationFn: () => skillSourcesApi.settings(companyId, source.id, { revision: source.revision, connectionId }),
onError: async error => {
if (error instanceof ApiError && error.status === 409) {
await client.invalidateQueries({ queryKey: queryKeys.skillSources.all(companyId) });
}
},
onSuccess: async () => { await onSaved(); onClose(); } });
const connections = query.data?.connections ?? [];
return <Dialog open onOpenChange={open => { if (!open && !save.isPending) onClose(); }}>
<DialogContent className="flex max-h-(--sz-calc-18) flex-col overflow-hidden sm:max-w-md"
onCloseAutoFocus={event => { if (onReturnFocus) { event.preventDefault(); onReturnFocus(); } }}>
<DialogHeader className="shrink-0">
<DialogTitle>Choose GitHub connection</DialogTitle>
<DialogDescription>Automatic updates will use your access to this connection. Installed skills and selections stay in place.</DialogDescription>
</DialogHeader>
<div className="min-h-0 overflow-y-auto space-y-4">
{query.isPending && <p role="status" className="text-sm text-muted-foreground">Loading connections…</p>}
{query.error && <p role="alert" className="text-sm text-destructive">{query.error.message} <Button variant="ghost" size="sm" onClick={() => void query.refetch()}>Try again</Button></p>}
<RadioCardGroup ariaLabel="GitHub connection" value={connectionId} onValueChange={setConnectionId} disabled={save.isPending}
options={connections.map(connection => ({ value: connection.id, title: connection.name }))} />
{query.isSuccess && connections.length === 0 && <p className="text-sm text-muted-foreground">Connect GitHub to continue.</p>}
<Button asChild variant="link" className="justify-start p-0"><Link to="/apps">Manage GitHub connections</Link></Button>
{save.error && <p role="alert" className="text-sm text-destructive">{save.error instanceof ApiError && save.error.status === 409
? 'This source was updated. Your connection is still selected. Save again to apply it.' : save.error.message}</p>}
</div>
<footer className="flex shrink-0 items-center justify-between gap-3 border-t border-border pt-4">
<Button variant="ghost" disabled={save.isPending} onClick={onClose}>Cancel</Button>
<Button disabled={!connectionId || save.isPending} onClick={() => save.mutate()}>{save.isPending ? 'Saving…' : 'Save connection'}</Button>
</footer>
</DialogContent>
</Dialog>;
}
ui/src/components/skill-studio/SkillSourceSyncControls.tsx
+23
View File
@@ -0,0 +1,23 @@
import { RefreshCw } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { ToggleSwitch } from '@/components/ui/toggle-switch';
export function SkillSourceSyncControls({ automaticUpdates, refreshing, disabled, onRefresh, onAutomaticUpdates, showPreference = true }: {
automaticUpdates: boolean;
showPreference?: boolean;
refreshing?: boolean;
disabled?: boolean;
onRefresh: () => void;
onAutomaticUpdates: (enabled: boolean) => void;
}) {
return <div className="flex flex-wrap items-center gap-3">
{showPreference && <label className="flex items-center gap-2 text-xs text-muted-foreground">
<ToggleSwitch checked={automaticUpdates} disabled={disabled} onCheckedChange={onAutomaticUpdates} aria-label="Automatic updates" />
Automatic updates
</label>}
<Button size="sm" variant="outline" disabled={disabled || refreshing} onClick={onRefresh}>
<RefreshCw className={refreshing ? 'size-3.5 animate-spin motion-reduce:animate-none' : 'size-3.5'} />
{refreshing ? 'Refreshing…' : 'Refresh'}
</Button>
</div>;
}
ui/src/context/LiveUpdatesProvider.test.ts
+11
View File
@@ -16,6 +16,17 @@ import { __liveUpdatesTestUtils } from "./LiveUpdatesProvider";
import { queryKeys } from "../lib/queryKeys";
describe("LiveUpdatesProvider issue invalidation", () => {
it('refreshes sources, legacy repositories and installed skills after a background import', () => {
const client = new QueryClient();
const invalidate = vi.spyOn(client, 'invalidateQueries');
__liveUpdatesTestUtils.invalidateActivityQueries(client, 'company-1', {
entityType: 'company_skill_source', entityId: 'source-1', action: 'company.skill_source_refreshed',
}, { userId: 'user-1', agentId: null });
for (const queryKey of [queryKeys.skillSources.all('company-1'), queryKeys.skillSources.projects('company-1'), queryKeys.companySkills.list('company-1')]) {
expect(invalidate).toHaveBeenCalledWith({ queryKey });
}
client.clear();
});
it("refreshes personal membership state when a primary changes on another device", () => {
const client = new QueryClient();
const invalidate = vi.spyOn(client, "invalidateQueries");
ui/src/context/LiveUpdatesProvider.tsx
+7
View File
@@ -1498,6 +1498,13 @@ function invalidateActivityQueries(
return;
}
if (entityType === "company_skill_source") {
queryClient.invalidateQueries({ queryKey: queryKeys.skillSources.all(companyId) });
queryClient.invalidateQueries({ queryKey: queryKeys.companySkills.list(companyId) });
queryClient.invalidateQueries({ queryKey: queryKeys.skillSources.projects(companyId) });
return;
}
if (entityType === "company_skill") {
const sourceIssueId = readString((payload.details as Record<string, unknown> | undefined)?.sourceIssueId);
if (sourceIssueId) queryClient.invalidateQueries({ queryKey: queryKeys.issues.activity(sourceIssueId) });
ui/src/features/connections/ConnectionIntentInteractionBody.test.tsx
+9
View File
@@ -628,6 +628,15 @@ describe("AI repair inside the card", () => {
expect(document.body.textContent).toContain("Dotta must reconnect My Codex account");
expect(document.querySelector('[data-testid="shared-ai-credentials"]')).toBeNull();
});
it("shows a withdrawn AI request as terminal without another setup action", () => {
renderBody({ ...interaction, status: "cancelled", resolvedAt: new Date() });
expect(document.body.textContent).toContain("Connection request withdrawn");
expect(document.body.textContent).not.toContain("This task can’t run");
expect(button("Fix connection")).toBeUndefined();
expect(document.querySelector('[data-testid="connection-intent-terminal"]')).not.toBeNull();
expect(setupOptionsMock).not.toHaveBeenCalled();
expect(credentialRender).not.toHaveBeenCalled();
});
it("does not promise to run without credentials when declined", () => {
renderBody({ ...interaction, status: "rejected" });
expect(document.body.textContent).toContain("The task still needs a working AI connection");
ui/src/features/connections/ConnectionIntentInteractionBody.tsx
+6
View File
@@ -246,6 +246,12 @@ export function ConnectionIntentInteractionBody({
title: accessRequest ? `${serviceName} access granted` : interaction.payload.upstreamService ? "External provider connected" : `${serviceName} connected`,
body: accessRequest ? null : interaction.payload.upstreamService ? `${interaction.payload.requestingAgentName} can now verify and authorize ${interaction.payload.upstreamService.name} through this provider. The app is not yet verified.` : isAi ? "This agent can now use the connection." : `${interaction.payload.requestingAgentName} can use this connection on the continuation run.`,
}
: interaction.status === "cancelled"
? {
icon: XCircle,
title: "Connection request withdrawn",
body: "This request is no longer active.",
}
: interaction.status === "rejected"
? {
icon: XCircle,
ui/src/lib/queryKeys.ts
+1
View File
@@ -1,5 +1,6 @@
export const queryKeys = {
skillSources: {
projects: (companyId: string) => ['project-skill-repositories', companyId] as const,
preview: (companyId: string, repositoryUrl: string, connectionId: string | null, commitSha: string | null, skillPath: string, filePath: string) => ['skill-sources', companyId, 'preview', repositoryUrl, connectionId, commitSha, skillPath, filePath] as const,
all: (companyId: string) => ["skill-sources", companyId] as const,
repositories: (companyId: string) => ["skill-source-repositories", companyId] as const,
ui/src/pages/SkillSources.test.tsx
+3 -2
View File
@@ -16,7 +16,7 @@ vi.mock('@/lib/router', () => ({
useNavigate: () => context.navigate,
useParams: () => ({ sourceId: context.sourceId }),
}));
vi.mock('@/api/skillSources', () => ({ skillSourcesApi: { list: vi.fn(), repositories: vi.fn(), discoverStream: vi.fn(), create: vi.fn(), select: vi.fn(), refresh: vi.fn(), disconnect: vi.fn() } }));
vi.mock('@/api/skillSources', () => ({ skillSourcesApi: { list: vi.fn(), projectRepositories: vi.fn(), adoptProject: vi.fn(), settings: vi.fn(), repositories: vi.fn(), discoverStream: vi.fn(), create: vi.fn(), select: vi.fn(), refresh: vi.fn(), disconnect: vi.fn() } }));
const repos = {
connections: [{ id: 'personal', name: 'Personal' }, { id: 'shared', name: 'Engineering' }],
@@ -50,6 +50,7 @@ beforeEach(() => {
vi.stubGlobal('ResizeObserver', class { observe() {} unobserve() {} disconnect() {} });
vi.spyOn(HTMLElement.prototype, 'scrollIntoView').mockImplementation(() => {});
vi.mocked(skillSourcesApi.list).mockResolvedValue([]);
vi.mocked(skillSourcesApi.projectRepositories).mockResolvedValue([]);
vi.mocked(skillSourcesApi.repositories).mockResolvedValue(structuredClone(repos));
vi.mocked(skillSourcesApi.discoverStream).mockResolvedValue(discovery);
context.sourceId = 'new';
@@ -308,7 +309,7 @@ describe('GitHub skill source import', () => {
expect(button('Disconnect source').disabled).toBe(false);
expect(document.querySelector('section[aria-label="acme/team-skills"]')).not.toBeNull();
const other = [...document.querySelectorAll<HTMLButtonElement>('[role="radio"]')].find(el => el.textContent?.startsWith('Delete skills'))!;
await act(async () => other.click());
await act(async () => other.click()); await flush();
expect(document.querySelector('[role="dialog"] [role="alert"]')).toBeNull();
});
ui/src/pages/SkillSources.tsx
+55 -15
View File
@@ -1,3 +1,6 @@
import { ProjectSkillRepositoryRow } from "@/components/skill-studio/ProjectSkillRepositoryRow";
import { SkillSourceConnectionRepair } from "@/components/skill-studio/SkillSourceConnectionRepair";
import { SkillSourceSyncControls } from "@/components/skill-studio/SkillSourceSyncControls";
import { GithubIcon } from "@/components/icons/github-icon";
import { useEffect, useRef, useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
@@ -15,7 +18,7 @@ import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { RadioCardGroup } from '@/components/ui/radio-card';
import { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle } from '@/components/ui/dialog';
import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuTrigger } from '@/components/ui/dropdown-menu';
import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuCheckboxItem, DropdownMenuTrigger } from '@/components/ui/dropdown-menu';
import { SkillImportProgress, type FoundSkill } from './skills/SkillImportProgress';
import { SkillPackagePreview } from './skills/SkillPackagePreview';
import { SkillSourceTree, type SkillTreeCandidate } from './skills/SkillSourceTree';
@@ -26,22 +29,31 @@ import { consumeSkillSourceReturn, rememberSkillSourceReturn } from '@/lib/skill
const sourceKey = (companyId: string) => queryKeys.skillSources.all(companyId);
export function SkillSources() {
const sourceActionButtons = useRef<Record<string, HTMLButtonElement | null>>({});
const { selectedCompanyId, selectedCompany } = useCompany();
const { sourceId } = useParams<{ sourceId: string }>();
const navigate = useNavigate();
const client = useQueryClient();
const { setBreadcrumbs } = useBreadcrumbs();
const [results, setResults] = useState<Record<string, { message: string; failedAt: number; operation: 'refresh' | 'settings' } | null>>({});
const [repairSourceId, setRepairSourceId] = useState<string | null>(null);
const [disconnectSource, setDisconnectSource] = useState<SkillSource | null>(null);
const [results, setResults] = useState<Record<string, string>>({});
const companyId = selectedCompanyId ?? '';
useEffect(() => { setBreadcrumbs([{ label: 'Skills', href: '/skills' }, { label: 'Sources' }]); }, [setBreadcrumbs]);
const query = useQuery({ queryKey: sourceKey(companyId), queryFn: () => skillSourcesApi.list(companyId), enabled: Boolean(companyId) });
const query = useQuery({ queryKey: sourceKey(companyId), queryFn: () => skillSourcesApi.list(companyId), enabled: Boolean(companyId), refetchInterval: 5000 });
const repairSource = query.data?.find(source => source.id === repairSourceId);
const projectRepositories = useQuery({ queryKey: queryKeys.skillSources.projects(companyId), queryFn: () => skillSourcesApi.projectRepositories(companyId), enabled: Boolean(companyId), refetchInterval: 5000 });
async function invalidate() {
await Promise.all([client.invalidateQueries({ queryKey: sourceKey(companyId) }), client.invalidateQueries({ queryKey: queryKeys.companySkills.list(companyId) })]);
await Promise.all([client.invalidateQueries({ queryKey: queryKeys.skillSources.projects(companyId) }), client.invalidateQueries({ queryKey: sourceKey(companyId) }), client.invalidateQueries({ queryKey: queryKeys.companySkills.list(companyId) })]);
}
const refresh = useMutation({ mutationFn: (id: string) => skillSourcesApi.refresh(companyId, id), onSuccess: async result => {
setResults(prev => ({ ...prev, [result.source.id]: '' })); await invalidate();
}, onError: (error, id) => { setResults(prev => ({ ...prev, [id]: error.message })); void invalidate(); } });
setResults(prev => ({ ...prev, [result.source.id]: null })); await invalidate();
}, onError: (error, id) => { setResults(prev => ({ ...prev, [id]: { message: error.message, failedAt: Date.now(), operation: 'refresh' } })); void invalidate(); } });
const adopt = useMutation({ mutationFn: (workspaceId: string) => skillSourcesApi.adoptProject(companyId, workspaceId), onSuccess: invalidate });
const settings = useMutation({ mutationFn: ({ source, enabled }: { source: SkillSource; enabled: boolean }) => skillSourcesApi.settings(companyId, source.id, { revision: source.revision, automaticUpdates: enabled }),
onSuccess: async (_result, variables) => { setResults(prev => ({ ...prev, [variables.source.id]: null })); await invalidate(); },
onError: (error, variables) => setResults(prev => ({ ...prev, [variables.source.id]: { message: error.message, failedAt: Date.now(), operation: 'settings' } })) });
const availableProjects = Array.isArray(projectRepositories.data) ? projectRepositories.data.filter(repo => !repo.sourceId) : [];
const disconnect = useMutation({
mutationFn: ({ source, skills }: { source: SkillSource; skills: SkillSourceDisconnectRequest['skills'] }) =>
skillSourcesApi.disconnect(companyId, source.id, { revision: source.revision, skills }),
@@ -64,32 +76,52 @@ export function SkillSources() {
if (!companyId) return <p className="p-6 text-sm text-muted-foreground">Select a company to manage skill sources.</p>;
return <div className="mx-auto flex w-full max-w-5xl flex-col gap-6 p-4 md:p-6">
<header className="flex flex-wrap items-start justify-between gap-4">
<div><h1 className="text-xl font-semibold">Skill sources</h1><p className="mt-1 text-sm text-muted-foreground">Import skills from GitHub and refresh them when you’re ready.</p></div>
<div><h1 className="text-xl font-semibold">Skill sources</h1><p className="mt-1 text-sm text-muted-foreground">Skills from your repositories stay up to date automatically.</p></div>
<Button onClick={() => navigate('/skills/sources/new')}><Plus className="size-4" />Import from GitHub</Button>
</header>
<Link to="/skills" className="text-sm text-muted-foreground hover:text-foreground">Installed skills</Link>
{query.isPending && <p role="status" className="text-sm text-muted-foreground">Loading sources…</p>}
{query.error && <p role="alert" className="text-sm text-destructive">{query.error.message} <Button variant="ghost" size="sm" onClick={() => void query.refetch()}>Try again</Button></p>}
{query.data?.length === 0 && <div className="flex flex-col items-start gap-3 py-8"><p className="text-sm text-muted-foreground">No repositories added yet. Import your skills to make them available in {selectedCompany?.name ?? 'this company'}.</p><Button variant="outline" onClick={() => navigate('/skills/sources/new')}>Import from GitHub</Button></div>}
{query.data?.length === 0 && availableProjects.length === 0 && <div className="flex flex-col items-start gap-3 py-8"><p className="text-sm text-muted-foreground">No repositories added yet. Import your skills to make them available in {selectedCompany?.name ?? 'this company'}.</p><Button variant="outline" onClick={() => navigate('/skills/sources/new')}>Import from GitHub</Button></div>}
{projectRepositories.error && <p role="alert" className="text-sm text-destructive">{projectRepositories.error.message} <Button variant="ghost" size="sm" onClick={() => void projectRepositories.refetch()}>Try again</Button></p>}
{adopt.error && <p role="alert" className="text-sm text-destructive">{adopt.error.message}</p>}
{availableProjects.length > 0 && <div className="divide-y divide-border" aria-label="Project repositories">
{availableProjects.map(repository => <ProjectSkillRepositoryRow key={repository.workspaceId} repository={repository}
pending={adopt.isPending && adopt.variables === repository.workspaceId} onImport={() => adopt.mutate(repository.workspaceId)} />)}
</div>}
<div className="divide-y divide-border">
{query.data?.map(source => {
const installed = source.entries.filter(entry => entry.skillId);
const newCount = source.entries.filter(entry => entry.selection === 'new' && entry.present).length;
const result = results[source.id];
const localError = result && result.message !== source.lastError
&& (result.operation === 'settings' || !source.lastSuccessAt || new Date(source.lastSuccessAt).getTime() < result.failedAt)
? result.message : null;
return <section key={source.id} className="flex flex-col gap-3 py-4" aria-label={source.fullName}>
<div className="flex items-center gap-3">
<div className="flex min-w-0 flex-1 items-center gap-3"><GithubIcon className="size-5 shrink-0 text-muted-foreground" /><div className="min-w-0">
<div className="flex flex-wrap items-center gap-3">
<div className="flex min-w-0 basis-full items-start gap-3 sm:flex-1 sm:basis-auto"><GithubIcon className="size-5 shrink-0 text-muted-foreground" /><div className="min-w-0 flex-1">
<a href={source.repositoryUrl} target="_blank" rel="noreferrer" title={source.fullName} className="block truncate rounded-sm text-sm font-medium hover:underline focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring">{source.fullName}</a>
<p className="truncate text-xs text-muted-foreground">{source.trackingRef === 'HEAD' ? 'Default branch' : source.trackingRef}{source.lastSuccessAt ? ` · Updated ${timeAgo(source.lastSuccessAt)}` : ''}{source.accessScope === 'projects' ? ' · Project access' : ''}</p>
<div className="flex flex-wrap gap-x-3 gap-y-1">{source.projects?.map(project => <Link key={project.id} to={`/projects/${project.id}`} className="break-words text-xs text-muted-foreground hover:underline">{project.name}</Link>)}</div>
{source.enabled && source.automaticUpdates === false && <p className="text-xs text-muted-foreground">Automatic updates paused</p>}
{source.trackingActive === false && <p className="text-xs text-muted-foreground">Repository tracking stopped · installed skills kept</p>}
{!source.enabled && <p className="text-xs text-muted-foreground">Disconnected</p>}
{refresh.isPending && refresh.variables === source.id && <p role="status" className="text-xs text-muted-foreground">Refreshing…</p>}
{newCount > 0 && <Link to={`/skills/sources/${source.id}`} className="text-xs underline">{newCount} new {newCount === 1 ? 'skill' : 'skills'} available</Link>}
</div></div>
<SkillSourceSyncControls showPreference={false} automaticUpdates={source.automaticUpdates ?? true}
refreshing={source.syncState === 'refreshing' || (refresh.isPending && refresh.variables === source.id)}
disabled={!source.enabled || settings.isPending} onRefresh={() => refresh.mutate(source.id)}
onAutomaticUpdates={enabled => settings.mutate({ source, enabled })} />
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button size="icon-sm" variant="ghost" className="shrink-0" aria-label={`More actions for ${source.fullName}`}><MoreVertical className="size-4" /></Button>
<Button ref={button => { sourceActionButtons.current[source.id] = button; }} size="icon-sm" variant="ghost" className="shrink-0" aria-label={`More actions for ${source.fullName}`}><MoreVertical className="size-4" /></Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem disabled={!source.enabled || refresh.isPending} onSelect={() => refresh.mutate(source.id)}>{refresh.isPending && refresh.variables === source.id ? 'Refreshing…' : 'Refresh'}</DropdownMenuItem>
<DropdownMenuCheckboxItem checked={source.automaticUpdates ?? true} disabled={!source.enabled || settings.isPending}
onCheckedChange={enabled => settings.mutate({ source, enabled })}>Automatic updates</DropdownMenuCheckboxItem>
<DropdownMenuItem onSelect={() => navigate(`/skills/sources/${source.id}`)}>Select skills</DropdownMenuItem>
<DropdownMenuItem onSelect={() => setRepairSourceId(source.id)}>Choose connection</DropdownMenuItem>
<DropdownMenuItem variant="destructive" disabled={disconnect.isPending} onSelect={() => { disconnect.reset(); reloadRemoval.reset(); setDisconnectSource(source); }}>{source.enabled ? 'Disconnect source' : 'Remove source'}</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
@@ -100,24 +132,32 @@ export function SkillSources() {
<FileText className="size-3.5 shrink-0 text-muted-foreground" />
<span className="truncate font-medium group-hover:underline">{entry.name}</span>
{entry.description && <span className="min-w-0 flex-1 truncate text-xs text-muted-foreground">{entry.description}</span>}
{!entry.present && <span className="text-xs text-muted-foreground">Removed from source · installed copy kept</span>}
</Link>
</li>)}
</ul>}
{results[source.id] && <p role="status" className="text-xs text-muted-foreground">{results[source.id]}</p>}
{installed.length === 0 && source.entries.length === 0 && source.syncState !== 'refreshing' && !source.lastError && <p className="ml-8 text-sm text-muted-foreground">
No skills found yet.{source.enabled && source.automaticUpdates && source.importNewSkills ? ' New skills will be imported automatically.' : ''}
</p>}
{localError && <p role="alert" className="text-sm text-destructive">{localError}</p>}
<SkillImportWarnings warnings={skillImportWarnings(source.entries.filter(entry => entry.selection === 'selected'), source.repositoryPackage)} />
{source.lastError && source.lastAttemptAt && <p className="text-xs text-muted-foreground">Last attempt {timeAgo(source.lastAttemptAt)}</p>}
{source.lastError && <p role="alert" className="text-sm text-destructive">{source.lastError} <Link to={`/skills/sources/${source.id}`} className="underline">Review source</Link></p>}
{source.lastError && <Button variant="outline" size="sm" className="self-start" onClick={() => setRepairSourceId(source.id)}>Choose connection</Button>}
{source.connectionId && <Link to={`/apps/${source.connectionId}/permissions`} className="text-xs text-muted-foreground hover:text-foreground">Manage GitHub connection</Link>}
</section>;
})}
</div>
{repairSource && <SkillSourceConnectionRepair companyId={companyId} source={repairSource} onClose={() => setRepairSourceId(null)} onReturnFocus={() => sourceActionButtons.current[repairSource.id]?.focus()} onSaved={async () => {
setResults(prev => ({ ...prev, [repairSource.id]: null })); await invalidate();
}} />}
{disconnectSource && <DisconnectSourceDialog key={`${disconnectSource.id}:${disconnectSource.revision}`} source={disconnectSource}
pending={disconnect.isPending || reloadRemoval.isPending} error={reloadRemoval.error?.message ?? disconnect.error?.message}
stale={staleRemoval} onReload={() => reloadRemoval.mutate()} onChoiceChange={() => { if (!staleRemoval) disconnect.reset(); }}
onClose={() => setDisconnectSource(null)}
onConfirm={skills => disconnect.mutate({ source: disconnectSource, skills })} />}
{sourceId && (sourceId === 'new' || activeSource) && <SourceDialog key={`${companyId}:${sourceId}`} companyId={companyId} source={activeSource} onClose={() => navigate('/skills/sources')} onSaved={async result => {
setResults(prev => ({ ...prev, [result.source.id]: '' })); await invalidate(); navigate('/skills/sources');
setResults(prev => ({ ...prev, [result.source.id]: null })); await invalidate(); navigate('/skills/sources');
}} />}
{sourceId && sourceId !== 'new' && query.isSuccess && !activeSource && <p role="alert" className="text-sm text-destructive">Source not found.</p>}
</div>;
@@ -139,7 +179,7 @@ function DisconnectSourceDialog({ source, pending, error, stale, onReload, onClo
{count > 0 && <RadioCardGroup ariaLabel="Imported skills" value={choice}
onValueChange={value => { setChoice(value as SkillSourceDisconnectRequest['skills']); onChoiceChange(); }} disabled={pending}
options={[
{ value: 'copy', title: 'Keep editable copies', description: 'Move them to My Skills. Assigned agents keep using them.' },
{ value: 'copy', title: 'Keep editable copies', description: source.accessScope === 'projects' ? 'Move them to My Skills, keeping project access restrictions. Assigned agents keep using them.' : 'Move them to My Skills. Assigned agents keep using them.' },
{ value: 'delete', title: 'Delete skills', description: 'Remove them from the library. Detach skills used by agents first.' },
]} />}
ui/src/pages/apps/chat/ChatExecutionDefaults.test.tsx
+16 -2
View File
@@ -4,6 +4,7 @@ import { createRoot, type Root } from "react-dom/client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { ChatExecutionDefaults } from "./ChatExecutionDefaults";
import type { ChatExecutionDefaults as Defaults } from "@paperclipai/shared";
const mocks = vi.hoisted(() => ({ projects: vi.fn(), workspaces: vi.fn(), save: vi.fn() }));
vi.mock("@/api/projects", () => ({ projectsApi: { list: mocks.projects } }));
vi.mock("@/api/execution-workspaces", () => ({ executionWorkspacesApi: { listSummaries: mocks.workspaces } }));
@@ -14,8 +15,8 @@ beforeEach(() => {
client = new QueryClient({ defaultOptions: { queries: { retry: false } } });
});
afterEach(async () => { await act(async () => root.unmount()); client.clear(); container.remove(); });
async function render() {
await act(async () => root.render(<QueryClientProvider client={client}><ChatExecutionDefaults companyId="company" value={{ projectId: null }} resource onSave={mocks.save} /></QueryClientProvider>));
async function render(value: Defaults | null = { projectId: null }) {
await act(async () => root.render(<QueryClientProvider client={client}><ChatExecutionDefaults companyId="company" value={value} resource onSave={mocks.save} /></QueryClientProvider>));
await vi.waitFor(() => expect(container.querySelectorAll("select")[1].disabled).toBe(false));
}
async function choose(value: string) {
@@ -39,6 +40,19 @@ it("keeps edits visible after a save failure and permits retry", async () => {
expect(container.textContent).toContain("Saved for new tasks.");
});
it("retains accepted selections until the parent refreshes and then follows later defaults", async () => {
await render();
await choose("task");
await save();
expect(container.querySelectorAll("select")[1].value).toBe("task");
expect(container.querySelector<HTMLButtonElement>('button[type="submit"]')!.disabled).toBe(true);
expect(container.textContent).toContain("Saved for new tasks.");
await render({ projectId: null, workspace: { kind: "task_directory" } });
await render({ projectId: null, workspace: null });
expect(container.querySelectorAll("select")[1].value).toBe("clear");
expect(mocks.save).toHaveBeenCalledTimes(1);
});
it("offers and saves an idle workspace from the shared task selection filter", async () => {
mocks.workspaces.mockResolvedValue([
{ id: "active-workspace", name: "Active files", status: "active", closedAt: null },
ui/src/pages/apps/chat/ChatExecutionDefaults.tsx
+12 -6
View File
@@ -1,4 +1,4 @@
import { useId, useState } from "react";
import { useEffect, useId, useState } from "react";
import { useQuery } from "@tanstack/react-query";
import type { ChatExecutionDefaults as Defaults, TaskWorkspaceSelection } from "@paperclipai/shared";
import { projectsApi } from "@/api/projects";
@@ -25,13 +25,19 @@ export function ChatExecutionDefaults({ companyId, value, resource = false, onSa
const [draft, setDraft] = useState<Defaults | null>(null);
const [pending, setPending] = useState(false);
const [error, setError] = useState<string | null>(null);
const [saved, setSaved] = useState(false);
const [savedFrom, setSavedFrom] = useState<string | null>(null);
const projects = useQuery({ queryKey: queryKeys.projects.list(companyId), queryFn: () => projectsApi.list(companyId) });
const workspaces = useQuery({
queryKey: ["channel-workspace-choices", companyId],
queryFn: () => executionWorkspacesApi.listSummaries(companyId, { selectableForTask: true }),
});
const current = draft ?? value ?? {};
const valueKey = JSON.stringify(value ?? {});
const saved = savedFrom !== null;
// Keep the accepted selections until the parent publishes refreshed defaults.
useEffect(() => {
if (savedFrom !== null && savedFrom !== valueKey) setDraft(null);
}, [savedFrom, valueKey]);
const projectValue = current.projectId === undefined ? "inherit" : current.projectId ?? "none";
const workspaceValue = selectionKey(current.workspace);
const choices = new Map<string, TaskWorkspaceSelection>();
@@ -46,13 +52,13 @@ export function ChatExecutionDefaults({ companyId, value, resource = false, onSa
for (const workspace of workspaces.data ?? []) {
choices.set(`existing:${workspace.id}`, { kind: "existing", workspaceId: workspace.id });
}
const dirty = JSON.stringify(current) !== JSON.stringify(value ?? {});
const change = (next: Defaults) => { setDraft(next); setSaved(false); setError(null); };
const dirty = !saved && JSON.stringify(current) !== valueKey;
const change = (next: Defaults) => { setDraft(next); setSavedFrom(null); setError(null); };
return <form className="space-y-3" aria-labelledby={`${id}-title`} onSubmit={async (event) => {
event.preventDefault();
if (pending || !dirty) return;
setPending(true); setError(null); setSaved(false);
try { await onSave(Object.keys(current).length ? current : null); setDraft(null); setSaved(true); }
setPending(true); setError(null); setSavedFrom(null);
try { await onSave(Object.keys(current).length ? current : null); setSavedFrom(valueKey); }
catch (cause) { setError(cause instanceof Error ? cause.message : "Couldn’t save task defaults. Try again."); }
finally { setPending(false); }
}}>
ui/storybook/fixtures/githubSkillSources.ts
+8 -1
View File
@@ -181,6 +181,13 @@ export function installFixtures(empty: boolean, needsConnection: boolean, option
return structuredClone({ source, imported, updated: [], unchanged, warnings });
}
skillSourcesApi.list = async () => structuredClone(sources);
skillSourcesApi.projectRepositories = async () => [];
skillSourcesApi.settings = async (_companyId, id, input) => {
const source = sources.find(item => item.id === id)!;
Object.assign(source, { automaticUpdates: input.automaticUpdates ?? source.automaticUpdates ?? true,
importNewSkills: input.importNewSkills ?? source.importNewSkills ?? false, revision: source.revision + 1 });
return structuredClone(source);
};
const repositoryScenario = options.repositories ?? 'single';
skillSourcesApi.repositories = async () => {
if (repositoryScenario === 'loading') return new Promise(() => {});
@@ -262,7 +269,7 @@ export function installFixtures(empty: boolean, needsConnection: boolean, option
if (options.saving) await wait(3600000);
const source = sourceFixture();
Object.assign(source, { id: `source-${sources.length + 1}`, packageMode: input.packageMode, repositoryPackage: input.packageMode === 'repository' ? repositoryFixture(options.packageScenario) : null, repositoryUrl: input.repositoryUrl, fullName: input.repositoryUrl.replace("https://github.com/", ""), trackingRef: input.trackingRef || "main", connectionId: input.connectionId ?? null });
source.entries = repositoryCandidates(options.packageScenario).filter(entry => discoveryCandidates.some(candidate => candidate.path === entry.path)).map((entry, index) => ({ ...entry, id: `entry-${index}`, sourceId: source.id, present: true, selection: 'new', skillId: null }));
source.entries = repositoryCandidates(options.packageScenario).filter(entry => discoveryCandidates.some(candidate => candidate.path === entry.path)).map((entry, index) => ({ ...entry, id: `entry-${sources.length * 100 + index}`, sourceId: source.id, present: true, selection: 'new', skillId: null }));
sources.push(source);
return save(source, input.selectedPaths, input.excludedFolders ?? []);
};
ui/storybook/public/project-skills-evidence/automatic-attachment.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 68 KiB

ui/storybook/public/project-skills-evidence/connection-repair-mobile.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 34 KiB

ui/storybook/public/project-skills-evidence/execution-master-final-marker.txt
+1
View File
@@ -0,0 +1 @@
project-skill-v2
ui/storybook/public/project-skills-evidence/execution-review-cli-marker.txt
+1
View File
@@ -0,0 +1 @@
project-skill-v2
ui/storybook/public/project-skills-evidence/execution-v1-marker.txt
+1
View File
@@ -0,0 +1 @@
project-skill-v1
ui/storybook/public/project-skills-evidence/execution-v2-marker.txt
+1
View File
@@ -0,0 +1 @@
project-skill-v2
ui/storybook/public/project-skills-evidence/index.html
+1
View File
@@ -0,0 +1 @@
<!doctype html><html lang="en"><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Project skills acceptance evidence</title><style>body{max-width:72rem;margin:2rem auto;padding:1rem;font:1rem/1.6 system-ui;color:#222;background:#fff}img{display:block;max-width:100%;height:auto;border:1px solid #ddd}section{margin-block:2rem}a{color:#145ab8}</style><h1>Project skills acceptance evidence</h1><p>These captures and receipts use disposable integration data. The Sources and execution captures are from the real isolated app. The repair-dialog capture uses the shipped component with Storybook fixtures. The full acceptance report and current qualification status are in the pull request.</p><p><a href="https://github.com/paperclipai/paperclip/pull/16066">PR #16066</a> · <a href="versions.json">Execution and version receipts</a></p><section><h2>Real Sources page</h2><img src="sources-desktop.jpg" alt="Project repositories and installed skills in the real Sources page"></section><section><h2>Mobile connection repair</h2><img src="connection-repair-mobile.jpg" alt="Mobile GitHub connection repair dialog with reachable Cancel and Save controls"></section><section><h2>Mobile repair after a concurrent update</h2><p>Shipped components with Storybook fixtures. The selected account remains checked and Save stays reachable.</p><img src="repair-conflict-mobile.jpg" alt="Mobile repair conflict with selected account and reachable retry"></section><section><h2>Real routine execution</h2><img src="routine-execution-final.jpg" alt="Real runner confirms writing the updated project skill marker"></section><section><h2>Post-review routine execution</h2><img src="routine-execution-reviewed.jpg" alt="The real ACP routine reports the pinned v2 skill result"></section><section><h2>Supported CLI execution</h2><p>The pinned Codex 0.160.0 client verified the existing v2 file and completed the task. The earlier unsupported-client attempts are retained in the receipts.</p><img src="routine-execution-cli.jpg" alt="The supported CLI verifies the v2 marker and completes the routine task"></section><section><h2>Final repository attachment</h2><p>The real board UI attached the disposable repository to a new project. The background worker imported both skills from the default branch without a Skills setup or Refresh action.</p><img src="automatic-attachment.png" alt="Final project repository attachment automatically imports both skills"></section></html>
ui/storybook/public/project-skills-evidence/repair-conflict-mobile.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 34 KiB

ui/storybook/public/project-skills-evidence/routine-execution-cli.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 92 KiB

ui/storybook/public/project-skills-evidence/routine-execution-final.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 87 KiB

ui/storybook/public/project-skills-evidence/routine-execution-reviewed.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 91 KiB

ui/storybook/public/project-skills-evidence/routine-execution.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 78 KiB

ui/storybook/public/project-skills-evidence/sources-desktop.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 63 KiB

ui/storybook/public/project-skills-evidence/versions.json
+172
View File
@@ -0,0 +1,172 @@
{
"schema": "paperclip.project-skills-acceptance.v1",
"scope": "Disposable integration data only; source and execution receipts are separate from Storybook fixtures.",
"repository": "https://github.com/cryppadotta/paperclip-project-skills-20261012-5abf7178",
"initialCommit": "d7e95f01903678531981ce684e721191ecb0a7da",
"updatedCommit": "f4b35ea0082150bda320cda7b3c0777d263bf430",
"executions": [
{
"runId": "2db73573-eeaf-420e-aad8-898a87377342",
"versionId": "1ccdfa45-4a82-44ad-96cb-9ffa6e20fc64",
"marker": "project-skill-v1\n",
"outputSha256": "6d60a24f686d7786d52f89ccbc0f5dc46c5ccd4c569fde7bc026fb686c294e16",
"outputFile": "execution-v1-marker.txt"
},
{
"runId": "ae685356-02a9-4369-9bb3-b76bb112bcce",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"marker": "project-skill-v2\n",
"outputSha256": "6f0fe9a6f60b84811f2ae26bf2a632fd174750c4df155ae1776f657301ac3c9d",
"outputFile": "execution-v2-marker.txt"
},
{
"runId": "f003baa3-37d7-4203-bd4d-a0c4b45171ca",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"marker": "project-skill-v2\n",
"selectedImmutableVersions": {
"paperclipai/paperclip/complain": "b2492ac5-2b0f-4947-801a-a052a3b32700",
"paperclipai/paperclip/paperclip": "c6d0ff4f-5525-4aca-84de-0da670527113",
"paperclipai/paperclip/suggestion-box": "13b9f990-c7b4-4f91-9da2-763e3015803d",
"github/1416220511/7aa5a68e3a0220b8/acceptance-marker": "2bd9801c-1b96-417b-b081-2867e65afdcf"
},
"selectionSnapshot": true
},
{
"runId": "e2cbfc2b-aa07-4fa0-b240-82033d6d1035",
"implementationSha": "da5eace45e37546606c171f3acaaf4e19eacbea8",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"marker": "project-skill-v2\n",
"outputSha256": "6f0fe9a6f60b84811f2ae26bf2a632fd174750c4df155ae1776f657301ac3c9d",
"outputFile": "execution-v2-marker.txt",
"selectedImmutableVersions": {
"paperclipai/paperclip/complain": "b2492ac5-2b0f-4947-801a-a052a3b32700",
"paperclipai/paperclip/paperclip": "c6d0ff4f-5525-4aca-84de-0da670527113",
"paperclipai/paperclip/suggestion-box": "13b9f990-c7b4-4f91-9da2-763e3015803d",
"github/1416220511/7aa5a68e3a0220b8/acceptance-marker": "2bd9801c-1b96-417b-b081-2867e65afdcf"
},
"selectionSnapshot": true,
"model": "gpt-6.1-sol"
},
{
"runId": "b91818ac-9e9a-4a98-bd6b-63c0a58e4a1a",
"implementationSha": "614a2d1d72892128c7fc949d2485308756e7c687",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"marker": "project-skill-v2\n",
"outputSha256": "6f0fe9a6f60b84811f2ae26bf2a632fd174750c4df155ae1776f657301ac3c9d",
"outputFile": "execution-v2-marker.txt",
"selectedImmutableVersions": {
"paperclipai/paperclip/complain": "b2492ac5-2b0f-4947-801a-a052a3b32700",
"paperclipai/paperclip/paperclip": "5c62670f-f846-43dd-a76a-d8ec307030d0",
"paperclipai/paperclip/suggestion-box": "13b9f990-c7b4-4f91-9da2-763e3015803d",
"github/1416220511/7aa5a68e3a0220b8/acceptance-marker": "2bd9801c-1b96-417b-b081-2867e65afdcf"
},
"selectionSnapshot": true,
"earlierExecutionVersionId": "1ccdfa45-4a82-44ad-96cb-9ffa6e20fc64",
"attachmentId": "c2f15ba6-0c25-4a49-87fc-1b411c5b5a3b",
"model": "gpt-6.1-sol"
},
{
"runId": "83f6f277-5a4a-4814-ba55-8a7ed4a517e1",
"implementationSha": "cea9e474ba462ed9385ce2f8acb58c245585b310",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"marker": "project-skill-v2",
"outputSha256": "6f0fe9a6f60b84811f2ae26bf2a632fd174750c4df155ae1776f657301ac3c9d",
"outputFile": "execution-master-final-marker.txt",
"selectedImmutableVersions": {
"paperclipai/paperclip/complain": "b2492ac5-2b0f-4947-801a-a052a3b32700",
"paperclipai/paperclip/paperclip": "5c62670f-f846-43dd-a76a-d8ec307030d0",
"paperclipai/paperclip/suggestion-box": "13b9f990-c7b4-4f91-9da2-763e3015803d",
"github/1416220511/7aa5a68e3a0220b8/acceptance-marker": "2bd9801c-1b96-417b-b081-2867e65afdcf"
},
"selectionSnapshot": true,
"earlierExecutionVersionId": "1ccdfa45-4a82-44ad-96cb-9ffa6e20fc64",
"attachmentId": null,
"model": "gpt-6.1-sol",
"outputKind": "verified_workspace_file"
},
{
"runId": "95d14fcf-5643-4d5f-9512-d160efcb2446",
"implementationSha": "50b49124dc1617a045adbe05937bcccca8486d86",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"marker": "project-skill-v2",
"selectedImmutableVersions": {
"paperclipai/paperclip/complain": "b2492ac5-2b0f-4947-801a-a052a3b32700",
"paperclipai/paperclip/paperclip": "5c62670f-f846-43dd-a76a-d8ec307030d0",
"paperclipai/paperclip/suggestion-box": "13b9f990-c7b4-4f91-9da2-763e3015803d",
"github/1416220511/7aa5a68e3a0220b8/acceptance-marker": "2bd9801c-1b96-417b-b081-2867e65afdcf"
},
"selectionSnapshot": true,
"model": "gpt-6.1-sol",
"engine": "acp",
"outputKind": "provider_reported_workspace_file",
"outputFile": null,
"attachmentId": null
},
{
"runId": "d94d7009-8787-4483-81e4-71ec0b512ed6",
"implementationSha": "50b49124dc1617a045adbe05937bcccca8486d86",
"versionId": "2bd9801c-1b96-417b-b081-2867e65afdcf",
"selectedImmutableVersions": {
"paperclipai/paperclip/complain": "b2492ac5-2b0f-4947-801a-a052a3b32700",
"paperclipai/paperclip/paperclip": "5c62670f-f846-43dd-a76a-d8ec307030d0",
"paperclipai/paperclip/suggestion-box": "13b9f990-c7b4-4f91-9da2-763e3015803d",
"github/1416220511/7aa5a68e3a0220b8/acceptance-marker": "2bd9801c-1b96-417b-b081-2867e65afdcf"
},
"marker": "project-skill-v2",
"outputSha256": "6f0fe9a6f60b84811f2ae26bf2a632fd174750c4df155ae1776f657301ac3c9d",
"outputFile": "execution-review-cli-marker.txt",
"outputKind": "verified_existing_workspace_file",
"existingOutputPreserved": true,
"selectionSnapshot": true,
"model": "gpt-6.1-sol",
"engine": "cli",
"codexCliVersion": "0.160.0",
"homeScopedToPermittedInventory": true,
"attachmentId": null
}
],
"upstreamUpdate": {
"observedAt": "2026-10-12T23:21:32.296Z",
"sources": [
{
"id": "96c64152-bf43-440d-9a2c-6c05c213cc90",
"repositoryUrl": "https://github.com/cryppadotta/paperclip-project-skills-20261012-5abf7178",
"trackingRef": "main",
"lastScanCommit": "f4b35ea0082150bda320cda7b3c0777d263bf430",
"lastSuccessAt": "2026-10-12T23:21:03.269Z",
"entries": [
{
"path": ".agents/skills/acceptance-marker/SKILL.md",
"name": "acceptance-marker",
"skillId": "2c6b5d31-782e-4dca-9c65-91efbcfbc346",
"selection": "selected",
"present": true
},
{
"path": ".agents/skills/new-project-skill/SKILL.md",
"name": "new-project-skill",
"skillId": "7b4ea060-617f-4164-82bf-9cfa161e579e",
"selection": "selected",
"present": true
}
]
}
]
},
"failedProbes": [
{
"runId": "4a4f5265-73ff-4430-a5e8-28cc35ebe06b",
"engine": "cli",
"hostCliVersion": "0.153.4",
"model": "gpt-6.1-sol",
"reason": "Provider rejected model for the stale client."
},
{
"runId": "483f9b3a-e03e-4ba4-9080-432de15163f5",
"engine": "cli",
"hostCliVersion": "0.153.4",
"model": "gpt-5.4",
"reason": "Provider rejected model for the stale client."
}
]
}
ui/storybook/stories/connection-setup/in-feed-connections.stories.tsx
+8
View File
@@ -331,6 +331,14 @@ const aiRepair: Story = {
await expect(within(document.body).queryByRole("dialog")).not.toBeInTheDocument();
},
};
export const AiWithdrawn: Story = {
...card({ ...aiPending, status: "cancelled", resolvedAt: new Date("2026-10-13T14:09:07Z") }, { ai: true }),
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(await canvas.findByText("Connection request withdrawn")).toBeVisible();
await expect(canvas.queryByRole("button", { name: "Fix connection" })).not.toBeInTheDocument();
},
};
export const AiInlineRepair = aiRepair;
export const AiInlineRepairNarrow: Story = { ...aiRepair, globals: { viewport: { value: "mobile1", isRotated: false } } };
export const AiRepairCancel: Story = { ...aiRepair, play: async context => {
ui/storybook/stories/skills/project-repository-skills/README.md
+21
View File
@@ -0,0 +1,21 @@
# Project repository skills
Open **Agents and reusable capabilities → Skills and Skill Studio → Project repository skills** in Storybook.
- **01 Components**: legacy adoption, progress, connection errors, private access,
automatic-update preference and manual refresh controls.
- **02 Journeys**: production project creation, Sources, library and routine pages
in the production app shell. Create a project with `acme/team-skills`, then
navigate to Skills → Sources. Legacy adoption starts before import. Background
update introduces skills without a click. Retry the failed refresh, pause updates,
inspect the private/shared source and try the mobile and light-theme variants.
Connection repair chooses an existing authorized account without changing skill
selections. Also inspect removed skills, invalid upstream content, two private
projects and the large-list story.
All GitHub, import, scheduled update and routine execution responses are scoped
in-memory fixtures, reset between stories. They do not prove live synchronization,
credential refresh or model execution. Production acceptance evidence is recorded
in `doc/plans/2026-10-12-project-repository-skills.md`.
Browser checks: `tests/storybook-visual/project-repository-skills.spec.ts`.
ui/storybook/stories/skills/project-repository-skills/components.stories.tsx
+23
View File
@@ -0,0 +1,23 @@
import { useState } from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { ProjectSkillRepositoryRow } from '@/components/skill-studio/ProjectSkillRepositoryRow';
import { SkillSourceSyncControls } from '@/components/skill-studio/SkillSourceSyncControls';
import { projectRepository } from '../../../support/projectRepositorySkills';
function Components({ state = 'available' }: { state?: 'available' | 'pending' | 'error' | 'private' | 'paused' | 'refreshing' }) {
const [imported, setImported] = useState(false), [automatic, setAutomatic] = useState(state !== 'paused');
return <div className="mx-auto flex max-w-3xl flex-col gap-6 p-6">
<ProjectSkillRepositoryRow repository={{ ...projectRepository, status: imported || state === 'pending' ? 'pending' : state === 'error' ? 'error' : 'available',
projectVisibility: state === 'private' ? 'private' : 'open', lastError: state === 'error' ? 'GitHub access is unavailable. Reconnect GitHub and retry.' : null }} onImport={() => setImported(true)} />
<SkillSourceSyncControls automaticUpdates={automatic} refreshing={state === 'refreshing'} onAutomaticUpdates={setAutomatic} onRefresh={() => setImported(true)} />
</div>;
}
const meta = { id: 'skills-project-repository-skills-01-components', title: 'Agents and reusable capabilities/Skills and Skill Studio/Project repository skills/01 Components', component: Components, parameters: { paperclip: {"capabilities": ["skills/discovery", "skills/versions"], "status": "current", "implementation": "production", "boundary": "fixtures", "notes": "Production Sources controls with fixture data cover empty, updating, paused, removed, invalid and private-project states. Provider access, persistence and runtime delivery require live acceptance."}, layout: 'fullscreen' } } satisfies Meta<typeof Components>;
export default meta;
type Story = StoryObj<typeof meta>;
export const Available: Story = {};
export const Pending: Story = { args: { state: 'pending' } };
export const Error: Story = { args: { state: 'error' } };
export const Private: Story = { args: { state: 'private' } };
export const Paused: Story = { args: { state: 'paused' } };
export const Refreshing: Story = { args: { state: 'refreshing' } };
ui/storybook/stories/skills/project-repository-skills/journeys.stories.tsx
+64
View File
@@ -0,0 +1,64 @@
import { useEffect, useRef, useState } from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { useQueryClient } from '@tanstack/react-query';
import { Layout } from '@/components/Layout';
import { SkillSources } from '@/pages/SkillSources';
import { CompanySkills } from '@/pages/CompanySkills';
import { Projects } from '@/pages/Projects';
import { ProjectDetail } from '@/pages/ProjectDetail';
import { RoutineDetail } from '@/pages/RoutineDetail';
import { Routines } from '@/pages/Routines';
import { IssueDetail } from '@/pages/IssueDetail';
import { PluginLauncherProvider } from '@/plugins/launchers';
import { Route, Routes, useNavigate } from '@/lib/router';
import { installProjectSkillFixtures, ROUTINE_ID, type Scenario } from '../../../support/projectRepositorySkills';
function Journey({ scenario = 'ready' }: { scenario?: Scenario }) {
const client = useQueryClient(), navigate = useNavigate();
const initial = useRef(navigate);
const [ready, setReady] = useState(false);
useEffect(() => {
const cleanup = installProjectSkillFixtures(scenario, client);
initial.current(scenario === 'new-project' ? '/projects' : scenario === 'attach-repository' ? '/projects/project-skills/configuration' : scenario === 'latest' ? `/routines/${ROUTINE_ID}` : '/skills/sources', { replace: true });
setReady(true); return cleanup;
}, [scenario, client]);
if (!ready) return null;
return <PluginLauncherProvider><Routes><Route path="/:companyPrefix" element={<Layout />}>
<Route path="projects" element={<Projects />} />
<Route path="projects/:projectId/*" element={<ProjectDetail />} />
<Route path="skills/sources" element={<SkillSources />} />
<Route path="skills/sources/:sourceId" element={<SkillSources />} />
<Route path="skills/*" element={<CompanySkills />} />
<Route path="routines/:routineId" element={<RoutineDetail />} />
<Route path="routines/:routineId/:section" element={<RoutineDetail />} />
<Route path="routines" element={<Routines />} />
<Route path="issues/:issueId" element={<IssueDetail />} />
</Route></Routes></PluginLauncherProvider>;
}
const meta = { id: 'skills-project-repository-skills-02-journeys', title: 'Agents and reusable capabilities/Skills and Skill Studio/Project repository skills/02 Journeys', component: Journey, tags: ['!autodocs'],
parameters: { paperclip: {"capabilities": ["skills/discovery", "skills/versions", "projects/repositories", "routines/run-now", "routines/history"], "status": "current", "implementation": "production", "boundary": "fixtures", "notes": "Production app shell and pages exercise project creation/attachment, legacy adoption, refresh/repair and routine revision journeys with in-memory GitHub and execution fixtures. Live authorization and GitHub-to-run delivery are verified separately."}, layout: 'fullscreen', docs: { description: { component: 'Production pages and app shell. Imports, GitHub responses, background refresh and routine executions are in-memory fixtures. Start with Create project or Legacy adoption; all actions remain interactive. Live integration is verified separately.' } } },
} satisfies Meta<typeof Journey>;
export default meta;
type Story = StoryObj<typeof meta>;
export const CreateProject: Story = { name: '01 · Create a project', args: { scenario: 'new-project' } };
export const LegacyAdoption: Story = { name: '02 · Import from an older project', args: { scenario: 'legacy' } };
export const Synced: Story = { name: '03 · Automatically imported skills' };
export const AutomaticUpdate: Story = { name: '04 · Background update and a new skill', args: { scenario: 'automatic' } };
export const Refreshing: Story = { name: '05 · Refresh in progress', args: { scenario: 'refreshing' } };
export const FailedRefresh: Story = { name: '06 · Retained skills and retry', args: { scenario: 'error' } };
export const EmptyRepository: Story = { name: '07 · No skills yet', args: { scenario: 'empty' } };
export const PrivateProject: Story = { name: '08 · Private project access', args: { scenario: 'private' } };
export const SharedRepository: Story = { name: '09 · Access through either project', args: { scenario: 'shared' } };
export const RoutineReference: Story = { name: '10 · Routine follows the installed skill', args: { scenario: 'latest' } };
export const LongRepository: Story = { name: '11 · Long repository name', args: { scenario: 'long' } };
export const Mobile: Story = { name: '12 · Mobile', globals: { viewport: { value: 'mobile1', isRotated: false } } };
export const Light: Story = { name: '13 · Light theme', globals: { theme: 'light' } };
export const PausedUpdates: Story = { name: '14 · Automatic updates paused', args: { scenario: 'paused' } };
export const RemovedSkill: Story = { name: '15 · Removed skill retained', args: { scenario: 'removed' } };
export const InvalidUpdate: Story = { name: '16 · Invalid upstream content', args: { scenario: 'validation' } };
export const ConnectionRepair: Story = { name: '17 · Repair setup connection', args: { scenario: 'repair' } };
export const TwoPrivateProjects: Story = { name: '18 · Two private projects', args: { scenario: 'shared-private' } };
export const ManySkills: Story = { name: '19 · Large skill list', args: { scenario: 'many' } };
export const AttachRepository: Story = { name: '20 · Attach another repository', args: { scenario: 'attach-repository' } };
export const RepairConflict: Story = { name: '21 · Repair during a background update', args: { scenario: 'repair-conflict' } };
Loaded 100 of 101 files, more files were not shown because too many files have changed in this diff. Show more