mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 16:27:18 +02:00
aaeff418cf72341c674ff3393cd94f40dc4855f6
1130
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
aaeff418cf |
fix(acpx): add a positive control to stalled-tool permission evidence (#16769)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The ACPX engine runs Claude agents. A watchdog ends a turn when a pending tool call gets no update for 30 minutes. > - #16688 added a field to stalled runs. The field records if the stalled call's permission request reached Paperclip. > - Recent stalled runs show `acpStalledToolPermissionRequested: false`. We can read this as "the call stopped before its permission request". That reading is correct only if calls of the same kind usually send a request. > - We have no data that shows if finished calls of that kind sent requests. Thus we cannot tell if the missing request is a real signal. > - This pull request records a positive control: the number of finished calls of the stalled call's kind, and how many of them sent a permission request. > - The benefit: the next stall event tells us if the missing request locates the stall, or if it gives no information. ## Linked Issues or Issue Description Refs #16688 **What happened?** Runs stop with the stalled-tool watchdog error. The diagnostics show that the stalled call sent no permission request. The diagnostics do not show if finished calls of the same kind sent requests. **Expected behavior** The stalled-run diagnostics contain enough data to show if a missing permission request is a real signal. **Steps to reproduce** A pending tool call gets no ACP update for 30 minutes. The watchdog ends the turn. The run result contains the stalled-tool fields. ## What Changed - `packages/adapter-utils/src/acpx-engine/execute.ts`: when the watchdog fires, count the finished calls of the stalled call's kind, and count how many of them sent a permission request. Write the counts to the run result as `acpSettledSameKindToolCount` and `acpSettledSameKindToolPermissionRequestedCount`. - `server/src/services/run-failure-diagnostics.ts`: allow the two new fields as non-negative safe integers. Other values are dropped. - `doc/observability.md`: explain how to read the two fields with the existing permission-request field. - Tests: extend the existing stalled-tool tests in `execute.test.ts` and `run-failure-diagnostics.test.ts`. ## Verification - `pnpm exec vitest run src/acpx-engine/execute.test.ts` in `packages/adapter-utils`: 244/244 pass. - `pnpm exec vitest run src/services/__tests__/run-failure-diagnostics.test.ts` in `server`: 115/115 pass. - Mutation check: remove the kind filter, or count every finished call as "requested". Each mutation makes the stalled-tool test fail. - `tsc --noEmit` passes in `packages/adapter-utils` and in `server`. Note: the full `server` `typecheck` script did not run locally, because the runner vendor build step failed in this sandbox. I ran `tsc --noEmit` directly. ## Risks - Low risk. The change only adds two counts to the run result after the watchdog fires. It does not change the decision to stop the turn, or any permission decision. - The fields hold only counts. They contain no tool titles, inputs, or identifiers. This is the run-log and run-result path. It is not Telemetry. The fields reach Sentry through the existing allowlist in `run-failure-diagnostics.ts`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
1558d9962c |
fix(idle): exclude completed feedback history from background work (#16764)
Allow private feedback history and acknowledged uploads to stop blocking idle sleep. Pending, failed, unfamiliar, and inconsistent delivery records remain blocked; preserve all other work and cleanup gates. Verified with 220 idle-safety tests, workspace typecheck/build, full CI, and Greptile Apex 5/5 on the reviewed head. Local full-suite limitations and isolated rechecks are documented in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
5dced3c766 |
perf(tools): narrow profile and attention catalog reads (#16761)
Read only the catalog, connection, and invocation fields needed by profile summaries, reviews, and attention counts. Preserve company scope and response contents while reducing query payloads. Verified with 445 affected service tests, workspace typecheck/build, full CI, and Greptile Apex 5/5 on the reviewed head. Local full-suite limitations and isolated rechecks are documented in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c9669a6c22 |
docs(grok): record production qualification and Cloud setup (#16722)
## Thinking Path > - Paperclip lets people manage AI agents and their work. > - Grok Bot uses the shared Runner and an agent MCP connection. > - The experimental Cloud release now passed real production setup and work. > - The setup guide does not list the Cloud opt-in controls or the observed routine-panel workaround. > - The readiness plan still describes the production rollout as pending. > - This pull request records the release evidence and the supported setup path. ## Linked Issues or Issue Description **Issue type** Outdated documentation and missing setup instructions. **Where is the issue?** `doc/grok-bot-agent.md` and `doc/plans/2026-10-13-grok-bot-production-readiness.md`. **What's wrong?** The guide starts after the opt-in step. The plan predates completed production acceptance and the visibility rollout. **Suggested fix** List the three opt-ins and the Agents entry path. Record the exact tested versions, rollout result and limits. Related runtime work: [#16394](https://github.com/paperclipai/paperclip/pull/16394), [#16444](https://github.com/paperclipai/paperclip/pull/16444), [#16445](https://github.com/paperclipai/paperclip/pull/16445). ## What Changed - Add the Cloud MCP, Runner and Grok opt-in steps. - Explain the supported external-agent entry and the Grok Webhook-panel link workaround. - Record both tested production versions, assigned and proactive results, cleanup, the 43-target rollout and a fresh-workspace check. - Keep the historical failure and evidence limits explicit. ## Verification - `git diff --check origin/master...HEAD` passed. - Compared the report with saved production receipts, serving-version observations, provider image digest, authenticated challenge, download hashes and cleanup receipts. - Real assigned work, question/answer, document/file delivery and proactive work passed on Core `4415665d67b16ffefbe4519c4bdd521f37bb87b9`. - Current production source CI passed its required lanes. The collected non-export checks contain 50 successes, 29 skips and no failures. - A fresh Cloud workspace exposed the opt-ins, saved them after reload and offered Grok in the external-agent picker. All test switches were then restored to off. - This change edits only Markdown. Runtime typecheck, tests, build, token and Storybook checks were not rerun locally for this documentation commit. The plan distinguishes earlier staging recovery proof from current production tests. ## Risks Low runtime risk. This change edits no source, schema, workflow or feature setting. The report qualifies the named versions only. The first-company model wizard still does not offer external agents. Grok usage, spending and remote termination remain unknown. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools and browser/native-app testing. The session does not expose the exact deployed model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, including the runtime metadata that is available - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the applicable local documentation checks and they pass; runtime tests were not rerun for Markdown-only edits - [x] I have added or updated tests where applicable; no new tests apply to these documentation edits - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (55 passed; two optional Storybook skips) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bfadb66556 |
fix(daytona): retain already archived workspaces on stop (#16710)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Stop-only recovery must keep unfinished remote workspace files safe. > - Daytona can archive a stopped sandbox while those files remain in object storage. > - The stop handler sent another Stop request for that archived source. > - Daytona rejected that request, so recovery could not record its required receipt. > - This change accepts the freshly confirmed archived state after retention is verified. > - Recovery can proceed while the source stays cold and its files remain available. ## Linked Issues or Issue Description Refs: #16583, #16700. **What happened?** Stop-only cleanup rejected a retained workspace after Daytona archived it. The live provider returned `DaytonaValidationError` for Stop while its state was `archived` and auto-delete was already disabled. The host correctly kept the recovery gate closed. **Expected behavior** Confirm the exact source and its retention policy. Treat an archived source as having no running process. Record the stop-only receipt without starting or deleting the source. **Steps to reproduce** 1. Keep an unfinished native workspace with stop-and-retain intent. 2. Let its stopped Daytona sandbox become archived. 3. Request stop-only cleanup. 4. Observe the rejected Stop request and missing receipt. ## What Changed - Accept `archived` after a fresh provider read confirms that auto-delete is disabled. - Stop the source if that fresh read instead shows it has resumed. - Add regression cases for confirmed retention, unconfirmed retention, and a resumed cached handle. - Update the recovery runbook and capability map. ## Verification - The new baseline fixture failed on the old handler: 1 failed, 2 passed. - Daytona provider suites: 417 passed, 6 skipped. - Full workspace `pnpm -r typecheck`: passed. - Full workspace `pnpm build`: passed on `fc8b1c1bb8`. - Maps and Git whitespace checks: passed. - The live provider rejection was reproduced on the exact retained source. No model turn, file change or delete was performed. The fixed source handler was then run against that same archived allocation with the declared SDK version. It returned the stopped-process receipt while the provider stayed archived with auto-delete disabled. No process was started. All 58 exact-head CI checks passed; independent review is 5/5. A full local run is also in progress on a separate frozen checkout of this revision; it is not reported as complete. - [Daytona lifecycle documentation](https://www.daytona.io/docs/en/sandboxes/) confirms that archive follows Stop and preserves the filesystem in object storage. ## Risks - An archived source grants a generic stopped-process receipt. Its physical provider state stays archived. - Retention confirmation remains required. A stale archived handle that has resumed still requires Stop. - The host still checks the original allocation, ownership, stop receipt and file-recovery evidence before continuation. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6bec153410 |
fix(runtime): retain cancelled workspaces before reconstruction (#16700)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs must preserve files when Stop ends a turn before its result is saved. > - The cancellation path could skip file retention and leave a prepared remote workspace active. > - An older controller could also admit an extra run while an earlier lost-source hold was unresolved. > - Rejecting every later run then prevented recovery even after that extra run was stopped and its files were reconciled. > - This change retains cancelled workspaces and records the exact proof for a safely reconciled intervening cancellation. > - Recovery can proceed without changing cancelled history, replaying actions, or weakening execution gates. ## Linked Issues or Issue Description Refs: #16583, #16690, #16696. **What happened?** Stop could acknowledge a native turn without recording its unsaved file obligation. Reconstruction also rejected a later cancelled run after its exact environment and files had been reconciled. The task could remain inactive with an `in_progress` status, which the reconstruction endpoint rejected. **Expected behavior** Retain unsaved files after Stop. Permit a board operator to reconstruct from saved sources only after each intervening cancellation has a confirmed stop and a file-only repair record. Preserve the original failed export and the cancelled run. **Steps to reproduce** 1. Prepare a native workspace and cancel the turn before a structured result exists. 2. Inspect its file retention and remote lease state. 3. For a task with an older destroyed export, reconcile the later cancelled run's exact files and unperformed actions. 4. Request reconstruction from verified same-task attachments. ## What Changed - Pin prepared native files before cancellation settlement. Defer teardown if pinning fails. Use the existing stop-only cleanup worker. - Repair active warm and ephemeral sources, plus stopping and confirmed-stopped historical cancellations in bounded startup and periodic batches. Preserve the existing exact-source checks. Bound row-lock waits and advance past a busy source. - Accept only same-owner native run-only cancellations with no result or current coordinator, confirmed stopped leases, and board file-only repair with `not_performed` action outcomes. - Save hashes of each intervening repair and recheck them at queue claim and final dispatch. Reject changed evidence or another later run. - Expose saved-source reconstruction in the issue API for inactive `blocked` and `in_progress` tasks, and hide it when an execution or checkout owner exists. Permit the same inactive source at submission. Keep the source, owner, version, attachment, budget, pause and retry checks. - Update the shared evidence type, recovery runbook and capability maps. Add an optional cancelled-native no-action confirmation to file recovery, resetting evidence when its source changes. No migration, new endpoint, permission or dependency. ## Verification - The new fixture fails on the original validator with `workspace_reconstruction_stale` after valid cancelled-run file repair. - Focused source and Stop retention tests: 94 passed before the final contention case. The subsequent three-suite run passed 123 tests, including real PostgreSQL lock contention and public startup/API/final-dispatch paths. The final narrow-query rerun passed all 123 tests. Review correction: all 143 tests passed across four suites, including real lock contention through the teardown boundary and stopping/stopped allocations. - Six broader cancellation, executor, heartbeat and restart suites: 831 passed, 9 existing skips. - Final correction: all 145 backend tests and all 36 recovery UI tests passed. Real contention covers both warm reusable and ephemeral sources. Optional no-action confirmation defaults to uncertified, and changing source clears evidence. Workspace `pnpm -r typecheck` and token gates passed on these sources. - Maps, module boundaries and Git whitespace checks: passed. Full `pnpm build` passed on the final `4e8c0e73ac` revision as well as the prior reviewed heads. A full local run of the preceding fix completed with 21,724 passes, 109 skips and three failures in webhook body limits, a status-card timeout, and a port-allocation fixture. All three suites then passed unchanged in an isolated rerun (209 tests). The full local run remains recorded as failed; the final head's full CI passed (55 completed checks, no failures). A fresh full local run is also in progress on the frozen reviewed head. - Preserved failures: the first attempt lacked a built plugin SDK; after building prerequisites, the regression reached the expected validator rejection. Treatment tests caught the file-only action's deliberately null continuation owner. A real lock test then caught the database driver's wrapped timeout error. Both defects were corrected and retested. Independent review then found a teardown race after a failed retention write. A second review found active warm sources excluded by the historical policy filter; the regression reproduced both warm failures before correcting the query. Its real-row-lock fixture reproduced the failure; the fixed release guard and historical stopping/stopped cases pass. No checks were weakened. - A pre-deployment walkthrough caught the issue API still hiding reconstruction for an inactive `in_progress` task. The status/capability baseline reproduced three failures, including inappropriate capability flags for execution/checkout owners. The fix uses the same inactive-state requirements as submission; its positive fixture reads the assembled issue API before reconstructing. All 117 tests in those three recovery suites pass, and all 28 heartbeat-recovery tests pass separately. Full workspace typecheck passed again. Full build passed with the exact `4e8c0e73ac` stamp; latest-head independent review is 5/5 with no open findings. All 55 latest-head checks completed with no failures. - Live tenant acceptance remains pending. No successful reconstruction, export or warm-process proof is claimed by these local tests. ## Risks - Stop retains unfinished work, so storage remains until file recovery and explicit cleanup complete. - Reconstruction remains held for unknown or performed external actions, foreign owners, live or rebound environments, accepted results, changed evidence, and unrelated later runs. - The sweep examines at most five candidates per pass. It uses a fair cursor and bounded transaction waits. It performs no provider call and starts no new turn. - Intervening verification reads only the needed JSON fields. The source transaction still serializes recovery for the same task. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
69ac928d2b |
fix(runtime): preserve bounded retries after workspace reconstruction (#16696)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An operator can rebuild a destroyed workspace from saved task attachments. > - The first reconstruction run must pass current source and execution checks. > - A temporary bootstrap failure can create a bounded retry before provider work starts. > - The retry has a new run ID, so the first-run-only reconstruction check rejects it. > - This change verifies the retry chain back to the approved first run and keeps current safety checks. ## Linked Issues or Issue Description Refs #16690. **What happened?** After an operator records workspace reconstruction, a safe bootstrap retry is cancelled before provider startup. The normal retry scheduler creates the successor. The reconstruction check then rejects its immediate predecessor because it expects the original failed export run. **Expected behavior** A verified automatic retry can continue under the same reconstruction decision and remaining retry budget. Changed sources, unrelated executions, uncertain outcomes, pause and later human blocks must still stop startup. **Steps to reproduce** Record reconstruction with a retry budget remaining. Let its first run fail before provider work starts with recorded bootstrap evidence. Use the normal bounded retry scheduler and the locked dispatch gate. Before this change, the scheduler succeeds but dispatch returns false. ## What Changed - Verify the predecessor chain back to the recovery action's exact first continuation. - Check company, task, agent, terminal predecessor state and each scheduler-owned wake. Reject cycles, foreign parents and forged wakes. - Exclude only those verified predecessors from the unrelated-newer-run check. Keep source hashes, original result identity and all current execution gates. - Reuse the existing reconciliation proof for failed predecessors. Keep the original retry limit. - Add real PostgreSQL scheduler and assembled-heartbeat regressions. Update the recovery guide and feature map. ## Verification - The baseline regression failed at final dispatch after the normal retry scheduler created its successor. - The corrected focused group passes 133 tests across reconstruction, retry scheduling and dispatch. It admits two bounded bootstrap retries and refuses a third. The assembled heartbeat enters the provider once for an eligible retry and zero times after a controlled post-claim human block. - An intermediate negative case exposed missing predecessor-outcome validation. Reusing the existing reconciliation proof fixes it. Deleted sources, pause, forged wakes, foreign parents, unrelated runs and new recovery holds remain refused. - Full workspace typecheck and build pass on rebased head `bd301ec8a703fa1cdc9e1db2638440d203e8d57a`. The rebased focused group passes all 133 tests. All CI gates pass on that exact head, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 with no unresolved threads. A clean full local `pnpm test:run` is running; the local source stays frozen for that run. - Module boundaries, maps and diff whitespace checks pass. Live staging acceptance follows the qualified full Cloud deployment. - The earlier full local run for #16690 failed while files changed and its PATH hid `lsof`. All five affected suites then passed in a clean run: 246 passed, three skipped. Those results do not establish a full local pass for this follow-up. ## Risks - Retry ancestry is a rare recovery path. Its primary-key reads grow with the chain and stop at 64 predecessors. Ordinary runs perform no new ancestry reads. - Unknown effects and active recovery obligations stay held. This does not reset budgets, change provider lifecycle policy, grant access or replay completed actions. - No schema migration or new API is included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks; full-run scope is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2befcd9a0a |
Harden Board CLI approval and native verification fixtures (#16482)
Preserve governed Board CLI authority and strict verification fixtures on the Muse stack.
Keep approval/cancel ownership, current membership/key authority and durable first-change audit atomic. Bound Retry-After scheduling within the original challenge; never automatically resubmit consent. Retain descriptor ownership until identity-based cleanup and close it on success and error. Keep strict Git-origin and synchronized provider-entry fixture assertions.
Related public work: LukeAbell's existing PR #13919 proposes the same production descriptor-lifetime fix. This independently measured local integration retains the original tests and adds real-descriptor coverage. Do not treat this local integration as ownership of that contribution.
Carry actual merged UI source
|
||
|
|
63306f6660 |
fix(acpx): record whether a stalled tool call's permission request arrived (#16688)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted `claude_local` runs use the ACPX engine. Claude can start
background subagents, and those subagents call the runtime MCP tools
that Paperclip attaches.
> - #16671 added a stalled-tool watchdog. It ends a Claude turn when a
pending tool call gets no ACP update for 30 minutes. That PR said the
most likely cause of the stall is a lost ACP permission round trip, but
it could not prove it.
> - A staging run tripped the watchdog. Two background subagents started
12 MCP calls. ACP reported each call as `pending`, and no call got
another update. The MCP gateway logs show the 9 earlier calls from the
parent agent, but no request from the 12 subagent calls.
`MCP_TOOL_TIMEOUT` (15 minutes) did not end them. So the calls stalled
before Claude sent them to the MCP server.
> - The run record cannot say if the permission request for a stalled
call reached Paperclip. Without that fact, the next stall gives no new
evidence about the cause.
> - This pull request records, for a stalled run, whether the stalled
call's permission request reached Paperclip, and how many pending calls
had a permission request.
> - The benefit is that the next stall tells us which side of the
permission round trip stopped, so the real fix can go to the correct
layer.
## Linked Issues or Issue Description
- Refs #16671 (the stalled-tool watchdog; this PR tests the cause that
PR suspected)
- Refs #16360 (the MCP client deadline that cannot reach a call that was
never sent)
**What happened?**
A hosted staging `claude_local` run started two background subagents.
Each subagent started six `execute-sentry-tool` MCP calls. ACP reported
all 12 calls as `pending`. None got a result, an update, or an error.
The MCP gateway received no request for them. The parent turn finished
its answer but stayed open for the subagents. After 31 minutes the
watchdog ended the turn with `acpx_tool_stalled`. A single background
subagent with six parallel calls to the same tool did not stall in a
later test.
**Expected behavior**
When a tool call stalls, the run record shows if the call's permission
request reached Paperclip. Then the stall can be put on the agent side
or on the relay side of the round trip.
**Steps to reproduce**
1. Start an ACP `claude_local` run in a sandbox that uses two background
subagents to call runtime MCP tools.
2. Wait until a subagent's tool calls stay `pending` with no further ACP
update (in staging this happened without a known trigger).
3. See that the watchdog fails the run with `acpx_tool_stalled`, but the
record has no evidence about the permission round trip.
**Paperclip version or commit**
`master` at
|
||
|
DottaandPaperclip
|
896f3c7f3b |
Add personal Muse operator setup and composer controls (#16450)
Add Muse invitation, connection readiness and scoped operator controls.
Keep connection state tied to the current company, person, agent and binding. Surface failed reads and invalidate stale cleanup observations; require current authority and exact readback. Preserve Dot/Grok setup, source-trust limits and existing design tokens.
Carry actual merged backend
|
||
|
DottaandPaperclip
|
7c7d2d8d2c |
fix(runtime): continue from saved sources after workspace loss (#16690)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runtime must preserve saved work and prevent duplicate execution. > - Older cleanup could destroy a sandbox before its workspace export finished. > - The existing recovery controls require either the original sandbox or recovered original files. > - Surviving attachments can support a rebuild, but they do not prove that missing edits were recovered. > - This pull request adds an explicit reconstruction decision and one fresh continuation through the normal execution gates. > - Operators can continue useful work while the original failed run remains truthful. ## Linked Issues or Issue Description **What happened?** A historical failed export can have an accepted result, a prepared workspace descriptor, and a confirmed destroyed sandbox. Its notice can also say `new_source_execution_path` and disappear. The task then has neither a truthful original-file repair path nor a supported reconstruction path. **Expected behavior** Keep the missing-file obligation visible. Let an authorized operator choose surviving task attachments and record lost work and external action outcomes. Start one fresh continuation after current execution checks. Keep the old result and failed export intact. **Steps to reproduce** 1. Seed a failed native export with a prepared descriptor and an exact destroyed-lease receipt. 2. Resolve its notice with the historical `new_source_execution_path` bookkeeping value. 3. Read the task. The baseline hides the hold even though no source reconciliation exists. 4. With this change, choose **Continue from saved sources** and submit a reconstruction decision. Related public work: #16451 preserves healthy warm sessions. #16583 prevents destructive cleanup of unexported work. This change handles historical source loss that those fixes cannot undo. Searches of open and closed work found no equivalent reconstruction endpoint. ## What Changed - Add a board-only reconstruction endpoint with `runtime:manage` and task visibility checks. - Validate the exact run, result, descriptor and destroyed lease. Reject active copyback, competing owners, newer work, stale task status, and foreign attachments. - Record attachment hashes, missing work, and external action outcomes. Preserve the original failed run, accepted result, prepared descriptor and terminal failure. - Hold other continuation paths until the durable reconstruction intent admits its successor. Record that successor in the same transaction and preserve the retry count. - Add an operator control that distinguishes reconstruction from original-file recovery. Update the recovery runbook and capability maps. ## Verification - Exact baseline source returns no blocker for the destroyed-source fixture. The same fixture returns a required recovery hold with this change. - New real PostgreSQL and assembled API/heartbeat suite: 25/25 pass. It covers duplicate submissions, lost acknowledgement, context reset, source identity, task ownership, concurrent execution attachment scope, later human blocks, deleted sources and agent pause. - Existing export retry, explicit continuation and recovery-action suites: 386 cases pass. One new-fixture constraint error in the combined run was corrected; the full new suite then passed. - UI recovery suites: 46/46 pass across three files. They cover disabled actions, loss acknowledgement, failure retention and runtime authority. - `pnpm -r typecheck` and `pnpm build` pass. The first sandboxed typecheck attempt could not open the TypeScript IPC socket; the authorized rerun passed. - Module boundaries, map checks, token gates and diff whitespace checks pass. - OpenAPI route coverage: 21/21 pass. The first CI head exposed a missing endpoint registration; this head adds the registration and the focused suite passes. - Review identified a missing dispatch-time check and an isolated attachment cache. Both are fixed. Claim and final handoff recheck the exact successor, sources and current execution gates. A controlled post-claim operator block prevents the assembled heartbeat from entering the provider. The shared attachment cache refreshes uploads and removes deleted selections. - Final focused run: 180/180 pass across six suites, including reconstruction, existing dispatch/stale-queue contracts and UI recovery. - A second review reproduced cancellation by an already satisfied historical connection card while the task was `todo`. Read-only connection availability now works before checkout; active-task mutation checks remain strict. The baseline failed two cases; after the fix all 74 connection and reconstruction tests pass, including revoked access, pending questions and pending approvals. - All current-head CI gates pass on `9d197236da11fa24f1ceadf5589cb2b81179d824`, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 for that exact head with no unresolved threads. The full local `pnpm test:run` is still running its serial server group; no full local pass is claimed. Live staging recovery is pending deployment. ## Risks - Reconstruction cannot restore missing bytes. The operator must identify surviving sources and actions that must not be repeated. The agent must verify those attachments before rebuilding. - This path is limited to exact, confirmed destroyed remote sources with an accepted result. Retained or uncertain sources keep their existing repair path. - No schema migration, credential change, budget increase or warm-session policy change is included. Normal pause, ownership, budget, approval and dispatch gates still apply. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run limitation recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
790b729086 |
fix: restore nested project repositories and settings after expiration (#16691)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use isolated Git workspaces with managed project repository copies. > - Finished workspaces can expire when Git can reproduce their contents. > - The saved recipe records the main commit but omits nested commits and local settings. > - Reopening can therefore use newer nested code and lose merged user settings. > - This pull request saves and restores those contents before workspace provisioning. > - It also permits verified FETCH_HEAD metadata so ordinary fetches do not block safe expiration. ## Linked Issues or Issue Description Refs #16457, #16586, #16621. Related performance work: #16579. **What happened?** After a finished workspace expires, its nested project checkout can reopen at the current upstream commit instead of its previous commit. Cleanup also treats local Claude settings as disposable, although that file can contain user settings. Ordinary linked-worktree FETCH_HEAD metadata prevents otherwise safe cleanup. **Expected behavior** Restore the main and nested repositories at their saved commits. Preserve local settings bytes and permissions. Skip files and Git state that cannot be reproduced. Permit standard fetched metadata only when its commits remain recoverable. **Steps to reproduce** 1. Prepare a managed nested project repository at commit A and add ignored local settings. 2. Advance upstream to B, finish the task tree, and expire its workspace and base. 3. Reopen the workspace through the normal service path. Before this fix, nested preparation selects B. ## What Changed - Extend the existing restoration recipe with exact nested commits, branches, source identity, and encrypted local settings. Keep the schema compatible with older recipes. - Restore nested repositories and settings before provisioning. Recheck company/project source access and preserve conflicting files on retry. - Recheck recorded contents under cleanup locks before deletion. Persist lock ownership before atomic publication so process exits recover safely and foreign locks stay intact. Preserve changed or unsupported contents, including tracked settings. - Keep saved project repository selections through normal preparation. Checkpoint restored contents before provisioning so failed setup can retry without overwriting later edits. - Accept bounded linked-worktree FETCH_HEAD records only after local ancestry or fresh upstream proof. Preserve local-only fetched commits. - Add real Git/Postgres lifecycle cases and focused content, access, publication-retry, and fetched-state controls. Document the behavior. ## Verification - `pnpm -r typecheck` passed. - `pnpm build` passed. - The original real Git/Postgres experiment failed with nested B instead of A. The same path passes with this change. - The integrated lifecycle suite passed 42 cases in 91.66s. Related preparation, credential, content, and Git metadata controls passed 46 cases in 24.41s. The final heartbeat checkpoint persistence case passed separately in 8.17s. - Coverage includes exact main/nested commits, encrypted settings and modes, upstream failure and retry, provisioning order, concurrent reopening, changed unique files, tracked settings, malformed metadata, and company boundaries. - Exact-head CI passed the implementation, typecheck, build, security, and test matrices. One browser shard passed on a single rerun after an unrelated fixture teardown raced an activity-log write; its feature assertions passed in the initial attempt. - The broad local test run was stopped after about 12 minutes of unrelated serial tests. Full parallel CI is required before merge; no local full-suite pass is claimed. - Single-workspace deployment, real-backlog reclamation, and ordinary product reopening remain required acceptance work after merge. ## Risks - Expired Git repositories still depend on upstream access. A failed restoration returns the existing retryable error and never substitutes the default branch. - Saved settings use the existing local encryption provider. Recovery requires that encryption key to remain available. - Unsupported files, tracked settings, unknown sources, and unverified Git state remain on disk. This can reduce reclaimed space. - No database migration, recovery-retention change, public API, or fleet-default change is included. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and an independent agent review. The runtime does not expose a more specific model identifier. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
63fbb714e6 |
fix(server): release idle tracking after aborted static UI files (#16685)
## Thinking Path > - Paperclip runs work for AI agents. > - Hosts must prove that accepted work has finished before idle sleep. > - HTTP request tracking keeps unfinished writes active after a client leaves. > - Static file middleware has no returned completion promise. > - An aborted UI download can leave its request active forever. > - This PR completes only a matched file response and keeps other work tracked. ## Linked Issues or Issue Description Refs #15522. Found while qualifying #16679. A search found no duplicate static-download idle fix. **What happened?** Closing a client during a static UI download leaves one tracked request active. The response has closed, but the file middleware never calls the wrapped response end method. The instance cannot report idle after that request. **Expected behavior** A closed, matched static file response must not leave a permanent work hold. Unfinished application writes must remain tracked. A missing file must keep the normal fallback and error paths. **Steps to reproduce** 1. Serve a large local file through the static UI middleware with idle admission enabled. 2. Start the download over HTTP. 3. Disconnect the client after its first data chunk. 4. Observe that the tracked request remains active with the original middleware. **Paperclip version or commit** Reproduced on the behavior in `d30648dee89720358271c276b9fc28cf83f18be7`. This branch starts at `cfa9dce295970a85b3d8ee14cab0eccf50bc72e2`. ## What Changed - Mark the completion of a known read-only response without dropping pending middleware promises. - Wrap static UI file serving. A successful file match is required before a closed response can complete its tracking. A late match rechecks an already closed response. Missing files keep the normal fallback. - Use the wrapper for hashed assets, other static UI files, and public development files. Preserve caller headers and cache options. - Test both HTTP abort timings, nested routers, conservative mounted-application holds, parent writes, mutating GET fallbacks, and callback-only work. - Update the idle contract and its verification map. Correct the paused-message recipe from hosted qualification: an explicit Pause cancels older queued work, so test new accepted input after the pause and await the normal resume lifecycle. ## Verification - The original two real stream-abort regressions fail with the original Express static behavior. - Three review regressions fail on `e260f4694afab8447b0cb203f439cb3ccad254e8`: a client closing before file lookup, an unfinished parent in a mounted application, and an unfinished parent in a root without installed handler tracking. Fixed sources were restored byte for byte after the comparison. - Two focused suites: 16 passed. They cover nested routers, both abort timings, hidden parent writes, late missing-file fallbacks, contents, HEAD, conditional responses, and caller headers. - Module and map checks pass. Map checks are structural only. - `pnpm -r typecheck` and `pnpm build`: passed on `f242ee739344484995d3addf81635512370778c8`. - Full CI matrix: 54 successful checks and 2 intentional Storybook skips. - Apex: 5/5 on the exact current head, with zero unresolved threads. - Full local maintained test groups are still running before final handoff. - The initial local broad run was stopped before changing source for the P1 review findings. Two file-mode fixture failures in that run came from the scratch runner's inherited `0077` mask; the same suite passes with the normal child test mask `0022`, without source or host configuration changes. The replacement run uses `0022` for test children and command-scoped `init.defaultBranch=master` for existing Git fixture assumptions. - This PR has not been deployed to a hosted environment. ## Risks Only a matched file served by the admitted application with installed handler tracking can complete this way. Routers share that scope. Mounted Express applications and roots without handler tracking keep the existing conservative response hold; this change does not claim their hidden writes are tracked. An unfinished tracked parent promise still blocks sleep after a file client disconnects. Redirects, missing files, application callbacks, and other GET requests retain their guards. No authentication, company scope, provider cleanup, or scheduling gate is removed. No schema or API response changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source editing, and shell tools. The session does not expose a more specific serving model ID or its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1f27427ffd |
perf(server): batch retained workspace reader identities and policies (#16683)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task startup checks who can read each physical workspace. > - A shared root retains the identities of earlier runs. > - The service reloads the same agents, tasks, projects and run policies for each reader. > - Database round trips grow with the retained run history and hold the company provenance lock. > - This pull request batches these reads within each guarded operation. Each run keeps its own identity and policy. > - The benefit is less task preparation time while current access checks remain in place. ## Linked Issues or Issue Description Refs #16424. **What existing behavior does this improve?** Workspace admission, repository publication and batched read authorization. **Current behavior** A synthetic shared workspace with 64 retained readers makes 680 SQL calls during a publication check. The same principals are loaded repeatedly. **Proposed behavior** Load retained identities and saved policies in batches of at most 500 IDs. Load project settings once per guarded operation. Reload records and permission grants on the next operation. **Reason and benefit** The local service benchmark with an injected 4 ms loopback round trip changed from about 7.0 seconds to 0.5 seconds and from 680 SQL calls to 47. This measures synthetic PostgreSQL data, not hosted response latency. ## What Changed - Extract retained-reader identity loading and acquisition trust checks into a focused service module. Select only the identity and trust fields used by these checks. - Batch company-scoped run, agent and task lookups. Keep each run's task binding, responsible user and policy checks. - Batch saved run-policy reads in read authorization. Keep company and agent identity checks and ordinary lookup behavior for noncanonical IDs. - Reuse project settings within one root authorization operation. - Add query budgets, identity failures, batch-boundary coverage, policy comparisons and grant-revocation tests. - Document snapshot lifetime and query scope. ## Verification - PASS: 197 focused workspace and authorization tests. - PASS: `pnpm -r typecheck`. - PASS: `pnpm build`. - PASS: all current-head hosted CI test groups, Runner checks, E2E shards, security checks and builds (55 successful checks; two unrelated Storybook jobs skipped). The local monolithic `pnpm test:run` was stopped after all equivalent hosted CI groups passed. It has no complete local result; the hosted matrix supplies the full-suite proof. - Synthetic real-PostgreSQL benchmark: 383 workspaces and 64 retained readers; 4 ms simulated loopback round trip. Publication check: 680 -> 47 SQL calls; 7.0-7.1 s -> 0.48-0.51 s. Three concurrent checks: 20.0-23.1 s -> 1.52-2.12 s. Setup and provider execution are excluded. Live reply timing remains unverified. - Regression coverage compares complete batched and ordinary authorization decisions. It checks distinct saved policies for the same principals, missing identities, company mismatch, task mismatch, responsible-user mismatch, revoked project grants and changes after the first operation. - No new schema, API or UI contract. ## Risks - A batch must not combine different runs' identities or policies. Each lookup and policy decision retains its run, company and agent checks. - Snapshots last for one guarded operation. Later admissions and publications reload data. No authorization cache survives the operation. - Restricted project trust checks keep their existing per-run validation path. Hosted latency remains an acceptance gate. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with xhigh reasoning, code execution and tool use. The runtime does not expose the configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
cfa9dce295 |
refactor(runner): remove unreferenced Paperclip Runner test fixtures (#16543)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner package keeps fake agent servers and versioned
profile identity declarations under `test/fixtures/` for its test
suites.
> - Five of those fixture files have no reference anywhere in the
repository: no test, script, generator, workflow step, or document loads
or spawns them.
> - Dead fixtures add maintenance cost and make the referenced sibling
fixtures harder to trust.
> - This pull request deletes only those five unreferenced fixture files
and adds an audit receipt with the reference evidence.
> - The benefit is a smaller test-fixture surface with zero loss of
executed coverage.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The test-suite hygiene of the Paperclip Runner package. The repository
keeps unused test-support files that no suite loads. This slows audits
and makes the referenced fixtures harder to trust.
**Subsystem affected**
`packages/paperclip-runner` test fixtures.
**Current behavior**
Five files under `packages/paperclip-runner/test/fixtures/` have zero
references anywhere in the repository:
- `fake-native-resume-codex-app-server.mjs` (277 LOC): an old fake codex
app-server for resume testing. Resume coverage now runs through
`fake-final-burst-codex-app-server.mjs` (used by
`server/src/services/native-runtime/native-session-resume.test.ts:287`)
and `fake-durable-codex-app-server.mjs` (used by
`packages/paperclip-runner/src/live/live-session.test.ts:2192`).
- `copilot-profile-v{3,4,5,6}-identity.json` (64 LOC total): orphaned
profile identity declarations. Copilot profile tests,
`native-backend-factory.test.ts`, and
`scripts/generate-acpx-profiles.mjs` reference only v7 through v19 by
literal path. No dynamic `copilot-profile-v${...}` path exists in the
repository (the only template-literal fixture path is
`test-fixtures/pi-acp/profile-v${version}-identity.json`).
**Proposed behavior**
Delete only the five unreferenced fixture files. Keep every referenced
fixture. Keep all registered test cases unchanged.
**Reason and benefit**
Smaller test-fixture surface: 341 test/support lines removed with zero
loss of executed coverage. The audit receipt
`doc/plans/2026-10-16-runner-dead-fixture-test-audit.md` records the
per-file reference evidence.
**Breaking changes**
None. No production code, no registered test case, and no test selection
path changes.
**Additional context**
A repository-wide `git grep` for each removed filename stem returns zero
external references across all tracked file types. The
`hashFiles('packages/paperclip-runner/test/fixtures/**')` occurrences in
the workflows are Rust cache-key inputs, not execution selectors;
deleting files only changes that cache hash.
## What Changed
- Deleted
`packages/paperclip-runner/test/fixtures/fake-native-resume-codex-app-server.mjs`.
- Deleted
`packages/paperclip-runner/test/fixtures/copilot-profile-v{3,4,5,6}-identity.json`.
- Added the audit receipt
`doc/plans/2026-10-16-runner-dead-fixture-test-audit.md` with per-file
reference evidence.
- 341 test/support lines removed. Zero production, documentation, or
registered test-case changes.
## Verification
- Repo-wide `git grep -i` for each removed filename stem
(`fake-native-resume`, `native-resume-codex`, `copilot-profile-v3` …
`copilot-profile-v6`) across all tracked file types returns zero
external references.
- Selection audit: `packages/paperclip-runner/vitest.config.ts` includes
only `src/**`, `devtools/**`, `examples/**`, and two named
`scripts/*.test.mjs`; the server project selects `server/src/**` and
`server/scripts/**`; the `test:typescript:prep` `node --test` list names
no fixture. No workflow executes anything under `test/fixtures/` — the
`pr-trusted.yml`/`release-verify.yml`
`hashFiles('packages/paperclip-runner/test/fixtures/**')` occurrences
are Rust cache-key inputs, not execution selectors.
- Rust tests use their own `fake-codex-app-server` binary in
`runner/crates/runner-core`, not these `.mjs` fixtures.
- Searched open and merged PRs for each filename: zero overlap (`gh
search prs`).
- Repo-wide `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were
not run this session: the removed files are unreferenced, absent from
every vitest/node-test include pattern, and excluded from all tsconfigs,
so no compiled or executed path changes. No runtime CI savings are
claimed.
- All Paperclip CI gates green on head `ce24ab3` (verified 2026-10-16
~09:00Z): every `ci /` shard (General tests, e2e shards, serialized
server suites, Verify Paperclip Runner rust/vitest/static, Build,
Typecheck + Release Registry, policy, Docker context integrity, Canary
Dry Run, gate), `review`, `Superagent Security Scan`, `Socket Security`
PR alerts + project report, and `Contributor trust` all pass; the only
skipped jobs are the Storybook deploy/visual-regression jobs, which are
not required for this change. Greptile Review: 5/5 confidence, "Low
impact", no actionable issues, last reviewed commit `ce24ab3`.
## Risks
Low risk. No executed coverage or selection path changes. The only side
effect is a one-time change to the `hashFiles` Rust cache key that lists
`test/fixtures/**`, which simply invalidates that cache on the next CI
run.
## Model Used
Zhipu `z-ai/glm-5.3` (GLM), served through OpenRouter, running as the
Paperclip Test Gardener agent. Context window size unavailable to the
agent. Capabilities used: shell tooling, git, GitHub CLI search, and
file inspection; no image or code execution capabilities were used.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
Devin FoleyandPaperclip
|
d30648dee8 |
Preserve accepted messages for paused recipients (#16679)
Keep accepted deferred requests pending while their recipient is paused and deliver through normal gates after resume. Preserve company, task hold, and cleanup checks. Verification: 247 focused checks, recursive typecheck, the full CI test and build matrix, and Apex 5/5. The local Git fixture initial-branch limitation is documented in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
391dd15ff2 |
fix: recover deferred messages and completed unblock waits (#16673)
Recover accepted deferred messages through the latest finished issue run and recognize exact completed agent unblock notifications as passive idle waits. Preserve assignment handoffs, pause and Stop, failed outcomes, and cleanup ownership. Honor local and SSH release contracts and atomically settle cleanup claims. Add PostgreSQL recovery, concurrency, and interruption regressions. Validation: recursive typecheck and build pass; 678 focused tests pass; full CI passes; Apex review is 5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
27ab902d3d |
perf(docker): start the server from a precompiled runtime bundle (#16672)
The production image started the server through the tsx loader, which resolved about 3,100 modules and transpiled about 1,200 workspace TypeScript files on every boot (7.6-20 s before listen on a live instance). A build:runtime step now bundles server/dist/index.js with esbuild, compiling in the workspace packages, and the image starts node server/dist-runtime/index.mjs. Server dependencies stay bare so runtime resolution (including the cloud image's optional peers) is unchanged; each module keeps its original import.meta location; dynamic imports stay split; the entry presents the original dist/index.js as argv[1]. The avatar worker loads through tsx's API when the main thread has no loader, tini runs as a subreaper (-s), and PR CI boots the bundle with scripts/server-boot-smoke.sh. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
938a441944 |
fix(acpx): end Claude turns whose pending tool call stops receiving updates (#16671)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted `claude_local` runs use the ACPX engine. Paperclip sends
runtime MCP servers to Claude, and Claude can start subagents (the Task
tool) that call those MCP tools.
> - #16360 set `MCP_TOOL_TIMEOUT` for Claude. That bound works: in a
local test with the same Claude binary, a direct MCP call and a subagent
MCP call to a server that never answers both ended at the bound.
> - A hosted run still hit the 4h sandbox timeout on a release that has
#16360. Its run log shows a subagent that started four MCP calls. ACP
reported the four calls as `pending`, and they never got another update.
About 40 other MCP calls in the same minutes finished in 3–25 seconds.
So the four calls most likely never reached the MCP server, and
`MCP_TOOL_TIMEOUT` did not apply to them.
> - The subagent's Task call got an `in_progress` heartbeat every 30
seconds for the full 4 hours. The turn looked active, so no existing
bound stopped it.
> - This pull request adds a stalled-tool watchdog to the ACPX engine.
It ends the turn when a pending tool call gets no update for 30 minutes
and no other tool call finishes in that time.
> - The benefit is that a stalled run fails in about 30 minutes with a
clear error code, not after 4 hours with a general timeout.
## Linked Issues or Issue Description
- Refs #16360 (MCP client deadline; this PR covers the stall that the
deadline cannot reach)
- Refs #15962 (stop-time diagnostics that showed the pending tools and
their age)
**What happened?**
A hosted `claude_local` run gave lookups to two subagents. One subagent
started four MCP tool calls. ACP reported each call as `pending`. No
call got a result, an update, or an error. The other subagent finished
in 3 minutes. The stalled subagent's Task call got a heartbeat every 30
seconds, so the last-event age at stop was only 28 seconds. The oldest
pending tool was 3h58m old. The sandbox execution timeout ended the run
after 4 hours.
**Expected behavior**
A tool call that never gets a result ends the turn after a bounded time.
The run fails with an error that names the stall.
**Steps to reproduce**
1. Start an ACP `claude_local` run that uses a subagent to call runtime
MCP tools.
2. Make one of the subagent's tool calls stay `pending` with no further
ACP update (in production this happened without a known trigger).
3. See that the run continues until the adapter execution timeout,
because the Task heartbeats keep the turn active.
**Paperclip version or commit**
`master` at
|
||
|
Devin FoleyandPaperclip
|
4415665d67 |
fix(server): allow proved idle waits and verify historical cleanup (#16633)
## Thinking Path > - Paperclip manages agents, task continuations, integrations, and their execution resources. > - Automatic idle sleep needs durable evidence that accepted work and provider cleanup are settled. > - Response-driven questions, passive board escalations, and completed connection receipts currently look like unfinished work. > - Legacy ephemeral leases and terminal image sessions need actual deletion proof before their history can become idle. > - This PR adds narrow evidence-based exceptions and read-only provider verification while preserving queues, outcomes, and ownership fences. > - Idle instances can sleep without losing responses, automatic reconciliation, or cleanup obligations. ## Linked Issues or Issue Description **What happened?** Instances with no executable work can remain awake because pending human responses, passive recovery records, outbound integration configuration, or completed delivery history are counted as background work. Older failed ephemeral leases and finished setup sessions also lack the evidence needed to distinguish retained resources from deleted allocations. **Expected behavior** Permit idle sleep only for proved response-driven waits and settled history. Continue blocking real queued work, inbound integrations without a wake contract, unfinished accounting/retries, retained resources, and uncertain provider cleanup. **Steps to reproduce** Finish an issue's producing run, leave a pending question or confirmation, and request an owned scheduled idle report. Separately retain a processed connection receipt or a failed ephemeral lease whose provider allocation has already been deleted. The old checks keep those instances awake even after executable work has finished. Related public groundwork: #15522, #15599, #15761, #15897, #15907, and #16325. Searched existing public idle-sleep PRs; this addresses additional blockers in the current predicates. ## What Changed - Recognize supported response-driven waits and passive board recoveries without changing task state, response audiences, or continuation policies. Bind passive issue generation into routine-only alarms too. Keep automatically polled merge-confirmation cards awake. - Treat processed events, delivered intents, disabled drafts, and outbound MCP configuration as history/configuration where proved safe. Preserve pending deliveries, downstream wake requests, and inbound GitHub bindings. - Permit only pristine Slack setup drafts with a matching disabled channel connection, no provisioning registration, and no prior provider identity/activity. A saved slash-command name remains inert configuration. - Repair missed routine execution-issue status hooks in bounded awake-only maintenance. Lock and re-read exact owners, apply existing completion/transient-failure rules, and preserve normal continuation after an issue resumes. - Allow human-only connection setup to wait only when no reusable active connection of its purpose exists; available access reblocks sleep for reconciliation. The delivery worker shares this proof, releases its local idle hold, and retains maintenance only while already awake. Passive-only ticks pause before admission during an owned drain, allowing slow backups to finish; new producer notifications still reconcile and invalidate the backup. - Add optional `environmentInspectLease` SDK discovery/dispatch and read-only Daytona/fake-provider implementations. Old workers remain conservative. Daytona status inspection uses a fresh provider GET without reconnecting or poisoning unused-worker eligibility. - Bind historical credential reads to an unrotated initial secret version and its immutable resolved-value hash; reject ambiguous histories and ambient Daytona credential/API-endpoint fallbacks before querying provider state. - Verify bounded rotating batches of historical resources on already-awake Cloud instances, outside readiness. Store host-owned deletion evidence bound to exact row identity/version and reject concurrent ownership changes. Preserve failure outcomes, cleanup/export intents, and run continuation authority. - Preserve cancellation provenance, defer contended routine rows individually, and cap actual inspection RPCs across caller timeouts and worker exits. Add database, SDK, provider, configuration-scope, and race regression coverage and update idle protocol documentation. ## Verification - `pnpm -r typecheck` and `pnpm build` passed on `24fd1c0f5ec`; server typecheck/build passed again after the final coordinator-only fix in `60f23e6e4b6`. - Final server regressions: 258 sleep-safety/connection cases and all 17 coordinator cases passed. Earlier focused checks also covered 331 combined idle/cleanup cases, 34 custom-image cases, 39 credential/cleanup cases, 39 connection-history cases, and 80 existing routine cases. Counts overlap and are not additive. - SDK: 153 passed. Daytona: 406 passed with 14 existing opt-in skips. The provider implementation also typechecks against the locked SDK used by the Cloud wrapper. - Dependency boundaries were exercised with real child workers holding provider inspection replies beyond caller timeouts, a real PostgreSQL row lock delaying one routine owner, and real connection creation → queue notification → worker hold release → resumption after access is enabled without another notification. A controlled six-second backup goes through the real coordinator, task drain, work signals, and idle-safety generation checks; maintenance permits completion, while a new committed transaction invalidates it. - Full local `pnpm test:run` completed its general-server phase with 21,122 passing, 29 failing, and 109 skipped tests; later phases did not run. That long run spanned source edits and is not a final-revision pass. The affected Slack draft assertion passes on current source. Restoring `/usr/sbin` for `lsof` produced 233 passing tests and two Git-fixture failures across the other five files (3 existing skips). Both remaining assertions pass with command-scoped `init.defaultBranch=master`; this host defaults to main and leaves both branches in their fixture. No source/global Git setting changed for that comparison. Original failure logs are retained. - A parallel typecheck/build attempt raced generated Runner outputs and failed an import. Running the build sequentially passed with no source change. - Local database checks use the maintained test-constructor seam with native PostgreSQL 16.14 because bundled PostgreSQL 18 is unreliable on this host. No database assertions were skipped in the focused runs. An interrupted combined run was superseded by the separate passing coordinator and safety/connection runs; no owned test database processes remain running. - Apex reviewed final head `60f23e6e4b615a08591e19d86443a6c35c091ce1`: **5/5**, zero unresolved threads. All current-head CI gates are green, with expected policy skips. The branch is conflict-free and GitHub reports a clean merge state. - Current-head CI initially failed `plan-task-fixture.spec.ts:66` during fixture deletion: the final company DELETE hit `activity_log_company_id_companies_id_fk` after the feature assertions completed. The deletion path is unchanged here; the final diff only changes drain-gated passive maintenance. The preceding revision passed this shard. Failure evidence is retained in [the original CI job](https://github.com/paperclipai/paperclip/actions/runs/39006936642/job/117321244396); the targeted retry passed on the unchanged commit ([retry job](https://github.com/paperclipai/paperclip/actions/runs/39006936642/job/117322535013)). This shows recovery, not that the underlying company-deletion race is resolved. - These checks do not claim staging/production deployment or live provider cleanup proof. Provider tests simulate responses. Full Cloud staging verification requires a compatible Core/private-plugin composition after the Core revision is on master. ## Risks Sleep eligibility changes require the existing owned drain, final generation validation, local-work checks, and plugin drain. Unsupported states fail closed. Queued agent input and routine execution remain blocked. Outbound MCP exemptions exclude inbound GitHub grants; other unsupported inbound integrations remain awake. The inspection RPC is optional and requires a matching live worker method. Its actual outstanding calls are capped at two across workers even after the caller times out. Historical lease inspection requires its recorded plugin/config; setup inspection requires an unchanged recorded config fingerprint. Not-found under replacement account credentials is not historical deletion proof: ambiguous/rotated secret histories defer verification, and initial-version reads must match their recorded value hashes. Missing provenance (including legacy image sessions without a recorded config fingerprint), live/stopped resources, provider authorization failures, and concurrent changes defer verification. Routine repair replays existing completion/transient-failure behavior and timestamps at reconciliation time; it does not execute or discard deferred comments. No migration or destructive provider operation is added. The new marker does not grant run-resume authority. Full Cloud staging validation requires a compatible Core/private-plugin composition; no production deployment is included. ## Model Used OpenAI GPT-6 Codex, with repository/tool access and code execution. Exact model identifier and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run relevant tests locally; passing focused checks and full-run limitations are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
5555817c2b |
fix(server): let Git expiration verify public commits without a run identity (#16621)
Git expiration created its auth provider with only the issue ID; with a managed GitHub connection that provider throws outside a run ("No managed GitHub identity is available for this run"), so every upstream commit proof failed and was logged as inspection_failed. Expiration now falls back to anonymous access when the provider throws: public repositories verify, private ones stay kept. Upstream verification failures are reported as upstream_commit_unverified, and a managed checkout whose upstream cannot be reached or authenticated as managed_checkout_upstream_unavailable; local proof-store failures keep their own error.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
|
||
|
DottaandPaperclip
|
669d329f25 |
perf(slack): avoid unused catalog revision scans (#16630)
Slack catalog discovery now checks live task, user, endpoint and credentials without calculating an unused compatibility revision. Full authority callers retain the unchanged revision and search-token path. Validation: 44 affected cases, full repository typecheck/build, current-head full CI/E2E and security checks, Greptile Base 5/5 with no unresolved findings. Identical production catalog output with revision statements reduced from two to zero in real-PostgreSQL comparisons. Hosted latency acceptance remains pending. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0528cc7ee0 |
perf: overlap native GitHub and assigned-tool setup (#16620)
Prepare managed GitHub access and fresh assigned-MCP discovery concurrently for remote native Codex. Wait for both before provider construction and drain both on failure through the existing run cleanup. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
5f5602bba7 |
feat(plugins): synchronize standing connection assignments (#16603)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Lifecycle plugins prepare agent runtimes outside the control plane. > - They need committed connection assignment changes without polling. > - A reader and its revision hooks must cover the same fields. > - This pull request adds a bounded standing-assignment snapshot and its matching hooks. > - Plugins can react to assignment changes while execution keeps its existing access checks. ## Linked Issues or Issue Description **What happened?** The agent runtime reader does not expose standing connection assignments. Changes to connection installs, active profiles, and connector feature switches do not request synchronization. Adding company-wide invalidation also exposes lock inversions in assignment writers and AI adoption. **Expected behavior** The owning lifecycle plugin can read a bounded assignment snapshot. Every committed change to its stored fields advances the runtime revision. Rollback removes both the change and its event. Assignment writers finish without a lock-order cycle. **Steps to reproduce** 1. Install a connection or bind an active profile to an agent. 2. Read its runtime inputs through the owning lifecycle plugin. 3. Rename or disable the connection, revoke the assignment, or switch connector features. 4. Expect a committed runtime event and an updated snapshot without periodic polling. **Additional context** Extracted from #16503. Catalog publication #16602 and the separate test maintenance PRs #16601 and #16610 are merged. This PR now targets master and is the active review. Tool-definition hooks #16604 and access/credential hooks #16605 remain drafts without review requests. ## What Changed - Add SDK types and an owning-plugin runtime snapshot for installed connections, active profile IDs, and effective connector feature switches. - Reject more than 512 connections or 128 active profiles instead of returning partial data. - Add migration 0354 for the exact stored fields used by the snapshot, with transaction-coalesced invalidation. Agent-targeted bindings and installs notify only their old/new target agents; project/routine/issue/gateway bindings are outside the snapshot and produce no notification. - Use ordered agent locks that are compatible with foreign-key references and exclude runtime readers. Prelock complete old/new target sets before replacing assignments or restoring managed profiles. - Record one company baseline at delivery startup for process-managed feature overlays. - Lock assignment inputs before install saves, task completion, and email assignment. Setup finalization reads the locked connection state and prelocks company agents before activating, re-enabling, or reviving a profile. - Make profile binding and reassignment writes transactional and lock parent profiles first. Keep target validation and profile/detail reads on the reserved transaction connection, including with a one-connection pool. - Lock company agents in order before validated AI adoption. Keep adoption and shared configuration writes at `FOR NO KEY UPDATE`, avoiding an upgrade that can deadlock foreign-key inserts during gateway creation. Configuration fields cannot change agent identity; lifecycle transitions and deletion retain their existing locks. - Document supported coverage and the inputs reserved for later PRs. ## Verification - Thirteen affected suites passed: 757 tests covering runtime hooks/assignments/skills, concurrent skill publication, tool access/catalog, task connection intents, lifecycle delivery, email, AI accounts, migration histories, snapshot drift, and the agent lifecycle module. This includes the merged catalog PR's OAuth and overlapping-refresh regressions. - The assignment suite passes all 24 cases, including bounded snapshots, company/agent isolation, rollback, health-field no-ops, process-start feature overlays, and real lock contention. - Four one-connection PostgreSQL regressions reproduce the earlier helper-read hangs before the correction; bind, unbind, delete, and reassign complete with it. The earlier adoption/gateway probe exposed both lock upgrades: changing only validation still deadlocks in shared configuration. Current scoped coverage verifies adoption with agent-targeted binding writes; gateway-only bindings no longer invalidate this snapshot. - The original ordered-adoption regression reproduces PostgreSQL `40P01` without ordered company-agent locking. Scoped-recipient tests reproduce unrelated-peer journaling before the trigger correction and verify repeated bind/unbind operations leave peers untouched, ignored binding types produce no events, both reassignment recipients are reached, and coalescing remains intact. - Six replacement contention cases cover installs, existing setup, activation, re-enabling, profile revival, and profile reassignment. Negative controls reproduce `40P01` without the corresponding target/company prelocks; all cases pass with the guards. Activation decisions use the connection row reread under lock. - An initial contention fixture failed because its settings connection lacked an MCP URL. Supplying the fixture URL and controlled lookup corrected that test setup error. - Server typecheck, database migration numbering/safety, five-snapshot retention, feature-map structural validation, and `git diff --check` passed. Migration metadata preserves historical journal entries and adds no schema change. The complete diff was reviewed for secrets and personal data before push. - Earlier hosted PR CI runs 38983440861 and 38985530069 passed; the superseded promotion run 38983440470 was cancelled. Superagent acknowledged the recipient fix and accepted the documented company-wide cost at `359b5b2f30a8a0248177367e458663d5379810d1`. Its two standard Greptile summaries were conditional on a later setup correction and were not counted as independent acceptance; the setup correction is now included here. - At `7da246aab46274db1a120a0546abbb404c5427c7`, hosted PR run 38986958988 passed: all 55 checks succeeded, with two optional Storybook checks skipped. Superagent passed, all review threads are resolved, and standard Greptile independently returned 5/5 with no blocking findings (review completed 2026-10-17 00:40 UTC). Apex independently returned 5/5 with no actionable findings (completed 2026-10-17 00:50 UTC). CI was rechecked after Apex and remains green; the branch is mergeable against master. Full local build and repository-wide tests were not run. ## Risks Migration 0354 adds database functions and triggers without changing table schema. Apply it through the normal migration flow. Agent-targeted bindings and installs only notify their affected agents. Explicit company-wide assignments, connection metadata, profile activation/removal, and connector feature switches retain synchronous, transaction-atomic company fanout. Each such transaction takes ordered row locks and inserts up to one connector runtime event per eligible company agent: O(number of eligible agents) work and journal storage, with no company-size cap or cross-transaction mutation quota. This remains an intentional scope tradeoff for this PR; deferred fanout is not implemented. Many of these conservative changes can request preparation even if an individual snapshot stays unchanged. Snapshot limits fail closed. This PR provides desired assignments, not credentials or task authorization. Later PRs must add mutation coverage before consumers can rely on immediate synchronization of tool definitions or credentials. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
190080642e |
perf(tools): publish catalog refreshes in bounded chunks (#16602)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - MCP discovery publishes tools that setup and access profiles use. > - Large catalogs currently write one row at a time. > - Repeated page entries can produce duplicate results, and profile writes can partly commit. > - This pull request publishes tools in bounded chunks and makes profile synchronization atomic. > - The result supports large catalogs before connector runtime hooks are enabled. ## Linked Issues or Issue Description **What happened?** Catalog refresh performs one database write per tool. Repeated tool names across discovery pages appear more than once in the result. Default-profile synchronization uses separate statements, so failure can leave a profile without its binding. OAuth callbacks also acquire connection/application locks in the reverse order from publication and setup, allowing a concurrent refresh to deadlock. **Expected behavior** Publish at most 500 tools per catalog statement. Keep one result per tool name, using its last descriptor. Preserve existing IDs, first-seen timestamps, disabled actions, and unchanged review decisions. Commit or roll back default-profile changes together. OAuth persistence must acquire application then connection locks before secret or connection writes. **Steps to reproduce** 1. Refresh a synthetic MCP catalog with 2,500 tools. 2. Return the same tool on two discovery pages with different descriptions. 3. Fail the profile-binding insert during default-profile creation. 4. Overlap personal, organization, or managed OAuth persistence with setup finalization or discovery of a new tool. **Additional context** This is the first feature extracted from #16503. The separate test-only maintenance PRs #16601 and #16610 are merged. This PR now targets master and is the active review. Standing assignments #16603, tool definitions #16604, and access/credential hooks #16605 remain stacked drafts without review requests. ## What Changed - Deduplicate discovery results by tool name and keep the last descriptor. - Upsert catalog rows in transactions of at most 500 tools. - Preserve row identity and operator-disabled status on conflict. Determine default-access candidates from the rows read under each publication lock, so an overlapping refresh cannot restore an action turned Off after another refresh finishes. - Lock the application before the connection during publication, setup finalization, removal, and all three OAuth persistence paths. The OAuth ordering fix is included here so this PR is independently safe; it does not depend on #16605. - Synchronize the managed default profile, its entries, bindings, and managed policies in one transaction. - Add PostgreSQL publication tests and document partial-failure recovery. - Target the existing callback failure injection at profile creation instead of a transaction ordinal. ## Verification - `pnpm exec vitest run server/src/__tests__/tool-catalog-publication.test.ts server/src/__tests__/tool-access-service.test.ts`: 436 tests passed with the OAuth and overlapping-refresh corrections on top of the merged #16610 fix. - The new tests exercise real PostgreSQL with 250 and 2,500 tools, repeated discovery pages, preserved choices, lock contention, and failures between chunks and during profile creation. - Four selected regression cases fail with the production source restored to the parent commit. The implementation was then restored and rechecked. - Six additional real-PostgreSQL regressions cover personal, organization, and managed OAuth persistence against both setup finalization and new-tool discovery. All six reproduce PostgreSQL deadlock `40P01` with the OAuth ordering fix removed; all six pass with it restored. Discovery uses synthetic credential headers to isolate publication locking from credential rotation. - Apex initially returned 3/5 on `0d6013a9649388041975406fe4b0247cd0e695aa`, identifying an overlapping-refresh permission race. Two new PostgreSQL regressions cover both absent and quarantined tools; both prove an agent goes from deny back to allow without the correction, and remains denied with it. The test pauses one refresh after its initial catalog snapshot, completes another refresh, turns the action Off through the service, and resumes the stale refresh. - `pnpm --filter @paperclipai/server typecheck`: passed. - `node scripts/check-maps.mjs`: passed (structural validation only). - `git diff --check`: passed. - Reviewed the complete diff for secrets and personal data before push. - Earlier hosted run 38975428064 failed the maximum-size newline skill-audit watchdog; other interrupted jobs reported runner shutdown/cancellation. The original watchdog failure did not identify whether startup or auditing timed out, so its exact cause remains unknown. The separate merged #16610 isolates the audit deadline from module startup and preserves the audit hang guard. This branch is rebased onto that fix. - Full local build and repository-wide tests were not run. At the previous head `0d6013a9649388041975406fe4b0247cd0e695aa`, all 55 hosted checks succeeded (two optional Storybook checks skipped), and standard Greptile returned 5/5 before Apex identified the permission race above. At corrected head `dc120d2bc879c54c14347db30e74c753bcff61e6`, all 55 hosted checks succeeded (two optional Storybook checks skipped), and a fresh standard review returned 5/5 with no open findings. The second Apex review also returned 5/5 on that exact head (successful Apex check completed 2026-10-16T23:36:44Z). Both earlier findings are addressed, there are no open review threads, and the branch is mergeable against master. ## Risks Each successful catalog chunk remains committed if a later operation fails. A retry completes publication. Provider discovery occurs outside database locks. Default-profile changes commit separately from the catalog chunks and roll back together on failure. Tests use a controlled MCP response; they do not qualify a live provider. There is no schema migration or new plugin API in this PR. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dc499ae02b |
perf(server): reject shared Git cleanup paths before history scans (#16614)
Check the existing global execution/project shared-path veto before scanning retained run contexts. Preserve the full unshared-path query, lifecycle locks and all deletion-phase rechecks. Verify with 35 focused real PostgreSQL/Git cases, full repository typecheck/build, exact-head full CI/E2E and Greptile Base 5/5. A read-only same-snapshot query pair retained the false verdict while reducing round trip from1089ms to9.4ms; this is not a full-agent latency result. The first security scan was inconclusive after an infrastructure checkout failure; one actual rescan passed. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9a3290fe10 |
perf(runner): overlap default remote artifact preparation (#16608)
Prepare the native runner and default image Codex harness together on providers that declare concurrent sync support. Preserve all executable checks, wait for both operations on failure, and retain serial explicit harness installs. Verified with 881 affected cases, full typecheck/build, exact-head full CI and Greptile Base 5/5. The real Daytona component median fell from 839ms to 432ms; full deployed reply latency remains unproven. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
73659c6b27 |
fix(server): let Git expiration discard Paperclip-managed checkouts (#16586)
Reconstructable Git expiration rejected nearly every terminal worktree with unique_ignored_files, because Paperclip writes two ignored paths into every worktree itself: .claude/settings.local.json and the managed project checkouts under .paperclip-repositories/. Expiration now accepts the settings file, and accepts a managed checkout only through a prover the expiration worker supplies. The prover maps the checkout back to its project workspace row and proves it reproducible against that row's repository URL: worktree and stock-storage checks, no linked worktrees, no unreachable objects, and every ref, reflog and HEAD commit reachable from upstream refs fetched fresh into a proof store. Every inspection Git command now runs with core.fsmonitor disabled. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3db33c131e |
perf(slack): overlap endpoint preparation and narrow credential reads (#16593)
Prepare Slack endpoints in pairs with a per-call concurrency limit of two, and resolve only the current bot token for live tool authority. Preserve candidate-order errors, active-sibling settlement, credential rotation and authorization revocation. Verified with real PostgreSQL regression controls, full repository CI/E2E, security checks and current-head Greptile Plus5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2cc7992369 |
Share the external Runner lifecycle and add strict Muse contracts (#16444)
Add the external Runner foundation with distinct Dot/Grok V6, canonical home-mode V7 and Muse V8 contracts. Keep generated schemas and profiles synchronized. Bound default question-answer command IDs in a disjoint namespace while preserving existing IDs and durable replay. Preserve canonical migration history and add normally generated, replay-safe Muse schema milestones with literal preview-history fixtures. Refs #16444. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
4de6335c3b |
perf(server): trim saved run JSON from agent authentication (#16577)
## Thinking Path > - Paperclip coordinates AI agents that work for companies. > - Agent JWT authentication checks each request against its saved run. > - Saved runs can contain large prompts and result payloads. > - The first run lookup needs only identity fields, cancellation state, and the conversation flag. > - Reading all saved JSON transfers data that this lookup does not use. > - This PR selects the required JSON fields in PostgreSQL and preserves the existing authorization decisions. ## Linked Issues or Issue Description **What existing behavior does this improve?** The initial saved-run lookup in agent JWT authentication. **Subsystem affected** Server authentication and database queries. **Current behavior** The lookup transfers the full context and result JSON on each request. It uses only the cancellation state and conversation flag from those values. **Proposed behavior** Select the required JSON fields in PostgreSQL. Keep the run, company, and agent predicates. Read current state on every request. **Reason and benefit** A synthetic lookup with a 96 KiB prompt and 4 KiB result returns 102,520 JSON bytes before this change and 134 bytes after it. This reduces data transferred by this query. **Breaking changes** None intended. Identity fields, write revocation, read access, and cancellation response codes retain their current behavior. Related: #9280 addresses authentication round-trip counts. This PR only reduces the data returned by the existing initial run lookup. ## What Changed - Extract the initial run lookup and select only the JSON fields that authentication consumes. - Add real PostgreSQL tests for legacy JSON shapes, fresh cancellation state, payload size, and run/company/agent scope. - Add a reproducible benchmark that loads the original query from an explicit baseline commit. - Document the lookup and benchmark. ## Verification - Baseline: `06689c1b3410000bcd65ff40cb469e600209c4db`. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Final server typecheck: passed. - Final-head GitHub checks: all green at `2be1c38e6bf51616b69993cf84cee1eef9657517`. Apex: 5/5, zero unresolved threads, conflict-free. - `pnpm check:module-boundaries`: passed. The standalone synthetic benchmark now uses the existing root benchmark directory and resolves Drizzle through the server workspace. - Focused HTTP middleware and database tests: 29 passed on real Homebrew PostgreSQL 16.14. The SQL test compares 270 combinations of status, cancellation JSON, and conversation JSON. It also checks fresh state and company/agent/run scope. - Broad native verification: 873 passed across the HTTP middleware, SQL projection, run diagnostics, and native executor suites. - The default local full command completed its general server lane with 12,090 passed and 8,885 skipped tests. It could not finish the root workflow: the host could not start the bundled PostgreSQL; a diagnostics assertion matched a directory name; and Git fixtures retained the host default branch. These failures remain in the verification record. All 68 failed files now pass on replay with disposable Homebrew PostgreSQL, an isolated checkout, and command-scoped Git defaults. The 68-file replay passed 2,588 tests and left two email skill-resolution failures. Moving the checkout outside an unrelated ancestor skills directory resolved those last failures: all 44 tests in that file passed. Product code and test assertions were unchanged for these host adjustments. Benchmark command: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-agent-authentication.ts --baseline-ref=06689c1b3410000bcd65ff40cb469e600209c4db ``` The final benchmark ran at treatment commit `2be1c38e6bf51616b69993cf84cee1eef9657517` with a clean worktree. The local run used PostgreSQL 16.14 and the maintained test constructor seam to start disposable Homebrew clusters. The bundled PostgreSQL binary could not start on this host. Both versions use the same schema and predicates. Each case has two warmup rounds and ten measured rounds with alternating order. Random content prevents unrealistic compression. Complete authentication decisions match. | Context / result | Before JSON bytes | After JSON bytes | Before median | After median | | --- | ---: | ---: | ---: | ---: | | 1 KiB / 4 KiB | 5,240 | 134 | 0.157 ms | 0.157 ms | | 96 KiB / 4 KiB | 102,520 | 134 | 0.331 ms | 0.140 ms | These are local query round trips, including result transfer and decoding. The small control has no material latency gain. The large case improves by about 2.4x. Both plans read the same single row. The primary result is reduced response size. This does not measure HTTP latency, server CPU savings, or fleet-wide cost savings. Full samples and plans are retained with the verification evidence. ## Risks - JSON types must retain their meaning. The real database tests compare the consumed authorization contract with the original projection, including SQL null, JSON null, scalars, arrays, objects, string booleans, and cancellation states. - The change covers the first JWT run lookup. Later run identity capture keeps its existing full-run reads and transaction locks. Those reads remain a separate cost. - The change adds no schema migration or authorization cache. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, and shell tools. This session did not expose a more specific serving model ID or its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c91f7a6e5f |
perf(runner): overlap verified sandbox launch assets (#16585)
Native sandbox launch preparation stages fresh harness files and immutable runtime context in sequence. Overlap them only under the verified concurrent-sync capability, settle every started transfer before launch or cleanup, and preserve serial SSH behavior and error priority. A matched real-Daytona comparison reduces staging from 1,552 ms to 1,017 ms. The larger active-run difference includes model variation and is not attributed to this change. Add an explicit single-attempt diagnostic and synchronize stale Pi evidence metadata with its shipped profile. Validation: 764 native runner/tracing cases, 2,764 support cases, repository typecheck/build, fresh exact-head full CI/E2E/security and Apex 5/5 pass. Observability remains a silent no-op when no endpoint is configured. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f5534717b0 |
fix(runtime): preserve unexported native work during lease cleanup (#16583)
Preserve the exact native workspace source until its export is finalized. Apply the same guard to cleanup sweeps and direct environment teardown. Keep stop-only recovery and healthy warm retention available. Verified with 317 targeted tests, server typecheck, module boundary check, all current-head CI gates, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ce0eff7d1f |
perf: reduce fresh workspace clone and sandbox staging waits (#16487)
## Thinking Path > - Paperclip manages AI agents and their work. > - Each run needs a private workspace and safe persistence. > - Fresh isolated repository sources download upstream Git history again. > - Sandbox staging also verifies an optional seed cache before it uploads the workspace. > - That verification holds a company lock during a full extraction and hash scan. > - This pull request reuses pristine Git objects through an authenticated remote clone. > - It also overlaps seed verification with upload and moves verification outside the publication lock. > - The benefit is less repeated transfer and less serial preparation work, with the same task isolation and recovery contract. ## Linked Issues or Issue Description **What existing behavior does this improve?** Fresh task-private repository preparation and inbound sandbox workspace staging. **Current behavior** Each isolated remote source clones the full upstream history. Optional seed verification then extracts and hashes another full tree before upload. One workspace's verification can hold the company cache lock while other workspaces wait. **Proposed behavior** Use a company and credential scoped cache of pristine upstream objects for GitHub HTTPS sources. Each task still clones from the current remote. It disconnects from the cache before publication. Run seed verification alongside upload. Hold the company lock only for final capacity checks and publication. **Reason and benefit** These operations lie on the workspace preparation path. Removing repeated Git pack transfer and serial verification can reduce startup time. Local measurements below cover these boundaries. They do not establish hosted agent latency. Related: #16425, #16454, #16398, #16461, #16468, #16473. The cache inspection and durable folder proposal in #13376 covers separate product work. ## What Changed - Add a pristine upstream Git object cache for fresh isolated GitHub HTTPS sources. Key it by company, repository and current credential fingerprint. Do not store credentials. - Keep online clone authorization and current remote refs. Use `--reference-if-able --dissociate` so task objects remain independent. A reference failure retries once online without the cache and resolves credentials again. - Cap cache storage at 64 entries and 4 GiB per company, with 2 GiB per entry. Skip contended publication. Bound priming by a cooperative five-second budget and a 4,096-entry scan allowance. Keep the existing free-space reserve. - Overlap optional seed verification and required workspace upload. Settle both before returning or deleting temporary files. Recheck quotas and company directory identity under the publication lock. - Keep the independent fresh upstream reproducibility proof, task commits, dirty source copies, ignored files, nested repositories and durable recovery behavior. - Suppress optional Git server-info generation during dissociation so generated storage passes the existing safe-expiration checks. This uses [the supported Git setting](https://git-scm.com/docs/git-config/2.43.0#Documentation/git-config.txt-repackupdateServerInfo). - Add bounded timing phases, regression coverage and two repeatable local benchmarks. No first-party Telemetry or dependency changes. ## Verification - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Affected adapter suites: 240 tests passed. Native workspace persistence/history and tracing: 33 tests passed. Server checkout/isolation/tracing batch: 53 tests passed. Final object-cache authority, path and capacity suite: 12 tests passed. These batches overlap. - Cache and expiration/restoration lifecycle batch: 34 tests passed. A direct generated-storage check covers cached clones. - Real HTTP Git fixture: a matching cache does not bypass revoked remote access. Other fixtures cover current refs/tags, hidden cached commits, independent objects after cache removal, corruption fallback, credential/company scope, quota limits and publication contention. - Initial restricted runs hit SQLite write and Unix socket permission failures. The adapter batch passed with disposable writable instance state and socket access. The first typecheck rejected the test use of `Promise.withResolvers`; typed deferred gates now use the supported library contract. - Repository benchmark on final head `9047368bf3b7fa2061e00d698b8b51cf4ae1da32`: five alternating samples, real local `file://` Git transport, synthetic 32 MiB history, no injected latency. Median 1143.8 ms to 240.5 ms; ranges 1085.7–1486.6 ms and 188.0–254.8 ms. Received Git pack bytes fell from 33,569,788 to zero. Initial cache priming took 1208.4 ms and is reported separately. - Sandbox benchmark, five alternating samples per mode on 10,000 files with real packing, extraction and verification: with a controlled 3-second transfer delay, median 9.450 s to 7.620 s (ranges 9.280–12.115 s and 7.296–7.983 s). With no injected delay, median 6.808 s to 6.450 s (ranges 6.255–7.029 s and 5.709–6.902 s). The no-delay ranges overlap; local copying competes with verification for disk. Both modes verify final tree contents, ignored-file exclusion and Git HEAD. These are synthetic local fixtures, not hosted agent runs. - Reproduce with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-repository-object-cache.ts` and `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-staging.ts --baseline-ref=307eb16c0c --transfer-delay-ms=3000`. - Prior head `af766bc820` received Greptile 5/5 with no findings. Its E2E shard 7 failed in `plan-task-fixture` cleanup: company deletion raced an `activity_log` foreign key. That fixture has no provider run or Git workspace. The failure is retained here; final-head CI must pass. The corrected corruption fixture now forces a pack and verifies a real damaged archive. Current head `9047368bf3b7fa2061e00d698b8b51cf4ae1da32` has Greptile 5/5 with no review threads or findings. Final head has 55 passing checks and two optional Storybook skips. [The first workspace-shard attempt](https://github.com/paperclipai/paperclip/actions/runs/38802441857/job/116655485350) passed all 1,778 cases, then reported a late `EPIPE` from unchanged `server-utils.ts` while the ACP reuse test was active. That case mocks workspace preparation and passed locally in isolation. [Its single retry on the same commit](https://github.com/paperclipai/paperclip/actions/runs/38802441857/job/116659728034) passed, as did the dependent verification gate. The earlier E2E cleanup fixture passed on this final head. Both initial failures remain recorded here. A full local suite was not duplicated; the affected boundaries were run locally and the complete suite runs in CI. ## Risks - First cache population adds local copying. Cold clones, credential rotation, capacity exhaustion and a corrupt cache retain the online clone path. No eviction is added. Cached data is bounded, but many companies can each consume their allowance. - Individual filesystem calls and cleanup can exceed the cooperative priming budget. Failed reference clones make one additional online attempt. - Parallel verification and upload compete for disk and CPU. The local no-delay benchmark shows a smaller gain than a slow-transfer case. Verification still checks contents before cache publication, and required upload failures remain failures. - Local fixtures do not prove hosted throughput or the 15–30-second end-to-end target. The fresh checkout still writes its working tree, and model startup, copyback and other preparation costs remain. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed in this session. Used reasoning, source editing, shell tools and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal task identifiers - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
06689c1b34 |
perf(skills): coalesce inventory before run cache locking (#16576)
Share each active company skill-inventory refresh before taking the version-cache lock. Read actor visibility under the lock and keep snapshot publication protected through durable run pinning. Preserve empty-selection behavior and the list API. Validation: 176 affected cases, full repository typecheck/build, fresh full CI/E2E, security and exact-head Apex 5/5 pass. The local six-reader latency experiment improves from 50.4s to 18.3s; live response latency is a separate acceptance check. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
5fbece9dcc |
Synchronize assigned skills through agent runtime hooks (#16501)
## Thinking Path > - Paperclip manages shared skills and agent assignments. > - Runtime plugins need installed skills before a task starts. > - Skill content can change without changing the agent row. > - Source access and version pins also affect delivered files. > - This pull request feeds those changes into the durable runtime stream. > - The runtime reader resolves the same immutable skill sources used for execution. ## Linked Issues or Issue Description Builds on merged PR #16499. This pull request now targets `master` and contains only the five skill synchronization commits. **Problem** Plugins receive no durable notification when an assigned skill changes. A saved assignment can also become inaccessible after private-source access changes. **Proposed solution** Invalidate referencing agents at installed skill mutations. Invalidate inherited access when source bindings, project privacy, or policy change. Return authorized skill sources and version IDs through the owning runtime reader. **Alternatives considered** Activity-only subscriptions miss service writers. Per-task synchronization adds preparation work. Copying current files would ignore version pins. **Roadmap alignment** This extends existing skills, source synchronization, and plugin runtime support. ## What Changed - Defer multi-skill source runtime hooks until commit, after publication holds all skill rows. Keep file-save journaling immediate to preserve filesystem rollback. - Add transactional invalidations for skill content, companion files, versions, rename, deletion, policy, source access, and pin-setting changes. - Recognize canonical keys and supported legacy aliases. - Reuse the execution skill resolver, current agent access, and instance version-pin rules. - Resolve and materialize sources outside the agent row lock. Recheck ownership and runtime revision afterward. - Keep task-only mentions and running-task snapshots in the execution path. - Document conservative invalidations for pinned assignments and inherited access. ## Verification - Rebased onto origin/master `9f4a956b90`, which contains merged PR #16499. All five commits are unchanged according to `git range-diff`; no conflict resolutions or source changes were needed. - On rebased head `58b065941`, all 237 focused tests passed: runtime skill hooks/source publication (41), plus content service, assignment routes, policy routes, and project source access (196). - Server typecheck passed after rebuilding Runner with the installed Rust toolchain. Migration numbering/safety and map checks passed; snapshot pruning made no changes. Migration CLI checks required local IPC permission. - Real PostgreSQL coverage includes concurrent source publication/version saves, policy-only tightening/restoration, referencing-agent fanout, company isolation, no-op counters, supporting files, rename/deletion, rollback, private-source access, pinned/latest bytes, and unassignment. - Earlier stack validation passed workspace typecheck/build. Fresh hosted CI and review must verify the rebased head before handoff. - The broad local suite was previously limited by an unchanged diagnostic test that rejects "private" in an error stack under this /private/tmp worktree; that unchanged suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. - Diff whitespace and full PR secret/PII scan passed. - All hosted CI/security checks are green on `58b065941`; fresh standard Greptile and Apex both returned 5/5 with no findings or unresolved threads. The PR targets master and is mergeable without conflicts. ## Risks - Apply migration 0348 after the runtime-stream migration in #16499. - Content invalidations include pinned assignments. The resolved version stays pinned; consumers can avoid writes when its version/hash is unchanged. - Project access and source changes invalidate only agents assigned skills from the affected sources. Company policy still invalidates the company. Source polling bookkeeping and popularity counters do not create work. - The reader rebuilds installed caches without upstream requests. Concurrent changes cause a bounded retry and then a conflict. - Private transport integration and live remote computer qualification remain separate. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
9f4a956b90 |
Add durable agent runtime synchronization hooks (#16499)
## Thinking Path > - Paperclip manages agents and their runtime configuration. > - Plugins can maintain external agent computers. > - Configuration and file saves currently miss durable plugin delivery. > - A transient activity notification cannot recover after interruption. > - This pull request adds an opt-in agent runtime resource stream. > - The existing dispatcher delivers and acknowledges its committed work. > - Plugins can prepare computers when configuration changes. ## Linked Issues or Issue Description Related: #15631 and #16203 establish agent lifecycle and native durable delivery. **Problem** Plugins cannot reliably react to agent configuration and instruction changes without polling or waiting for execution. **Proposed solution** Add an opt-in `agent_runtime` stream to the existing resource journal. Record changes at the database mutation and managed file audit boundaries. Wake the dispatcher with content-free commit notifications. Provide scoped runtime inputs to the selected lifecycle plugin. **Alternatives considered** Route-only activity events miss service and background writers. Scheduled scans delay changes and repeatedly query unchanged state. **Roadmap alignment** This extends the existing plugin and agent configuration work. ## What Changed - Add durable update/delete events for agent runtime state, including an existing-agent baseline. - Capture configuration changes, instruction edits, deletion, restore, and managed run writeback. - Add commit notifications and a reconnect scan. Register one listener for configured, enabled runtime subscribers. Release it when none remain. - Keep HTTP startup independent of queued plugin callbacks. - Add `ctx.agents.getRuntime`. Require lifecycle capability, invocation company, enabled plugin, and owning lifecycle selection. - Keep legacy agent/project subscriptions isolated. Manual inbox reads retain both legacy streams. - Resolve recovered instruction paths without reading or hashing bundle contents. Use the company reporting graph for runtime health. - Repair missed file events on identical save/deletion retries after a journal failure; preserve content tokens and conditional-write checks. - Correlate file audits with runtime revisions to deduplicate recorded retries without a new table. Bound the lookup with the existing company/timestamp index. - Document the contract, migration, access boundaries, and session-connection requirement. ## Verification - Rebased the full stack onto origin/master after #16507. - Final replayed stack: `pnpm -r typecheck` and `pnpm build` passed. - All 132 affected runtime, skill, connector, publication, instruction, managed-agent, and delivery tests passed after the latest-master rebase. Workspace typecheck and build passed. - Journal failure injection verifies recovery on identical file writes, deletions, and instruction retries from an acknowledged inbox. - Deduplication follow-up: 80 runtime/instruction/delivery tests and server typecheck passed. Repeated retries create no audit/events; old hashes returning after an intervening commit still repair delivery. Built-in/native instruction writer regressions passed all 56 tests. - PR 1 follow-up tests cover valid/cyclic reporting chains, listener registration failure, multiple subscribers, global/company disable, re-enable, unload, and startup with blocked callbacks. - Database coverage includes rollback, journal failure, no-op suppression, file saves/deletion, ordering/replay, owner/company denial, independent-client commits, and real listener loss/reconnect. - SDK/shared suites passed 1,059 tests during implementation. CLI/delivery regressions also passed. - Migration safety, map checks, diff whitespace, and local secrets/PII scan passed. - Hosted CI and security checks are fully green at `b296405e2`. Standard Greptile and Apex both returned 5/5 with no unresolved findings. - The broad local suite encountered an unchanged diagnostic test that rejects the word "private" from an error stack. This worktree lives under /private/tmp. The same unchanged diagnostic suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. ## Risks - Apply migration 0347 before enabling a runtime subscriber. Runtime notifications require session-level PostgreSQL LISTEN. Transaction-mode poolers are unsupported for this opt-in feature. - Database triggers share the mutation transaction. Journal failure prevents configuration commits. Trigger work adds a journal insert to relevant changes. - Filesystem publication and SQL commit are separate boundaries. Direct filesystem edits do not emit hooks. External clients must fence stale revisions and retain conditional-write checks. - The reader returns only that plugin's own instance secrets. Keep them out of UI data, logs, plugin state, and agent files. - No private transport integration or live remote computer qualification is included. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b294abb9eb |
perf(server): append ordinary run events in one statement (#16569)
Ordinary run events now validate their binding, allocate a sequence and insert the decoded receipt in one atomic PostgreSQL statement. Native replay and retry-exhaustion transactions keep their existing behavior. Verification: exact-head full CI/E2E, Superagent, legacy security and Apex 5/5 passed. PostgreSQL concurrency, rollback and payload controls pass. Controlled network-delay comparison reduced median event persistence from about192ms to48ms; no staging response-time claim is made before deployment. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8556539364 |
perf(adapter-utils): batch remote GitHub launcher staging (#16552)
Stage the nine managed GitHub launcher files through one bounded control command. Preserve per-file locks, content verification, atomic replacement and control-channel dispatch. Real Daytona staging decreased from about one second to 0.24–0.34 seconds. Full CI, E2E, Superagent and Apex 5/5 passed on the reviewed head. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
8f1cb11a03 |
fix(workspaces): name the owner action when a shared source is restricted (#16545)
A reused task workspace in a shared project checkout keeps its source denial when another task's restricted worktree shares that checkout (#16095 deliberately does not clone over existing workspaces). The run then failed as an opaque `setup_failed` 403 and the task was blocked with a generic "no live execution path" notice. Convert that exact denial into a `configuration_incomplete` failure with reason `workspace_source_restricted`, give the task notice and the recovery action the owner step, and keep this pre-dispatch owner action out of the unexpected Sentry run-failure stream. The reason and predicate live in a dependency-free module. Access checks are unchanged. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
f1d4bc1617 |
perf(slack): resolve only signing secret during webhook checks (#16548)
Resolve only the current Slack signing secret during workspace checks and authenticated setup evidence. Preserve full runtime credential resolution, company bounds, rotation, and auditing. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
78b833c495 |
Dispatch routine and monitor work at saved deadlines (#16514)
## Thinking Path > - Paperclip manages work that agents run for a company. > - Routines and issue monitors store their next run time in the database. > - The app currently waits for the heartbeat interval to find that work. > - This can delay work after a workspace wakes before its saved deadline. > - This pull request uses the existing work coordinator to schedule each known deadline. > - It also parses each cron expression once, so sparse schedules do less work on the event loop. > - Agents can start due work sooner while the existing execution checks stay in place. ## Linked Issues or Issue Description **What happened?** Routine and issue-monitor dispatch depends on the heartbeat scan interval. A workspace can wake early and still wait for the next scan. A sparse cron search also parses the same expression for every minute it checks. In a local annual-schedule test, the deadline callback ran on time but schedule calculation delayed the routine record by 13.6 seconds. **Expected behavior** Use a saved future deadline to request the existing dispatch scan. Restore deadlines at startup and update them after a producer transaction settles. Do not let future timers keep an idle workspace active. Parse a cron expression once for each search. **Steps to reproduce** 1. Set an active routine and an assigned issue monitor to the same future deadline. 2. Choose a deadline between heartbeat scans. Use an annual routine expression to exercise a sparse search. 3. Compare the saved deadline with the routine firing and agent start times. **Paperclip version or commit** Source checkout based on |
||
|
DottaandPaperclip
|
0e58f29eb7 |
fix(observability): trace pre-dispatch controller waits (#16515)
Measure usage startup, MCP handoff, dispatch admission, and budget reservation with the existing endpoint-gated tracer. Preserve outer attempt context across asynchronous provider dispatch.
Validated with 135 focused tests, full typecheck/build, fresh full CI/E2E, Superagent, and Apex 5/5 on
|
||
|
Devin FoleyandPaperclip
|
c7ea5c0216 |
perf(server): skip settled committed finalizations in unscoped passes (#16485)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On restart, `/api/health` reports `starting` until startup recovery finishes, and deploys wait for it > - Native recovery and the orphan reap each run an unscoped native finalization reconciliation pass, and readiness waits for both > - The pass selects every committed finalization with a result, including runs that ended long ago, and spends dozens of sequential queries on each one > - On a busy instance this is about one minute per pass, so about two minutes of every restart, and the periodic recovery repeats it on every tick > - This pull request skips settled committed finalizations in unscoped passes, and keeps a small rotating upkeep page so their upkeep still runs > - The benefit is faster restarts and less steady database load, with no change for unsettled work ## Linked Issues or Issue Description Related: #16330 and #16329 (startup and reconciliation timing). **What existing behavior does this improve?** Startup recovery time and steady-state recovery load. **Current behavior** - The unscoped candidate scan in `reconcileNativeFinalizations` returns every native finalization with a result and phase other than `terminal_failure`. Nothing excludes rows that are already settled. - Each committed row then takes the full per-row path: pending-effect, assessment, evidence and decision reads, presentation repair (twice), the workspace barrier read, a `finalizeNativeRun` re-read of the committed coordinator, and workspace settlement (lease release check and agent status update). That's about 40–60 sequential round trips per row. - Startup runs the unscoped pass twice (native recovery, then the orphan reap), and readiness waits for both. The periodic orphan reap runs it again on every tick. - On one instance, all 268 candidates were committed finalizations whose runs had ended. Native recovery took 50–80 s and the orphan reap 47–80 s on every restart, with database round trips of about 3.4 ms. **Proposed behavior** An unscoped pass skips a committed finalization when: - its run is `succeeded`, `failed`, `cancelled` or `timed_out`, and - the run finished more than an hour ago, and - its decision has no `pending` or `failed` status-decision effect. Each unscoped pass also includes up to 10 such settled rows in a rotating order, so the presentation and evidence upkeep still reaches every row over time. Scoped passes (explicit run IDs) are unchanged. **Reason and benefit** These settled rows have nothing left to finish. Recent rows, unfinished runs and rows with undelivered effects stay in every pass, so crash-window repairs and effect delivery behave as before. Startup and periodic passes no longer scale with the full history of committed runs. **Breaking changes** None. A late repair on a settled row can be delayed until its turn in the upkeep rotation instead of the next pass. ## What Changed - `server/src/services/native-runtime/native-finalization-reconciler.ts`: - `settledCommittedFinalization(now)`: the settled predicate above, including a `NOT EXISTS` on undelivered `status_decision_effects` for the same company, issue and decision. - The unscoped candidate scan adds `NOT settled OR run in the upkeep page`. - `nextSettledCommittedUpkeep`: up to `NATIVE_SETTLED_COMMITTED_UPKEEP_PER_PASS` (10) settled run IDs after an in-process cursor, ordered by run ID, wrapping at the end. - New constants `NATIVE_SETTLED_COMMITTED_GRACE_MS` (1 hour) and `NATIVE_SETTLED_COMMITTED_UPKEEP_PER_PASS`, plus `resetSettledCommittedUpkeepForTests`. - New `server/src/__tests__/native-finalization-settled-candidates.test.ts`. - `doc/execution-semantics.md`: describes the settled rule and the upkeep rotation. ## Verification - `pnpm exec vitest run src/__tests__/native-finalization-settled-candidates.test.ts` (in `server/`): 4 passed. The test mocks the timing hook to capture the candidate count and stop the pass after the scan: - 25 settled rows give 10, 10, 5, then 10 again over four passes (rotation and wrap). - A row finished 10 minutes ago, a running row, and a terminal row without `finished_at` are all candidates. - A settled row whose decision has a `failed` effect is a candidate. - A scoped pass with an explicit run ID still selects a settled row. - `pnpm exec vitest run` for `native-finalization-recovery`, `legacy-finalization-regression`, `native-run-finalizer-telemetry`, `native-workspace-finalizer`, `native-finalization-reconciler-timing`, `external-chat-wait.integration` and `native-status-arbiter-corpus`: 316 passed. - `heartbeat/recovery.test.ts`: the 8 tests that do not need a database passed. The 20 database tests were skipped locally because embedded PostgreSQL could not start (the host's shared-memory segments were exhausted by other processes). CI runs them. - `pnpm typecheck` in `server/`: passed. ## Risks - A settled row that needs a late repair (for example a missing chat presentation) waits for its turn in the upkeep rotation, up to `ceil(settled rows / 10)` passes, instead of the next pass. - The upkeep cursor is in memory. After a restart the rotation starts from the beginning; the work is idempotent. - A run left `running` with a committed finalization is never considered settled, so it is still projected before the reaper's active-run loop. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs, database and log analysis). A separate Claude agent mapped the recovery call tree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Huanyi Xie
|
c4a16cbe98 |
fix(plugins): follow local package path changes in dev watcher (#15247)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local plugin development uses file changes to restart the plugin worker. > - A plugin can move to a different local package directory. > - The watcher rejected every later registration for the same plugin ID. > - This pull request replaces the watcher when the resolved package path changes. > - The benefit is that current plugin files trigger reloads and previous files stop doing so. ## Linked Issues or Issue Description Refs: #14955. This change addresses its second observation: the watcher keeps the previous local directory after its package path changes. The issue also tracks stored configuration replay as a separate problem. Related PRs: #11336 shares the lifecycle manager across routes and the dispatcher. #14381 controls when production file watching is enabled. #10100 orders configuration delivery. Those changes address other parts of plugin reload behavior. ## What Changed - Store the resolved package path with each watcher. Keep repeated registrations of the same path as a no-op. Replace the watcher for a different path. - Cancel pending changes from the previous directory. Ignore late change, error, and timer callbacks from a replaced watcher. - Apply only the latest pending directory lookup. Manual registration, unwatching, and shutdown invalidate earlier lookups. - Stop a failed watcher without cancelling its pending replacement lookup. Explicit unwatching, disable, unload, and shutdown still cancel pending registrations. - Stop the old watcher when the new directory is unavailable or the plugin no longer has a local package path. - Document the path-change behavior and add twelve regression cases. ## Verification - `pnpm exec vitest run server/src/__tests__/plugin-dev-watcher.test.ts server/src/__tests__/plugin-lifecycle-restart.test.ts`: 18 tests passed. Regressions cover path changes, stale callbacks, lookup order, explicit cancellation, and watcher errors during a pending replacement. The current-watcher error case fails before the stop-versus-cancel fix; the stale-watcher error control passes. - A local smoke check used real Chokidar watchers and two real temporary plugin directories. It confirmed that a pending old change is cancelled, later old-directory changes are ignored, and a new-directory change triggers one restart. - `pnpm -r typecheck` passed on the path and lookup guards. The final watcher-error fix passed `pnpm --filter @paperclipai/server exec tsc --noEmit`. - `pnpm build` passed on the initial path-change fix. The final source passed `pnpm --filter @paperclipai/server build`. - `git diff --check`: passed. - Greptile reviewed `bef3182` at 5/5 with no new findings. The watcher-error review thread is resolved. CI is still running. - The full `pnpm test:run` suite was not repeated for this watcher-only change. The affected suites ran locally. The broad test shards will run in CI. ## Risks - Low risk. The watcher follows the current registered path. An unavailable replacement stops watching until another valid registration arrives. - Closing a filesystem watcher is asynchronous. Callback identity checks prevent an old watcher from acting on its replacement. - No database migration, API shape, or dependency change is required. ## Model Used - OpenAI Codex based on GPT-6, with reasoning, tool use, and code execution. The runtime does not expose the exact serving model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
becdc3b08e |
perf(slack): remove delays before mention acknowledgements (#16491)
Slack mentions now start eyes reactions after the authorized task/comment commits, independently of attachments, subscription and scheduler acceptance. Omit unused optional URL-preview polling while retaining message text, links and files. Keep receipt failures and missing-file warnings independent, including terminal cleanup of older multi-line diagnostics. Verified with full current-head CI and E2E, Superagent and legacy security checks, Apex 5/5 with no unresolved findings, 19 selected real-PostgreSQL receipt/authorization/cleanup cases, and full frozen-source typecheck/build. The first CI attempt's unchanged Cursor probe timeout is retained in the PR; the one failed-only retry passed under unchanged deadlines. Live Slack latency still requires deployed acceptance. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
81cea89d6b |
fix(cli): require confirmation for instructions file deletes (#11504)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The CLI exposes board/operator controls for managing agent configuration and instructions. > - Agent instructions files influence future agent behavior, so deleting one is a destructive configuration mutation. > - Nearby destructive CLI commands already require explicit `--yes` confirmation before sending DELETE requests. > - `agent instructions-file:delete` skipped that local confirmation guard and could delete a bundle file from a scripted or mistyped command. > - This pull request aligns that command with the existing destructive-command safety convention. > - The benefit is a small, predictable operator-safety guard with focused regression coverage. ## Linked Issues or Issue Description Bug report: ### Pre-submission checklist - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip (or can reproduce on `master`). - [x] I have confirmed the error originates in Paperclip itself — not in my agent adapter, API provider, or local configuration. ### What happened? `paperclipai agent instructions-file:delete` deleted an agent instructions bundle file without requiring explicit CLI confirmation. ### Expected behavior Destructive CLI delete operations should require `--yes` before sending a DELETE request. ### Steps to reproduce 1. Run `npx paperclipai agent instructions-file:delete <agent-id> --path AGENTS.md`. 2. Observe that the CLI immediately issues `DELETE /api/agents/:agentId/instructions-bundle/file?path=AGENTS.md`. 3. No local confirmation guard blocks the destructive request. ### Paperclip version or commit `fd472d02ba950519ca5eff75ed9fd55cf976f4c2` ### Deployment mode Local dev (`pnpm dev`) ### Installation method Built from source (`pnpm dev` / `pnpm build`) ### Agent adapter(s) involved - [x] Not adapter-specific (core bug) ### Database mode Not database-related ### Access context Board (human operator) ### Node.js version `v20.x` ### Operating system macOS ### Relevant logs or output ```text Current behavior is local CLI behavior: the command proceeds to the DELETE call without requiring `--yes`. ``` ### Relevant config (if applicable) None. ### Additional context This command already sits next to other destructive CLI actions such as `agent delete`, which require explicit confirmation. Aligning the guard makes the CLI safety model consistent. ### Privacy checklist - [x] I have reviewed all pasted output for PII (usernames, file paths, API keys, tokens, company names) and redacted where necessary. ## What Changed - Added a `--yes` confirmation option to `agent instructions-file:delete`. - Refuse the command locally before resolving API context or sending a DELETE request when confirmation is missing. - Updated CLI parity coverage for confirmed and refused instructions-file deletion. - Updated the CLI documentation example to include `--yes`. ## Verification - `/Users/xinran/Downloads/dev/paperclip-pr-work/node_modules/.bin/vitest run cli/src/__tests__/agent-lifecycle.test.ts --config cli/vitest.config.ts` equivalent executed from the PR worktree using temporary dependency symlinks: 3 tests passed. - `git diff --check` ## Risks - Low risk: this only changes one destructive CLI command and matches existing `agent delete`, `issue delete`, and other delete-command confirmation behavior. - Scripts that intentionally delete instructions files must add `--yes`, which is the intended explicit confirmation path. - No database, API contract, or lockfile changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent in Codex desktop with shell/tool use and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: 馨冉 <xinxincui239@gmail.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
67a9d5cc37 |
perf(runner): bypass skill cache waits for empty selections (#16498)
Native runs with no skill files no longer queue behind company skill-cache work. Preserve saved empty selections and the existing exclusion for selected and legacy operational skills. Verified with real PostgreSQL held-lock regressions, 20 distinct affected cases, full repository typecheck/build, fresh full CI/E2E/security and exact-head Apex 5/5. Hosted latency remains to be verified. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
ec3eb1968b |
feat(runner): expose credential updates through PRP (#16463)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Runner uses PRP to send harness events to the host. > - A harness can replace its stored OAuth tokens during a run. > - The host has no PRP signal that tells it to read the updated credential. > - This pull request adds a durable credential update event and observers for supported stores. > - The host can react before it acknowledges the event. Token values stay outside PRP. ## Linked Issues or Issue Description **Subsystem affected** `packages/paperclip-runner`: native provider execution, PRP schemas, and host event delivery. **Problem or motivation** A Paperclip host that consumes PRP needs to react when a harness changes its refresh-token credential. Existing credential copy-back paths do not provide a common event during a warm session. **Proposed solution** Add `auth.credentials.updated` as a P0 runner event. Include only the provider, binding ID, revision, and detection method. Observe native Codex and Codex/Grok ACPX credential files. Retain each pending event across runner restart until the durable outbox accepts it. Raw PRP consumers opt in with `observeCredentials: true` before session open. The bundled transport opts in for fresh sessions. **Alternatives considered** `harness.diagnostic` would mix a runner observation with harness diagnostics. Sending token values through PRP would expose credentials to event logs. Both approaches are unnecessary. **Roadmap alignment** I checked `ROADMAP.md`. This change extends the existing Runner protocol and does not duplicate a planned core feature. **Additional context** Refs: #11094, #14056, #12696. These describe credential copy-back and staging work. This PR adds the notification contract. It does not replace those private credential channels or claim to fix their full scope. ## What Changed - Add strict credential update payload validation to PRP v1, v2, and v3. Regenerate validators, the manifest, and the event catalog. - Watch supported credential directories with `notify`. Add a periodic read as a fallback and check at terminal and close boundaries. - Persist a private baseline and pending event together. Notify on account identity changes and retain the new baseline for later refreshes. Ignore incomplete files, unsafe files, and metadata changes. - Forward the event through the host transport. Test commit-before-ACK delivery and replay. - Commit pending runner observations before suspend and shutdown commands complete. Preserve the provider event backlog. - Create the private ACPX runtime directory before configuring observation. Cover fresh session startup with observation enabled. - Fence asynchronous PRP commits to the current authenticated connection and run authority. Ignore replaced callbacks while preserving the maintenance drain of admitted work. - Deliver observations found during final provider cleanup before clearing terminal recovery state. Retain the outbox and recovery state when the host ACK is lost. - Advance the pending Copilot candidate profile to v19 for the changed shared PRP source hashes. Preserve its v18 history and pending qualification status. Include the current fixture in the Docker build context. - Commit credential updates through an awaited host callback before PRP ACK, including during idle and close. Persist them in a separate run-log source namespace with company, agent, session, and runner checks. - Preserve public event metadata through server redaction. Register the event in task run details. - Document opt-in behavior, supported stores, event semantics, and the local run-log path. ## Verification - All 45 durable runner control tests and 24 native executor integration tests pass. - Ten Rust credential observer tests pass. They cover atomic replacement, polling, recovery, account-only changes and later refreshes for Codex and Grok, unsafe files, fresh runtime creation, and the Grok close handoff. Account identity stays private. - The full Runner TypeScript suite passes: 3,565 tests in 221 files. An additional 12 tests are skipped by the suite. - All 214 focused tests pass for the contract, Codex event routing, catalog, and durable host delivery. - All 13 Node protocol contract tests pass. - The exact Docker context integrity command passes: `docker buildx build --file .github/docker-context-checks.Dockerfile .`. - Rust formatting and generated protocol checks pass. - `pnpm -r typecheck` and `pnpm build` pass. The full local `pnpm test:run` was attempted with CI-compatible `init.defaultBranch=master`, then stopped after a reproducible failure outside this change: `git-workspace-expiration.test.ts` expects one re-expired workspace but gets zero. The same case fails in isolation on this macOS checkout. Full repository validation is provided by CI; the focused Runner and host tests pass locally. - All 253 native transport and restart tests pass against the final release binary. - All 3 durable recovery tests and 139 candidate profile and native admission tests pass. - All 41 host redaction and task protocol surface tests pass. `pnpm check:token-gates` passes. - All 467 CI policy tests pass, including the 15 CI profiling tests. The upstream secret-reference test fix is retained from master. No workflow changes are included. - The Apex host-delivery fix passes 381 targeted Runner tests, six host database tests, repository typecheck, and the production build. The Runner tests cover commit-before-ACK, retry after a rejected host commit, warm attachment, and close after the turn queue ends. - Prior head `2f125cfcb` passed CI, standard Greptile 5/5, and Apex 5/5. One CI shard was interrupted by GitHub runner shutdown; its single rerun passed. Both Apex findings are addressed: idle/cleanup host delivery and late callback reconnect fencing. Current head `5db78f082` passes CI (56 successful statuses and four optional skips) and standard Greptile 5/5. Apex review of the account-change update is 5/5 on this exact head, with no actionable issues. - All 121 durable control-plane and credential callback tests pass, including account-only change delivery, subsequent refresh on the new account during close, host commit retry, late success/failure races, forward cursor delivery on a replacement connection, and the existing maintenance retirement contract. All 27 native provider and durable recovery integration tests pass after the account-change update. ## Risks - File changes signal a stored credential update. They do not prove that an OAuth refresh succeeded. Changes between reads can be combined. - Unix Codex and Codex/Grok ACPX stores are supported. Other harnesses and credentials held only in memory need their own integration. - Hosts must update their event schemas before opting in. Older hosts omit the flag and receive no new event. Older runners ignore the flag. - The host must use its existing private credential channel and account checks to read or copy the token. Account switches trigger notification but do not authorize accepting the new account. The event grants no credential access. - Tests use synthetic credentials. No live OAuth refresh has been forced. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`, with high reasoning effort, repository inspection, tool use, and code execution. The session does not expose its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8843887c24 |
fix(runner): preserve managed GitHub access in OpenCode (#16488)
## Thinking Path > - Paperclip manages AI agents and their authorized connections. > - Native OpenCode agents use managed Git and GitHub launchers for shell work. > - Each operation must obtain credentials from the active run's broker. > - Two OpenCode environment filters dropped that broker binding before the shell started. > - The shell therefore ran anonymously even when connector tools had authorized access. > - This pull request preserves the explicit broker environment through both filters. > - The benefit is working GitHub shell access with current identity and revocation checks. ## Linked Issues or Issue Description **What happened?** Native OpenCode shell commands reported `GitHub capability_missing` despite an authorized GitHub connection. Both runner startup and provider startup dropped the controller's broker settings. Connector MCP calls used a separate path and could still succeed. **Expected behavior** A managed shell operation obtains credentials for the current run. A later operation uses the new identity after rotation and loses access after revocation. Ambient host credentials do not supply authority. **Steps to reproduce** 1. Start the native GitHub broker with an active run binding. 2. Pass its environment through native OpenCode startup. 3. Run `gh api user` from the provider shell. 4. Observe anonymous access before this fix. **Paperclip version or commit** Reproduced on master `6943904bad` before this change. **Deployment mode** Native Paperclip Runner with OpenCode, including remote execution. Related: #16401 fixes the separate oversized OpenCode SSE failure and is already merged. The same live-driver regression was verified locally against its exact pre-fix source and current source. This PR does not duplicate that fix. Searches found no open PR for this native broker propagation defect; #15309 concerns the legacy OpenCode adapter. ## What Changed - Use the existing GitHub credential environment projection at both OpenCode launch boundaries. - Preserve explicit controller authority while excluding an ambient host broker. - Exercise the real launcher and broker from an OpenCode child process, directly and through runner projection. - Check operation-time identity rotation, revocation, and exclusion of controller API credentials and persistent GitHub tokens. - Document the managed OpenCode shell contract. ## Verification - Red: both new `native-github-access.test.ts` cases returned `anonymous` instead of `fixture-a` before production changes. - Green: both cases pass after the fix, including `fixture-a` to `fixture-b` rotation and anonymous operation after revocation. - Full native GitHub broker suite: 18 tests pass. - OpenCode, provider-event, and configured-environment suites: 141 tests pass across eight files. - Separate SSE red/green verification: the existing live-driver regression times out on `fd057f6ac4^` and passes with merged #16401. The full OpenCode group also passes after restoration. - Full repository build and typecheck pass locally. All CI jobs pass for `23d734293f6a5c6d6128cae0c8773661816daf9a`. Final GitHub rollup: 55 successful checks/statuses, two skipped checks, and one neutral check; none pending or failing. Greptile gave that commit 5/5 with no findings or unresolved threads. - Paperclip Review spent over 23 minutes preparing its runtime without a provider assessment. Its run was cancelled for recovery and ended neutral (review execution did not complete). This neutral result is not a code assessment. No check policy or branch protection was changed. - The full local Vitest attempt first hit embedded PostgreSQL startup failures. Running the unchanged `initdb --version` and `postgres --version` independently showed missing native library symlinks. The dependency had been installed with scripts disabled. Running its bundled `hydrate-symlinks.js` restored startup; all 15 affected native-session cases and all 112 AI connection tests then passed without application code changes. The original full-suite attempt was stopped after the setup failure and its log is retained. A complete local suite pass is not claimed. - The broker and executable are local fixtures. No live GitHub account, credential change, or test PR is required by these regressions. ## Risks The change affects credential delivery into OpenCode processes. It reuses the existing bounded GitHub projection and accepts broker authority only from explicit input. Each shell operation still uses the normal broker, scope, identity, and revocation rules. Unrelated controller secrets remain excluded. There are no database or permission-policy changes. ## Model Used OpenAI `gpt-6.1-sol`, through Codex, with xhigh reasoning and repository tools/code execution. The session does not report its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the focused regression and affected suites locally and they pass; the full local attempt and setup failure are disclosed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (Paperclip Review is neutral/incomplete, as disclosed above) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |