mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
ccbfefef5f23d312bc11130d14eab05aec97d900
1018
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
ccbfefef5f |
test(ui): retire 33 duplicate executions and share render setup (#16072)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
8b27af77db |
fix(heartbeat): wait for a live holder of a reused shared execution workspace (#16371)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat pre-dispatch gate decides if a run can use a shared workspace in its execution environment. > - A child task can inherit the shared execution workspace of its parent. The parent run is often still live when the child run starts. > - On a sandbox environment, the gate failed the child run with `configuration_incomplete`. It did this for every concurrency policy, also for the default `auto` policy. > - Recovery then blocked the task and told the operator to bind missing secrets. That was not the real cause. An operator had to select a different workspace by hand. > - Shared project workspaces already use the `auto`/`serialize` policy to wait through the `workspace_busy` retry path. > - This pull request applies the same policy to a reused shared execution workspace. > - The benefit: the child task waits for the parent run and then continues automatically. It does not fail and become blocked. ## Linked Issues or Issue Description **What happened?** A parent agent created a child task. The child inherited the parent's shared execution workspace. The child agent used a sandbox environment. The child run started about 1.5 seconds after the parent created the task, while the parent run was still live. The run failed before the adapter started with this error: "This environment cannot place concurrent agents in one physical shared folder. Use local/SSH sharing or isolated task workspaces." Recovery set the task to `blocked` with the message "required secret/env bindings are missing". **Expected behavior** With the default `auto` policy, the child run waits for the parent run (`workspace_busy`) and then continues automatically. This is the same behavior as for a shared project workspace on a sandbox. **Paperclip version or commit** Staging release `8ffc5748`. The same code is on `master`. **Deployment mode** Hosted cloud (staging), agent on a sandbox execution environment. **Steps to reproduce** 1. Use an agent with a sandbox environment. 2. Bind two issues to the same `shared_workspace` execution workspace. Set `executionWorkspacePreference: reuse_existing` on the second issue. 3. Keep a live run on the first issue. 4. Wake the agent on the second issue. Before this change, the run fails with `configuration_incomplete`. After this change, the run is deferred with `workspace_busy`. ## What Changed - `environment-selection.ts`: When a reused shared execution workspace has a live holder on a non-local/non-SSH environment, the gate now reads `sharedWorkspaceConcurrency`. With `auto` or `serialize`, it throws `WorkspaceBusyDeferral`. Only `allow` still fails with `shared_realization_unsupported`. - `retries.ts`: Added `findExecutionWorkspaceHolder` next to `findSharedWorkspaceHolder`. It keeps the old match rule. A running holder that has had no output for more than one hour (the existing staleness cutoff) is still returned, with `stale: true`. A live holder is preferred over a stale one. - `environment-selection.ts`: A stale holder still owns the folder, because silence does not prove that the run stopped. The child run therefore fails with `configuration_incomplete` (`shared_realization_unsupported`, `holderStale: true`). It does not share the folder with the holder, and it does not wait without limit. This is the same result as before this pull request, with a more specific message. - `retries.ts`: `WorkspaceBusyDeferral` can now carry an `executionWorkspaceId`. The run result, the run event, and the cancellation reason name the execution workspace when it applies. - `stranded-notice.ts`: Added a specific recovery notice for `shared_realization_unsupported`, and a second notice for a stale holder. They no longer tell the operator to bind missing secrets. - `doc/experimental/issue-worktree-support.md`: Documented that `allow` is only concurrent on local/SSH, and that a reused shared execution workspace follows the same rules. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-workspace-busy.test.ts -t "reused shared execution workspace"` covers 4 tests: - `auto` on a sandbox with a live holder: the run is cancelled with `workspace_busy`, a retry is scheduled, no Sentry run failure is reported, and the issue does not become blocked. - After the holder finishes, the scheduled retry is promoted and runs. The adapter gets the inherited shared folder, and the issue keeps its execution workspace. - `allow` on a sandbox: the run still fails with the shared-folder message and no retry is scheduled. - A holder that is still running but silent past the staleness cutoff: the run fails with `holderStale: true`, the adapter does not start, no retry is scheduled, and the holder is not touched. - `npx vitest run src/services/recovery/stranded-notice.test.ts src/services/heartbeat/environment-selection.test.ts src/services/heartbeat/retries.test.ts` covers the notice and the selection unit tests. - First commit (`d0b7a092`): the 3 original integration tests and the unit suites passed locally, and all CI checks passed. - Follow-up commit (`3c7e515`, which addresses the Greptile review): these tests were **not** run locally. Dependency install was killed twice by the sandbox's 4 GB memory limit. CI is the first run of the new and changed tests. - Known local failure: `does not hide ambiguous, malformed, provider, or transport cancellations behind workspace_busy` fails in this sandbox also on unchanged `master` (checked with `git stash`). This change does not cause it. - Not run locally: the full server `tsc --noEmit`, for the same memory reason. CI runs the typecheck. ## Risks - Behavior change: before, a run with a live holder of a reused shared execution workspace on a sandbox failed at once. Now it waits with `auto`/`serialize`. A long parent run can delay the child. If the holder has no output for more than one hour, the child fails as before instead of waiting without limit. - An explicit `allow` policy keeps the old failure. - Local and SSH environments do not change. - No schema or migration change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in the Paperclip agent runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (first commit only; see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
66724b8ce0 |
perf(auth): project only required run context fields (#16380)
Project only the required JSON fields for saved-run policy and direct-parent report authorization. Preserve current reads, JSON types, identity checks, permission decisions and legacy task-key precedence. Validation: 186 affected cases; 17 database transfer-budget cases after the shared startup-timeout correction; full typecheck/build; exact-head full CI, E2E, Superagent and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d73d7a8651 |
perf(skills): batch repository dependency reads (#16377)
Batch dependency Markdown reads through the bounded Git snapshot reader and queue newly discovered dependencies once. Preserve package contents, limits, authorization, audits, and reference order. Validation: 63 affected cases; full typecheck/build; exact-head full CI, E2E, Superagent and Apex 5/5. Local Git comparison: 502 individual reads reduced to three batches with identical entries. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
32ede08918 |
fix(grok): copy pairing codes during agent reconnect (#16373)
Grok Bot reconnect showed a masked one-use code without a copy action. Add the shared Copy code control, preserve masking and copy-error feedback, and keep the code out of Bot instructions. Cover the reconnect flow and readiness wait in tests and Storybook, and update the user guide.
Verified head
|
||
|
|
ccc77c5b5e |
fix(server): record legacy controller lease-loss timings (#16359)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat runner executes legacy agent runs, and a controller lease watchdog keeps durable ownership of each run > - The watchdog renews the lease every 10 seconds and stops the run when a renewal does not finish within the 60-second lease > - A staging run failed in setup with `LegacyControllerLeaseLostError` and the `renewal_deadline` cause, after about eight minutes of preparation and with no server restart > - The setup failure path did not record the cause, and no evidence showed why the renewal did not finish: a stalled database query or a blocked event loop > - This pull request records bounded watchdog timings with each lease loss, on the setup failure path and on the cancellation path > - The benefit is that the next event shows which of the two causes stops renewal, which is the missing fact to fix the root cause ## Linked Issues or Issue Description Refs #16084 (added the closed lease-loss cause for cancelled runs) Refs #16355 (retries a run that a restart stopped before dispatch; a different path) **What happened?** A legacy `paperclip_runner` run on staging failed with `setup_failed` and the message "Legacy controller lease lost". The stack shows the watchdog deadline timer (`renewal_deadline`). The run was in `executionStage: "preparing"` for about 467 seconds. The server started about 11 minutes before the run, so this was not a restart. The run record and the Sentry event do not show the lease-loss cause, because the setup failure path does not read it. Nothing shows if a renewal query was in flight at the deadline, or if the deadline callback ran late. **Expected behavior** When the watchdog loses the lease, the run records the closed cause and the watchdog timings. This applies when the loss fails setup and when it stops the adapter. The failure report contains only the closed cause and bounded numbers. **Steps to reproduce** 1. Start a legacy run with a controller lease. 2. During setup, make a lease renewal query hang for more than 60 seconds, or block the event loop past the deadline. 3. Let the watchdog abort the run. 4. Read `resultJson` on the failed run. Before this change, it has no lease-loss evidence. ## What Changed - `legacy-controller-lease.ts`: The watchdog tracks the last successful renewal, the renewals in flight, the scheduled deadline, and the renewal count. `LegacyControllerLeaseLostError` now carries `leaseLossTiming`: `sinceRenewalMs`, `renewalPendingMs` (or `null`), `deadlineLateMs` (only for `renewal_deadline`), and `renewalCount`. Ownership behavior does not change. - `run-cancellation.ts`: Add the `LegacyControllerLeaseLossTiming` type and `readControllerLeaseLoss()`. The reader accepts only a closed cause and non-negative finite numbers, and clamps each value to 24 hours. - `heartbeat/run-completion.ts`: Persist `resultJson.controllerLeaseLoss` with the cause and timings. This applies to a lease loss that stops the adapter and to a lease loss that fails setup (the thrown abort reason). - `run-failure-diagnostics.ts`: Report `controllerLeaseLossCause` and the four bounded timing fields from `controllerLeaseLoss`. A setup failure now gets the cause label too. - `doc/run-log-events.md`: Document the new record and how to read the timings. - Tests: watchdog timings for a stalled query, a blocked event loop, and a failing query; completion records the evidence for an adapter stop and for a setup failure; the failure report clamps values, drops unknown keys, and drops a record without a closed cause. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/services/legacy-controller-lease.test.ts src/services/heartbeat/run-completion.test.ts src/services/__tests__/run-failure-report.test.ts src/services/run-cancellation.test.ts` - I could not run these tests locally. The sandbox has a 4 GB memory limit, and `pnpm install` was stopped by the out-of-memory killer during lockfile resolution. CI is the test run for this change. - After deployment, a new lease loss shows `controllerLeaseRenewalPendingMs` and `controllerLeaseDeadlineLateMs` in the Sentry execution context. A pending time near 50 seconds or more points at the database. A large deadline lateness points at the event loop. ## Risks - Low risk. This change adds diagnostics. It does not change when the watchdog stops a run, and it does not prevent the lease loss. - The new `resultJson.controllerLeaseLoss` field is server-written. Readers ignore unknown values. Old rows do not have it. - The failure report gets five closed or numeric fields. It gets no reason text, IDs, or user content. - The event-loop test blocks the loop for about 80 ms to make the deadline late. It uses real timers, so a very slow CI host makes the lateness larger, not smaller. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, Sentry and GitHub connectors). The exact context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (not possible in this sandbox; see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
3dffd2197d |
test: consolidate owner fixtures and retain real regression proof (#16047)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tests protect sandbox execution, run recovery, credentials and the board UI. > - Several owner suites repeat the same setup and prove the same result more than once. > - These copies add maintenance work and can hide weak assertions. > - Each cut needs a retained test that proves every distinct contract. > - This pull request consolidates repeated setup and removes twelve redundant executions. > - It also replaces two weak assertions with checks of real outputs. > - The benefit is 2,831 fewer test lines with clearer regression proof. ## Linked Issues or Issue Description **What existing behavior does this improve?** The maintenance and regression strength of the existing Vitest owner suites. **Subsystem affected** Sandbox adapter utilities, server heartbeat recovery and environment capabilities, and the board UI. **Current behavior** Tests copy database rows, query builders, render calls and runtime inputs. Some cases repeat an existing contract. One token test omits a real bridge credential. A PostHog host comparison can compare absent controls. **Proposed behavior** Use small local helpers with fresh values and the same action order. Keep each distinct contract in an owner test. Check real bridge credentials and fixed PostHog method states. Carry the exact persisted first-run success assertion into the stronger cancellation keeper. **Reason and benefit** The ten test files shrink by 2,831 net lines. Twelve redundant executions retire. All other case tables, inputs, security boundaries, timeouts and cleanup rules remain. **Breaking changes** None. This round changes tests and audit evidence only. **Additional context** Refs: #15978, the merged first cleanup. Refs: #16022, the merged prior cleanup. Its squash merge is `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Related open work: #14634 repairs feedback fixture completion; #14477 supplies explicit sandbox fixture tools; #14198 changes cleanup retry behavior in another heartbeat file. This audit preserves those separate action and cleanup contracts. It does not apply their changes. This PR was first stacked on `codex/test-audit-followup`. After #16022 was squash merged, only the two Round 6 commits were replayed onto merged master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. The original stack head is preserved in a local snapshot branch. The current diff contains only ten test files and two evidence files. No merge is requested. ## What Changed - Consolidate repeated setup in three sandbox adapter suites. Remove seven redundant executions and retain the real protocol, cleanup and scratch-file checks. Save 875 lines. - Consolidate heartbeat database fixtures and read queries. Retain exact wake cancellation, total run count and the actual first-run success receipt. Remove one duplicate execution. Save 1,372 lines. - Remove two repeated rejected-capability cases. Carry the absent snapshot assertion into the retained runner case. Save 22 lines. - Consolidate UI fixtures and render calls. Remove two repeated connection cases. Replace the weak PostHog comparison with fixed method and required-key checks in both real hosts. Save 562 lines. - Record audit scope, decision and expansion counts, all twelve keeper mappings and deliberate mutation results in the [audit report](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6.md) and [controls evidence](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6-controls.json). ## Verification Source baseline: `a090035cafb29c06f6fca2dbe43561ad866b7c1b`. All ten original test programs are byte-identical at the stacked base `58c5c4aa241c85f4665a3159c192888bbb2a5cfa`. - Baseline: 1,841/1,841 cases passed in the main thirty-file owner/sibling run. Another 219/219 cases passed in nine extra files. - Preservation: all four proposal packages passed independent review. Whole-program helper expansion retains ordered inputs, callbacks, awaits, queries, fresh allocations and hooks outside the declared assertion migrations and repairs. - Candidate: all 2,048 cases passed across 39 complete owner/sibling files. The main run passed 1,831 cases. The extra run passed 217 cases. Both runs had zero failures and zero skips. - PostHog control: the repaired positive case passed. The original weak case passed with authentication choices removed. The repaired case failed at its fixed OAuth aria-checked assertion under the same mutation. - Eight accepted mutations failed at their intended assertions. They cover bridge-token exclusion, fallback cleanup, READY bytes, real scratch-file preservation, exact wake cancellation, the persisted first-run success receipt, rejected capability snapshots and reachable PostHog method controls. Two original weak cases passed with the same defects. All production source bytes were restored. - Rejected control: changing run status before legacy release timed out before wake cancellation. This result is not counted as assertion proof. Its replacement changed only the persisted receipt after real drain in the existing transaction. It failed at the direct fixed succeeded-row assertion. - No new code comments or 1,000-line threshold crossings were introduced. No production, schema, lockfile or workflow changes are included. On the exact final test sources, later committed as `d8de9422f0703d4508a9dc880b67a07f13e0f110`, `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build` passed. The required full `pnpm test:run` completed with **exit 1** after **52.14 minutes** on clean, immutable `d8de9422f0703d4508a9dc880b67a07f13e0f110`. HEAD and tracked sources remained unchanged. The first general-server group ran 889 files: **882 passed, one failed and six skipped**. Its 18,032 cases were **17,925 passed, two failed and 105 skipped**. The only failing file was `server/src/__tests__/remove-test-home.test.ts`. On macOS, the read-only runtime bundle and symlink cleanup cases returned `ENOTEMPTY`. The permission-error case expects `EACCES` or `EPERM`. Four reported failure entries cover these two cases, including their cleanup errors. The test and helper are byte-identical to previous master `4b766b872993045a2e787404f0475503683e5ded` and audit master `e364b92c82`. The default wrapper stopped after that group failed. **Later local groups did not run.** The earlier public-MCP, HTTP log redaction and workspace snapshot failures did not recur. The 39 focused owner/sibling files and all 48 remote CI jobs completed independently. This result is an unresolved macOS cleanup limitation, not full local success. No test, skip, retry, timeout or cleanup rule was changed to hide it. The three adapter files received a whitespace tidy after focused execution. Full runtime AST and string/template/regex byte equality link those tested sources to the final sources. All 134 adapter helper calls were checked again. Replayed code commit `3d3defd2a60595a623045d9e2c6c586f4acb3452` has the exact final test bytes. Evidence commit `e868e6645ce0d598d756d3458f0b3d71afd6db4a` adds the report and controls JSON, with one historical ancestry clarification. All ten accepted test hashes and all seven production-owner source guards match. No package, lockfile, workspace, TypeScript, Rust or Vitest dependency manifest changed since the audit snapshot. The [original stack CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38312876910) passed all 48 jobs on head `c171d6c0d603d22bb997c1f9a9bdcd5bcc5d22aa`. Those results are historical stack proof. The [fresh master-context CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38318223856) completed successfully with **all 48 jobs passed** on current head `e868e6645ce0d598d756d3458f0b3d71afd6db4a`. Its tested merge commit was `41292af60c6eb60d477637d23822bb17b08dc07e`, combining workflow base `88bf4c6c1d222f818faa56257efe79207c892a6c` with that head. The delivery commits remain based on pinned master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Both required master checks, `ci / verify` and `ci / e2e`, passed. Two separate optional Storybook checks were skipped by their scope gate. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/16047#issuecomment-6133323165) gives the current head **5/5**, with no findings, inline comments or unresolved review threads. The [current-head commitperclip review](https://github.com/paperclipai/paperclip/actions/runs/38318223666) also completed successfully. This PR remains open and draft; no merge is requested. The local macOS failure above remains unresolved. ## Risks - A helper could share a value or capture the wrong binding. Independent expansion and lexical binding checks guard the actual helper calls. - A retired case could contain a distinct assertion. Each cut has an exact keeper map and carried observations. - A mutation could fail before the intended assertion. Controls reject timeouts and earlier failures. The rejected receipt mutation remains in the evidence. - The delivery ancestry changed after the parent squash merge. Exact source hashes preserve the audited tests and producer receipts. Fresh CI and review passed in the current master context; the broad local run remains evidence from its immutable audit snapshot. - The complete local default failed in two unchanged macOS cleanup cases. Later local groups were not run. This limitation remains visible even though focused checks and remote CI passed. ## Model Used OpenAI Codex, based on GPT-6, assisted with this work. The exact served model ID and context window are not exposed in this session. It used repository tools, code execution, AST analysis and parallel audit agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
cafa25029c |
fix(native-runtime): recover unclaimed remote startup (#16349)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runtime preserves task execution across server restarts. > - Remote workspace preparation can finish before the executor claims its first provider attempt. > - The recovery classifier required a provider checkpoint even when the durable coordinator proved that execution had never started. > - A deployment during this interval therefore left safe startup work on a permanent recovery hold. > - This pull request admits that narrow startup state to the existing same-run bootstrap path. Existing execution and ownership checks remain in force. > - The original preparation yields when recovery owns the coordinator. It cannot fail or clean up the recovered run. > - Tasks can continue after this interruption without an operator retry or a duplicate provider turn. ## Linked Issues or Issue Description **What happened?** A server deployment interrupted a prepared remote workspace before the native executor's first claim. The coordinator had attempt zero, controller generation zero, and no owner, process, provider event, or checkpoint. Restart classification returned `ambiguous_remote_recovery_evidence` and blocked the task. **Expected behavior** The server should recognize this durable never-claimed state and continue the original run through normal execution admission. A prior claim, uncertain ownership, or contradictory execution evidence must retain the recovery hold. **Steps to reproduce** 1. Prepare a remote workspace for a native run. 2. Stop the controller before the executor commits its first claim. 3. Restart the controller and classify the running native task. 4. Observe that the old classifier blocks it because no provider checkpoint exists. Related: #16341 preserves remote recovery failure reasons. #16344 strengthens remote restart qualification. Neither changes admission for an untouched remote startup. ## What Changed - Require a server-owned fencing version written only with fresh native identities. Older unmarked preparations stay held during rolling upgrades; caller profile data cannot add that marker to an existing run. - Leave the current server's active preparation with its existing execution. - Yield through the controller-detachment path when a different execution owns the coordinator lease. Lease expiry does not grant takeover authority. - Preserve run-scoped gateway tokens during detachment so stale preparation cannot revoke the recovered worker's tools. - Admit a valid prepared remote workspace only when the locked coordinator proves attempt and controller generation zero, no owner, and no prior session or execution evidence. - Check launch receipts and provider source events before granting bootstrap recovery. - Keep the original run and session. Preserve atomic recovery claims, executor durable-state checks, and normal admission gates. - Add real PostgreSQL regression tests for competing controllers and seven contradictory evidence cases. - Update recovery documentation, the run-log reason, and both capability maps. ## Verification - The competing-controller regressions failed before the ownership fix and pass for both live and expired foreign leases. - On the final revision, 77 runtime-selection, restart recovery, real PostgreSQL, and heartbeat boundary tests pass with no skips, including all local runner process fixtures. The full native executor file passes all 698 tests. - Another 91 live-run route tests pass locally. An earlier CI timeout in that unrelated file passed on retry; the complete final-head CI passed without that failure. - `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check`: passed locally. The map check verifies structure only. - Local `pnpm test:run` did not finish: an early attempt had build-related collection failures, and later full-suite execution was stopped after complete CI passed. Targeted suites passed after build preparation. This is not a local full-suite pass. - The complete [PR workflow](https://github.com/paperclipai/paperclip/actions/runs/38669996667) passed on `2baffb3652bfeb4705a30ca3706fbda6c3d72aa4`, including all test shards, build/typecheck, runner verification, and browser E2E. All 55 applicable review/security/CI checks succeeded; two Storybook checks were skipped. - [Greptile Apex](https://github.com/paperclipai/paperclip/pull/16349#issuecomment-6180848957): 5/5 on the same final head, with no unresolved findings. - No live remote provider restart is claimed by the database regression. ## Risks - A bare executor can no longer take a foreign coordinator lease after expiry. A verified recovery claim must transfer authority first. - Existing preparations without the server-owned fencing marker retain their recovery hold during rolling upgrades. - Incorrect admission could duplicate provider work. This path requires a positive durable never-claimed receipt, rejects contradictory launch evidence, and keeps the executor's existing durable-root verification. - Previously claimed or incomplete remote recovery evidence remains blocked. This change does not clear historical terminal holds. - No schema, API, dependency, or UI changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution, and browser inspection. The exact serving snapshot ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing> |
||
|
DottaandPaperclip
|
ecc0ab8174 |
test(e2e): require remote restart identity and copyback receipts (#16344)
Require original remote run identity, sandbox lease continuity, native receipt continuity and successful workspace finalization in the Codex Daytona controller-restart cell. Add positive and negative calibrations, including immediate rejection when the original run ends before fault injection. All current-head CI checks pass and Greptile is 5/5. Live Daytona qualification remains separate. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ff0f976de1 |
fix(native-runtime): retain remote recovery failure reasons (#16341)
Preserve closed marker-read and process-verification reasons through native recovery failures and operator reports. Keep existing transport errors and reattachment ownership checks. Add shell, process-probe, persistence and privacy regression coverage. All current-head CI checks pass and Greptile is 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
3935e300dd |
fix(acpx): bound Claude MCP tool calls when runtime MCP servers attach (#16360)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted `claude_local` runs use the ACPX engine. Paperclip gives each
run its runtime MCP servers (for example the `paperclip-assigned` tool
gateway) as ACP `mcpServers` entries.
> - Claude's default MCP tool timeout is far longer than a run. ACP
`mcpServers` entries have no per-server timeout field. So the agent
waits on an MCP call until the call answers.
> - Hosted error reports show runs that end with "Run exceeded the
adapter execution timeout (sandbox)". The stop-time diagnostics from
#15962 show the problem: the agent went silent about 50 seconds into the
run, three tool calls of ACP kind `other` stayed pending, and nothing
ended them until the 4h sandbox backstop.
> - The gateway bounds the remote provider call (10s default, 60s max).
But other steps on the gateway `tools/call` path have no deadline: DB
queries, row locks and pool acquisition, the credential steps before the
provider timer starts, and the stdio child exit wait. One stuck step
holds the agent turn for hours.
> - This pull request gives the agent's MCP client a deadline:
`MCP_TOOL_TIMEOUT` for ACP Claude runs that receive runtime MCP servers.
> - The benefit is that a stalled gateway call becomes an MCP error that
the model can see. The turn continues and does not spend the full 4h
execution budget.
## Linked Issues or Issue Description
- Refs #15962 (diagnostics that identified the silent turn with pending
tools)
- Refs #15448, #15456, #15850 (related gateway timeout and
connection-health work; this PR does not overlap with them)
**What happened?**
A hosted `claude_local` run sent parallel MCP tool calls. No call
answered. The run stayed silent for about 4 hours. Then the sandbox
execution timeout ended it with "Run exceeded the adapter execution
timeout (sandbox)".
**Expected behavior**
A stalled MCP call ends with an error after a bounded time. The agent
sees the error and continues.
**Steps to reproduce**
1. Start an ACP `claude_local` run that has runtime MCP servers.
2. Make the gateway `tools/call` request stall (for example, hold a lock
that the request needs).
3. See that the run stays silent until the adapter execution timeout.
**Paperclip version or commit**
`master` at
|
||
|
Devin FoleyandPaperclip
|
6fae6f2cb8 |
fix(wake-queue): preserve pending work when a queued run is stopped (#16357)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The wake queue releases task locks and can promote saved follow-up work when a run ends. > - An operator can stop a queued run before dispatch selects its runtime. > - The legacy release path can promote the old follow-up after that Stop. > - For an obsolete routine cycle, this can conflict with the current active cycle after cancellation has already committed. > - This pull request preserves the deferred input without promoting it at that release boundary. > - The benefit is that a queued Stop stays stopped and does not return an error from starting an obsolete follow-up. ## Linked Issues or Issue Description Related: Refs #14300 and Refs #16339 describe other pre-dispatch cancellation paths. Related PR #16238 changes former-owner wake selection. This change addresses expected operator cancellation before legacy runtime selection, without changing those contracts. **What happened?** Stopping an unstarted legacy run committed its cancellation, then tried to promote a deferred wake. When another cycle of the same routine was already active, the issue lock update failed against `issues_open_routine_execution_uq`. The cancel request returned HTTP 500 even though the run was cancelled. Without a competing cycle, the Stop launched the old follow-up. **What did you expect to happen?** Release the stopped run's own locks and preserve deferred input for a later eligible instruction. Do not start the old queue as a side effect of Stop. **Steps to reproduce** Create two cycles of the same routine. Keep the current cycle active. Give the old cycle an unstarted legacy run with a verified bootstrap record and a deferred wake. Cancel that run as a board operator, then release its issue execution. The added database regression reproduces the failed lock update before the fix. **Paperclip version / commit** Reproduced on parent commit `976cdc77ed564dc3e014621505890b647e554a7f`. **Deployment mode** A remote runtime exposed the symptom. The regression uses embedded PostgreSQL and the real wake-queue adapter. ## What Changed - Recognize an expected operator Stop only when the legacy run has no runtime-selection time, start time, process-start time, or process identity. - Use the existing release boundary to preserve deferred input without promoting it. - Add database regressions for ordinary Stop, a competing routine cycle, repeated release, provider cancellation, runtime selection, and dispatched-run reconciliation. - Document the behavior and update both recovery maps. ## Verification - Before the fix, both queued Stop cases failed. The competing-cycle case raised `issues_open_routine_execution_uq`. The three control cases passed. - `node node_modules/vitest/vitest.mjs run --project @paperclipai/server server/src/modules/wake-queue`: 7 files and 214 tests passed, including all five new database cases. - `node scripts/check-maps.mjs` and `git diff --check` passed. - `pnpm -r typecheck`, `pnpm build`, and all 10 map-script tests passed locally. The complete CI suite is green on `8e204eef87f1a593df5c183abef764e4b9eb5fd1`. The remaining serial local full-suite run was stopped once that unchanged-head CI verification passed; a local full-suite pass is not claimed. Apex review passed at 5/5 on the same head with no review threads. ## Risks - The change is limited to recorded operator cancellation before legacy runtime selection. Selected runtimes, dispatched work, provider cancellation, and native stop acknowledgement keep their existing paths. - Deferred work is retained. A later eligible instruction must still use normal admission and all current gates. - No schema migration or new authority is required. This regression is not proof of a live provider stop. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact serving snapshot ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
976cdc77ed |
fix(recovery): expose settled holds and their preserved decisions (#16169)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task recovery preserves work when execution or tool outcomes are uncertain. > - Automatic recovery can settle its record while keeping replay blocked. > - The recovery-actions API omitted those records unless they held workspace files. > - Operators and agent tools could therefore see an empty recovery history for a task that still had an execution hold. > - This pull request lists those holds and shows their matching decision in the existing stopped-run notice. > - Existing stop verification, continuation, and file repair gates remain in force. ## Linked Issues or Issue Description **What happened?** A cancelled or failed task can retain a settled recovery record with `automaticRecovery.replay: blocked`. The task remains held, but the recovery-actions API returns no action. The board notice also lacks that record's decision. **Expected behavior** List settled execution holds that still block replay. Let an authorized board user inspect the current action and run in Recovery details. Preserve the existing guarded continuation and required-file repair paths. **Steps to reproduce** 1. Stop a run whose outcome requires inspection. 2. Let automatic recovery settle its record with replay blocked. 3. Read the task's recovery actions and open its stopped-run notice. 4. Before this change, the action list is empty unless workspace recovery is required. **Paperclip version or commit** This PR includes current master through `66424362a85`. **Deployment mode** Authenticated and local deployments with persisted execution recovery holds. The original API correction and route regression are by Tao Xie (@TaoXieSZ). This update preserves those commits and adds the board inspection path, current documentation, and regression coverage. Related: #14840 describes controller lease loss. This change exposes recovery records; it does not change controller lease renewal. ## What Changed - Include settled execution reconciliation holds with replay blocked in the company-scoped recovery-actions response. - Read the matching action from the existing stopped-run notice. Show its decision under Recovery details, including when active recovery is empty. - Reject mismatched historical records in the notice. Bind history to the current action/run and refresh visible decisions every five seconds. Keep history read errors visible beside the current guidance. - Preserve existing run inspection, new-message continuation, and required-file repair controls. No new reconciliation form is added. - Update the execution documentation and both recovery capability maps. ## Verification - Recovery route suite with real embedded PostgreSQL: 80 passed, none skipped. - Notice and retry-client suites: 36 passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm check:token-gates`: passed. - `node scripts/check-maps.mjs`: passed. This checks structure only. - `git diff --check`: passed. - Full `pnpm test:run` was started. It is not yet a full local pass. Initial database tests were affected by missing packaged macOS library aliases; the aliases were restored in this worktree's dependencies, and the complete recovery route suite then passed. - All CI checks passed on `09e4461e4d3a0960a4dc8bc270a2a10c628bf524`, including the full test matrix, typecheck, build and browser checks. Optional Storybook jobs were skipped. - Apex: 5/5 on the same head. Both review threads are resolved; the reviewer confirmed the existing guarded continuation design. ## Risks - The API response now includes settled records that still block replay. Consumers must distinguish a settled record from permission to continue. - Recovery history is supplemental. A failed history read keeps the current blocker and guidance visible. - The UI binds the decision to both the current action and run. It does not certify tool outcomes or grant execution authority. - No schema, dependency, telemetry, or runtime admission changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution, assisted with the board inspection and verification follow-up. The exact serving snapshot ID and context window size are not exposed in this session. The original API work is retained from Tao Xie's contribution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; full-suite limits above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Devin Foley <devin@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
66424362a8 |
perf(runtime): skip discovery without a Browser Use connection (#16345)
Skip full tool discovery when no enabled, active Browser Use connection exists for the run company. Configured connections retain existing authorization and discovery.
Verified exact head
|
||
|
DottaandPaperclip
|
c41e00bca4 |
fix(server): index workspace source task enumeration (#16342)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent startup checks access to every source retained by a workspace. > - Repository admission repeats these checks while it holds a company lock. > - PR #16334 indexed the workspace privacy query. The next source-task query still cast the task primary key to text. > - That second query scanned company history for every workspace and delayed other agent launches. > - This pull request keeps the task column indexed and guards the retained identifiers before conversion. > - The benefit is faster startup with the same source validation and access decisions. ## Linked Issues or Issue Description Refs #16334. **What happened?** Agent launches still spent minutes preparing repositories after the first authorization query fix. The source-task lookup scanned 14,191 task rows for one matching source. **Expected behavior** Source validation should use the task primary key and workspace-binding index. It must still reject missing, malformed, noncanonical, and foreign-company sources. **Steps to reproduce** 1. Seed company task history and a workspace with attached and retained source tasks. 2. Call `assertTaskWorkspaceAccess` and capture its source-task SQL. 3. Run `EXPLAIN (ANALYZE, FORMAT JSON)` on that query. The baseline scans unrelated task history. ## What Changed - Convert retained text identifiers with the existing canonical UUID guard. Keep the indexed task column as a UUID. - Exercise the real source-access service against PostgreSQL. Verify complete source results, malformed and uppercase input behavior, and rows examined by the plan. - Document the second source-validation query and its regression coverage. ## Verification - Red: the new regression failed because the source lookup examined 1,202 rows for two sources. - Green: four focused workspace suites passed, 147 tests total. The final regression also checks malformed and uppercase query inputs. - Live read-only comparison: two warmups, then five samples per variant in alternating order. Median client query time changed from 4.88 ms to 0.35 ms. Complete result rows matched. Shared buffer hits changed from 3,984 to 5. These measurements cover this SQL statement, not total agent startup. - The complete source-access helper also passed for the same retained reader in read-only transactions. Its median changed from 15.75 ms to 10.75 ms; repository admission still repeats other checks. - `pnpm -r typecheck` and `pnpm build` passed locally. The staged Linux build and all seven final regression cases passed on Node 24.20.0. - All 55 current-head checks/statuses passed; two Storybook checks were skipped. Greptile reviewed `6cfb755` at 5/5 with no findings. - The full local test run encountered a tracing-test failure in `native-session-executor.test.ts`. Both that test alone and the complete file passed on rerun (698 tests). The remaining full-suite run is still in progress. ## Risks - This changes SQL planning on an authorization path. The canonical guard preserves the former exact lowercase text match and avoids UUID parse errors. - Company filters, missing-source checks, project checks, and per-source access decisions remain in place. - No schema migration or new index is required. Repeated checks across large repository histories remain a separate source of startup work. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, shell tools, code execution, and browser verification. The exact model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
90a45ef254 |
perf(skills): skip inventory work for empty runtime selections (#16343)
Return before company skill inventory work for a known empty native or stored-key selection. Preserve nonempty stored-key precedence, current access/version checks and legacy automatic skills.
Verified head
|
||
|
Devin FoleyandPaperclip
|
cda77edc60 |
perf(server): stop readiness waiting on startup housekeeping (#16329)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On restart, `/api/health` reports `starting` until startup recovery finishes, and deploy tooling waits for `ok` > - After heartbeat recovery, readiness also waits for housekeeping that every scheduler tick repeats and that serving requests does not need > - #16330 added OpenTelemetry spans for the main startup phases; native recovery and orphan reaping are the slow ones, and both run the native finalization reconciliation pass, which has no inner breakdown > - This pull request stops readiness waiting on that housekeeping, and records the phases of slow reconciliation passes as spans > - The benefit is faster readiness on every restart, and traces that show where a slow reconciliation pass spends its time ## Linked Issues or Issue Description Builds on #16330, which records `controller.startup.<phase>` spans. Related to #16328. **What existing behavior does this improve?** Server readiness time after a restart, and diagnosis of slow native recovery. **Current behavior** - After heartbeat recovery, readiness waits for the custom-image setup cleanup, tool connection health (remote MCP checks), decision expiry, the adapter and setup-token login reapers, environment lease cleanup (provider calls) and the retention sweep. Each also runs on every scheduler tick. On one instance they added 11–28 s to every measured restart. A throw in any of them fails startup. - On the same instance, native recovery took from 33 s to 11.5 min and orphan reaping from 30 s to 4 min across restarts in one hour. Both run the native finalization reconciliation pass. The #16330 spans time each step as a whole, but not the phases inside that pass. **Proposed behavior** - The housekeeping steps run after readiness. The scheduler interval still starts after them. - With an OTLP endpoint, a reconciliation pass of at least one second records a span with one child span per phase. **Reason and benefit** Readiness depends only on what serving requests and run recovery need. Slow reconciliation passes become attributable to a phase in the existing traces, without adding trace volume for the periodic pass. **Breaking changes** None. A failed housekeeping step no longer fails startup; it is logged and the next step runs. ## What Changed - `server/src/index.ts`: moves the housekeeping steps into `runStartupHousekeeping`, which runs after readiness under `trackHeartbeatSchedulerWork` (idle sleep and shutdown wait for it). Each step catches and logs its own failure (`startup housekeeping step failed`). The heartbeat scheduler interval starts when housekeeping finishes, unless the scheduler was stopped, so the reapers still finish before the first timer tick. - New `server/src/services/native-runtime/native-reconciliation-timing.ts`: records the phase boundaries of one pass. When the pass took at least one second and an OTLP endpoint is configured, it emits a `controller.native_reconciliation` root span (`paperclip.native-reconciliation`) with child spans `legacy_workspace_exports`, `obsolete_policy_reviews`, `automatic_completion_reviews`, `candidate_scan` and `candidates`, using the recorded start and end times. It records `scoped` and the candidate count only. A failed pass marks the pass span and the running phase. Tracing failures are swallowed. - `native-finalization-reconciler.ts`: the exported function wraps the pass in that timing and finishes it in `finally`, so failed passes are recorded; the original error is rethrown. - `doc/execution-semantics.md`: what readiness waits for, and the housekeeping order. `doc/observability.md`: the reconciliation spans. ## Verification - `pnpm exec vitest run src/__tests__/server-startup-feedback-export.test.ts src/services/native-runtime/native-reconciliation-timing.test.ts src/services/native-runtime/native-finalization-reconciler-timing.test.ts src/startup-recovery-timing.test.ts` (in `server/`): 52 passed. New tests: - Readiness is `ready` while a housekeeping step hangs, and the heartbeat interval starts only after the step settles. - A failed housekeeping step is logged, and the scheduler still starts after the remaining steps. - Span timing: nothing for a fast pass or without an endpoint; a slow pass emits the pass span and one child per phase with the recorded times; a failed pass marks the running phase and the pass; a failing tracer does not throw. - The reconciler records its phases and a failed outcome when a pass throws, and rethrows. - Three existing startup tests settle housekeeping before they read the scheduler interval or the setup cleanup. - `pnpm exec vitest run src/services/native-runtime`: 4,399 passed. - `pnpm typecheck` in `server/`: passed. ## Risks - Housekeeping now runs concurrently with the first requests and with `waitForExternalAdapters`. None of these steps is needed to serve requests, and each one also runs on every scheduler tick. - Readiness is reported earlier; the heartbeat timer starts at the same point relative to housekeeping as before. - The new spans exist only with an OTLP endpoint and only for passes of at least one second. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aa04c5c2a7 |
fix(runtime): keep the copyback lock transaction active (#16340)
Keep the dedicated workspace-copyback transaction active with bounded SELECT 1 probes while filesystem work runs. Preserve existing connection, owner-token, takeover and publication fences.
Verified head
|
||
|
DottaandPaperclip
|
cc96c9f832 |
fix(server): probe libc asynchronously before serving requests (#16335)
Linux runner qualification generated a full Node diagnostic report to read one libc version field. Reverse DNS on open sockets could stop the event loop for minutes. Replace report generation with a one-time asynchronous getconf probe during module initialization. Bound it to one second and 1 KiB of output, cache the measured facts, and preserve unknown-libc failure behavior. Verification: 149 focused tests, repository typecheck and build, full CI, and fresh Greptile 5/5 with no comments. A timed-out local heartbeat fixture passed when rerun alone. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
35b3345033 |
fix(server): support issue monitor deadlines in idle sleep reports (#16325)
## Thinking Path > - Paperclip manages work for companies and AI agents. > - Hosted workspaces can sleep when their required work has a safe wake path. > - The current idle check treats every open issue as background work. > - Backlog, unassigned and human-waiting records can therefore keep an idle process running. > - Future automated monitors still need their saved check time and execution claims. > - This PR adds a signed host capability for passive issues and verified monitor deadlines. > - The host can save one alarm for the earliest required routine or monitor time. ## Linked Issues or Issue Description **What happened?** A quiet workspace with a backlog or unassigned todo issue reports background work as present. A future issue monitor also prevents sleep even when the host can save a durable wake alarm. **Expected behavior** Proven passive records can sleep. A validated future monitor can sleep only after the host saves its alarm and validates the same owned report. Active, unknown and unresolved work stays awake. **Steps to reproduce** 1. Create a quiet workspace with an unassigned todo issue and no other work. 2. Acquire a bounded, signed Cloud idle hold. 3. Request the routine-only safety report with `scheduledWake=1`. The issue blocks sleep. 4. Request the new capability with `scheduledWake=2`. A passive issue can now report no work. 5. Add a valid future monitor to an assigned in-progress issue. The second capability returns its saved deadline for host acknowledgement. **Paperclip version or commit** Base: `c30fb7d2b75ea30f72d63656b2a7cfff7fe0929d`. **Deployment mode** Cloud-managed authenticated instance. Local scheduling remains in the existing scheduler. Related protocol: [#16215](https://github.com/paperclipai/paperclip/pull/16215). Open scheduler and monitor PRs were checked for overlap. This change leaves monitor admission, dispatch and quota recovery with their existing owners. The roadmap already includes scheduled work; this is a focused idle-safety extension. ## What Changed - Add signed scheduling capability `2`; capability `1` retains the routine-only rules. - Allow untouched onboarding prompts with idle assignees to sleep using the same awaiting-user predicate as recovery; typing, answering the opening card or a prior run removes the exemption. - Inspect passive issue state and future monitors in the same bounded, read-only snapshot as other work. - Check ownership, company scope, known states, claims, policy/deadline agreement, attempts, timeout and dependency repair. - Combine routine and monitor deadlines into a known proof kind and catalog generation. - Preserve other sleep blockers, including task watchdogs, conversations, live runs, external credentials and recovery. - Update the protected route, OpenAPI contract, documentation and PostgreSQL regression tests. ## Verification - Full workspace typecheck and build passed. Final server typecheck and full build passed after the final guards. - Initial safety, scheduling, protected-route and control-authentication checks: 326 tests passed. - Final safety and heartbeat scheduling checks: 253 tests passed. Includes reporting a future deadline, restoring scheduling from PostgreSQL and one dispatch across competing schedulers. - Local general server checks: 20,260 passed; UI: 8,460 passed; CLI: 534 passed, including isolated reruns of two embedded-PostgreSQL startup failures. All remaining workspace groups and 158 serialized suites passed. The suite was completed in groups after the isolated infrastructure retries; the final onboarding change also passed the focused 674-test recovery/safety run. Earlier unchanged Git fixtures failed under Xcode's system `init.defaultBranch=main`, reproduced on the clean base and passed with a test-scoped fixture default; no source or machine configuration was changed. - Rebased issue, protected-route and scheduler checks: 115 tests passed; server typecheck passed. - The expanded change passed 674 idle-safety, scheduling and recovery regression tests, including six onboarding admission/invalidation cases. Final server typecheck and build passed. - Current-head Apex is 5/5 with no unresolved threads. All 54 current-head checks passed, with two expected skips. Four jobs were interrupted by runner shutdowns or lost communication on their first attempts; their targeted reruns passed. Live hosting qualification awaits the verified master release and matching host capability. ## Risks This intentionally covers a limited set of passive states and valid future monitors. Assigned active tasks without a proved waiting path, unknown states, claimed monitors and quota recovery still block sleep. It does not make every open issue safe to sleep. A matching host must recognize `routines_and_issue_monitors`, acknowledge the durable alarm and revalidate the exact owner, generation and deadline before stopping. The report alone grants no stop authority. Older hosts retain capability `1`. Independent database writers remain outside the admission contract. Existing scheduler claims and execution authority are unchanged. No schema migration is required. Roll back host negotiation to capability `1` to restore conservative admission, while retaining alarm delivery until outstanding alarms have been handled. ## Model Used - OpenAI GPT-6 (Codex), with repository editing, command execution and GitHub tools. The exact deployment ID and context-window metadata are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
aec80cbc8c |
fix(server): keep workspace compaction out of the startup sweep (#16328)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When the server starts, `/api/health` answers "starting" until startup recovery finishes, and deploy tooling waits for "ok" > - Startup recovery waits for the first sweep of the terminal-workspace worker > - #16304 added workspace compaction to that sweep. Compaction walks worktrees, runs Git, and deletes dependency trees, and it re-checks every eligible workspace after a restart > - Compaction is never needed to serve requests, so it should not delay readiness > - This pull request skips compaction in the startup sweep only; later sweeps compact as before > - The benefit is that a compaction backlog cannot make a restart slower ## Linked Issues or Issue Description Follow-up to #16304. **What existing behavior does this improve?** Server startup time after #16304. **Current behavior** `await terminalWorkspaceWorker.ready` blocks the readiness flip until the first terminal-workspace sweep ends. That sweep compacts every eligible workspace in its page. The per-cycle compaction memo is in memory, so after each restart every eligible workspace is walked and checked with Git again, and any directories found are deleted before the sweep ends. **Proposed behavior** The first sweep skips compaction. Pending compactions keep their deadlines, so the worker wakes and a later sweep compacts them. **Reason and benefit** Readiness depends only on work that is needed to serve requests. On one measured instance the first sweep took 5–8 s with compaction; with a larger backlog, compaction could add seconds per workspace. **Breaking changes** None. Compaction happens one sweep later after a restart. ## What Changed - `sweepTerminalWorkspaces(limit, signal, { compact })`: `compact: false` skips compaction for that sweep and keeps each pending compaction deadline. - `registerTerminalWorkspaceWork` passes `compact: false` to its first sweep (the one readiness waits for) and `compact: true` after that. - `server/src/index.ts` forwards the option through the logging wrapper. - `doc/DEVELOPING.md`: the compaction section says the startup sweep never compacts. ## Verification - `pnpm exec vitest run src/__tests__/terminal-workspace-work.test.ts src/__tests__/execution-workspace-compaction.test.ts src/__tests__/workspace-compaction.test.ts` (in `server/`): 35 passed. - New: the worker passes `compact: false` only to the startup sweep and `compact: true` to the next one. - New: a sweep with `compact: false` leaves the files in place, the compaction deadline stays due, and the next sweep compacts. - `pnpm typecheck` in `server/`: passed. ## Risks - Low risk. After a restart, compaction runs a few seconds later than before. Archive behavior does not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
538bbd5478 |
fix(ui): configure Grok Bot through its external connection (#16338)
## Thinking Path Paperclip coordinates an external Grok Bot through its authenticated agent connection and routine. The shared configuration form only excluded OpenAI Dot from local runtime controls, so Grok Bot inherited a model picker, local paths, an AI account picker, and a generic runtime test. These controls cannot configure the external Bot and the generic test is not evidence of its webhook-to-MCP connection. Extend the existing external-provider presentation gates and retain the dedicated connection verification workflow. Related implementation: #16219 and #16305. Searched open Grok PRs and issues; the remaining matches concern the separate coding runtime or unrelated coverage. ## Linked Issues or Issue Description **What happened?** An existing Grok Bot agent's Runtime page shows local model and runtime configuration alongside its actual external connection controls. **Expected behavior** Identify the harness as Grok Bot, show its connection and routine controls, and use Verify connection for authenticated two-way readiness. Omit inapplicable local-runtime controls and model catalog queries. **Steps to reproduce** Create a Grok Bot through Invite external agent, then open its Runtime settings. Before this correction, the shared form renders a model selector and Test your agent despite the Bot owning its runtime. **Paperclip version/commit** Master after #16219; reproduced against the shared form at |
||
|
DottaandPaperclip
|
ebceb2a744 |
trace(server): time workspace, native setup and startup recovery (#16330)
## Thinking Path > - Paperclip manages agents and their work. > - The controller prepares workspaces before it starts a Runner. > - A slow run can spend 30–80 seconds inside the combined workspace phase. > - Startup recovery can also delay queued work for minutes. > - Native runs also have gaps before Runner startup and after the provider result. > - Existing traces do not identify the awaited steps inside those intervals. > - This change adds closed timing boundaries around the existing operations. > - Operators can locate the wait without changing execution or exposing content. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operator-configured OpenTelemetry diagnosis of controller preparation and startup recovery. Related: #16178, #16236 and #16285 introduced the existing preparation timing and sampling contract. #16306 and #16312 reduce separate configuration and copyback work. **Current behavior** The workspace interval combines source resolution, trust checks, provisioning, database persistence, binding and repository admission. Startup awaits accounting and native recovery before queued work resumes, but neither has a separate duration. Native backend setup and post-turn cleanup also contain unmeasured awaits. CPU profiles and a fast database probe cannot identify which await is slow. **Proposed behavior** Record workspace steps and native controller setup and cleanup with the existing hashed run ID. Record startup phases as independent, operator-sampled root spans without custom attributes. Keep the same operation order, results, failures and authorization. **Reason and benefit** These boundaries distinguish source work, database writes, repository admission and restart recovery. This is a diagnosis change. It does not claim a response-time improvement or a solved latency budget. ## What Changed - Add 14 closed workspace step names around 16 existing awaits. Repeated updates and bindings produce separate spans. - Time startup tool reviews, gateway sessions, delivery-worker readiness, accounting, plugin readiness, native recovery, hot restart, orphan reaping, retries, queued work and goal recovery. - Time native GitHub setup, wake binding, backend construction, connector assignments, MCP discovery, tool authority, companion selection and binary resolution. Time selected post-turn GitHub, coordinator and instruction cleanup awaits. - Move preparation timing to a pure shared module. Keep the existing heartbeat import as an alias. Both heartbeat and native timing skip tracer lookup when the endpoint is unset or blank, including when another library registered a provider. - Tracing lookup, callback, status and end faults cannot repeat work or replace its value or error. - Add real PostgreSQL workspace timing coverage and silent no-op/fault controls. Update the tracing contract and paired run maps. - Keep span attributes closed. Startup spans add none; workspace steps use only the existing run hash. No commands, paths, returned records or exception messages are recorded. ## Verification - Diagnostic-gap baseline: the new real workspace test fails against the unchanged workspace implementation because the expected step spans are absent. It passes with the wrappers. - Final affected coverage: **804 distinct cases pass** across seven complete suites: 698 native-executor cases, 39 real PostgreSQL workspace cases, 61 timing/context/startup cases, and six execution-configuration integration cases. The final native and unit invocations pass with no selected skips. The workspace suite passed in the initial six-file invocation on the same final workspace source. - Preserve the initial six-file attempt as failed: **7 failed and 789 passed**. All seven errors occurred during embedded PostgreSQL startup. The log reports local macOS shared-memory exhaustion. A fresh 15-case database group passes, followed by the complete 698-case native suite. These runs overlap; do not add their counts. - The new native timing assertion fails against unchanged native code because the expected spans are absent. It passes with the wrappers and retains the credential-isolation assertions. - Full `pnpm -r typecheck` and `pnpm build`: passed on the final production source. Earlier nullable-closure errors are fixed with erased assertions inside existing guards. The failed attempts remain recorded. - Normalized AST comparison: removing only timing wrappers, timing imports/local handles and erased assertions restores the original workspace, startup and native executor programs. This checks call order and policy expressions; it is not live acceptance. - The expanded head first failed two configuration integration assertions because their test providers had no endpoint. Configure the endpoint explicitly in those two tracing tests, and restore it after each test. All six cases then pass with the original credential cleanup, redaction and exact span assertions. Production is unchanged by this repair. - The first PR head failed one unchanged attachment test at its 15-second deadline in CI. The complete 32-case attachment suite passes locally with that deadline. Do not label the failed CI attempt green. The expanded head needs fresh full CI. - `node scripts/check-maps.mjs`: passed, 35 areas and 176 capability IDs. `node scripts/storybook-catalog.mjs`: 283 files and 2,728 stories. Structural checks only; no browser, play or Storybook build claim. - `git diff --check`: passed. No added prose comments, lockfile changes or new 1,000-line crossings. - A duplicate full local test run was not started. Fresh exact-head full CI and E2E pass on `15a683bb2419960f611584f8c07cdcfc9ce3af0b`. All 56 check runs are successful or optional skipped. Superagent and legacy Snyk pass. Completed Apex review is 5/5 on this same head, with zero unresolved review threads. Commands: ```sh pnpm exec vitest run --project @paperclipai/server --maxWorkers=1 --fileParallelism=false server/src/services/native-runtime/native-session-executor.test.ts server/src/services/execution-preparation-timing.test.ts server/src/startup-recovery-timing.test.ts server/src/services/heartbeat/preparation-timing.test.ts server/src/services/heartbeat/preparation-timing-context.test.ts server/src/services/heartbeat/workspace-preparation.test.ts pnpm -r typecheck pnpm build node scripts/check-maps.mjs node scripts/storybook-catalog.mjs ``` ## Risks - Configured operators receive more spans. They still follow the configured sampler and exporter. - Startup spans measure the awaited phase, including remaining readiness waits. They do not measure earlier worker construction or a provider turn. - Workspace and native setup intervals overlap their containing phases. Do not add them to the parent duration. Native timing covers named awaits, not every statement or every error path. - These spans are not yet deployed or tested against a live latency run. No first-party Telemetry event, run-log write, schema or API changes are included. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact served variant and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
445feaa7aa |
fix(server): index retained workspace task authorization (#16334)
## Thinking Path > - Paperclip manages AI agents and their work. > - Before an agent starts, repository admission checks access to retained workspace files. > - Admission holds a company provenance lock while it checks each source and reader. > - The retained-task query cast the indexed task ID to text. PostgreSQL could evaluate privacy across company history before it matched one source ID. > - Repeated slow checks kept the lock for hours and blocked other agent launches. > - This change looks up the canonical source UUID through the existing primary key before it checks privacy. > - The query does less work and keeps the same access decisions. ## Linked Issues or Issue Description **What happened?** Nine runs appeared active but had no provider process or log file. Read-only database inspection found a transaction holding the company repository-provenance lock for more than ten hours. Eight other transactions waited for that lock. The owner repeatedly ran workspace authorization queries. The company had about 13,400 tasks and 8,500 workspaces. **Expected behavior** A retained-source check should find the referenced task before it evaluates that task's privacy. Unrelated task history should not multiply the privacy work for every source and reader. **Steps to reproduce** 1. Run the new `workspace-authorization-query.test.ts` against authorization code from `5c7a025032`. 2. The real PostgreSQL fixture has 1,200 unrelated tasks and one retained source. The regression fails with 2,402 recursive privacy traversals, above the bound of 100. The other five access tests pass. 3. Apply this change and run the same test. All six tests pass. Related: Refs #16330 adds workspace timing evidence. This PR fixes a measured SQL plan. It does not change those tracing boundaries. ## What Changed - Compare the UUID task column with a guarded canonical UUID from the retained-source JSON key. - Reuse `canonicalExecutionUuid`. Invalid and noncanonical keys still match nothing and cannot cause a UUID parse error. - Add real PostgreSQL coverage for query work, missing and foreign sources, malformed and uppercase IDs, both agent and sponsor grants, and grant revocation. - Add a reproducible baseline comparison and document its scope in `doc/DATABASE.md`. ## Verification - Red: the unchanged query fails the new work-bound regression. Five access cases pass. - Green: 101 tests pass across the new query suite, workspace binding, and the production privacy review suite. - Full `pnpm -r typecheck` and `pnpm build`: pass. Dependencies use the repository's pinned pnpm 9.15.4. - All current-head CI checks pass, including the full server, chat, workspace, serialized, Runner, and eight E2E test lanes. Build, typecheck, canary, and security checks pass. Head: `ad907f6e14c2d773fa8c20692e5ccc10215c9297`. - Greptile: 5/5 on that head, with no actionable findings or unresolved review threads. The branch is mergeable. - The local full `pnpm test:run` was started, then stopped after a concurrent disposable-database check hit the macOS shared-memory limit. This is not counted as a local full-suite pass. The benchmark passed when rerun alone; the complete suite passed in CI. - `git diff --check`: pass. The branch has no lockfile or workflow changes. Synthetic SQL comparison on PostgreSQL 18.1, with 13,429 tasks and one retained source: | Query | Median | Five measured samples | Recursive privacy traversals | | --- | --- | --- | --- | | Baseline `5c7a025032` | 42.57 ms | 42.57, 47.71, 44.51, 41.29, 41.44 ms | 26,860 | | Fixed | 2.37 ms | 2.76, 2.35, 2.37, 2.11, 3.33 ms | 4 | Both versions return identical complete rows. The benchmark alternates variants after two warmup rounds. It loads the baseline authorization module with current dependencies; the UUID helper is unchanged. The measured gain is about 18x for this SQL fixture. The final committed benchmark also passed a second isolated run: 55.44 ms baseline and 3.26 ms fixed, with identical complete rows (about 17x). Two read-only queries on the affected database also returned identical complete rows. One changed from 129 ms to 7 ms; a workspace with 49 retained sources changed from 576 ms to 11 ms. These are single paired measurements, not endpoint or startup benchmarks. Reproduce: ```sh pnpm exec vitest run server/src/__tests__/workspace-authorization-query.test.ts server/src/__tests__/task-workspace-binding.test.ts server/src/__tests__/privacy-production-review.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-authorization.ts --baseline-ref=5c7a025032 pnpm -r typecheck pnpm test:run pnpm build ``` ## Risks - A direct UUID cast could broaden or break access for malformed keys. The existing canonical guard and regression cases preserve the former text-match behavior. - Planner choices can differ across PostgreSQL versions and data sets. The regression bounds real privacy work rather than elapsed time or one exact plan shape. - Admission still checks every retained source and reader under its existing lock. This removes a measured query cost; it does not claim a bound for the full admission process. - No schema migration, permission change, lock removal, or live deployment is included. Recovery of the affected runs remains unverified until rollout. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact served model variant and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b226336762 |
ci: skip full CI for release-notes-only pull requests (#16207)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each beta release ends with a pull request that adds one release notes file under `releases/` > - PR CI runs the full build, test, and e2e matrix for that pull request > - No build, test, or e2e lane reads `releases/v*.md` or `releases/beta/v*.md` > - The full matrix takes about 30 minutes, and lost runners or flaky e2e specs can make it run again > - This pull request lets the existing CI scope gate skip the heavy lanes when a pull request changes only release notes > - The benefit is that release notes pull requests get a green result in minutes, and code pull requests keep full CI ## Linked Issues or Issue Description **Issue type** CI efficiency enhancement. **Where is the issue?** The `Select stacked PR CI scope` step in `.github/workflows/ci-routing.yml`. This step sets the `full_ci` output for PR CI. **What's wrong?** The step sets `full_ci=false` only for middle pull requests in a stack. A pull request that changes only one release notes markdown file runs the full matrix. For example, #16053 changed one file in `releases/beta/`. Its CI ran two times: on the first run, 8 shards lost their runners, and on the second run, one Playwright spec flaked and needed a rerun. **Suggested fix** Read the pull request diff from the GitHub compare API in the scope step. When every changed path is release notes, set `full_ci=false`. The callers already support this value. I found no open pull request or issue for this change. ## What Changed - `ci-routing.yml`: the scope step (renamed to `Select PR CI scope`) also sets `full_ci=false` when every path in the diff matches `releases/v*.md` or `releases/beta/v*.md`. - The step reads the diff from `compare/<event base SHA>...<event head SHA>`. The SHAs come from the event, so a later push cannot change the decision for this run. - For a renamed file, the step checks the old path and the new path. - The step keeps full CI when the event is not `pull_request`, a SHA is malformed, the API call fails, the diff is empty, or the file list has 300 entries. The compare API can truncate a list at 300 entries. - The step does not change the stacked pull request rules. It checks the paths only when the stack rules select full CI. - `ci-routing.test.mjs`: the GitHub stub now returns `files` for compare calls. 17 new cases cover release notes paths, mixed diffs, renames, non-markdown files, nested folders, empty and truncated lists, API failure, non-PR events, malformed SHAs, and stack interaction. - `e2e-shard.test.mjs`: the test now uses the new step name. - `doc/DEVELOPING.md`: added a short description of the CI scope rules. ## Verification - `node --test .github/scripts/tests/ci-routing.test.mjs scripts/__tests__/e2e-shard.test.mjs`: 73 pass, 0 fail. - `node --test '.github/scripts/tests/*.test.mjs'`: all pass. - I ran the jq filter on real compare API responses. #16053 (one file in `releases/beta/`) returns `true`. #16082 (10 files) returns `false`. - After merge: open a pull request that changes only a `releases/beta/v*.md` file. The `Select trusted runner` job shows the notice `release notes only; full_ci=false`. The heavy lanes show as skipped. The `policy`, `verify`, and `e2e` checks pass. Note: `pr.yml` calls `pr-trusted.yml@master`, and that workflow calls `ci-routing.yml@master`. This pull request does not change its own CI run. The change applies after merge. ## Risks - Low risk. The skip uses the existing `full_ci=false` path. The `verify` and `e2e` aggregate checks already accept that value, and the `policy` job always runs. - A future job that reads release notes files would not run for notes-only pull requests. Today, only the release workflows read these files, and those workflows do not run on pull requests. - The step now uses `GH_TOKEN` to call one more read-only API endpoint. The workflow still does not check out or run pull request code, and its permissions do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Agentic coding with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
82884443be |
perf(adapter-utils): skip host scans for unchanged restores (#16312)
Skip the host snapshot scan when an unchanged workspace return matches its baseline, while retaining writer locking, both authority hooks, and changed-export conflict handling. Verified with 270 affected tests, final-head CI/E2E, Superagent, and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
51b56706fb |
fix(grok): verify file completion and recover failed sessions (#16305)
## Thinking Path > - Paperclip manages work and authority for AI agents. > - Grok Bot runs outside Paperclip and uses authenticated upload grants for files. > - Native completion requires a verified file publication receipt. > - The upload saved the file but did not create that receipt. Real Grok work could not finish. > - A failed run also left an assignment that the Stop action could not fence. > - This pull request connects verified uploads to normal completion and repairs that recovery action. > - The result preserves file integrity and current work-session authority. ## Linked Issues or Issue Description Refs: #16219, #16274. **What happened?** A real Grok Bot saved a task document and CSV. Semantic completion rejected the CSV because its upload had no verified publication receipt. After the managed run failed, Stop rejected its remaining assignment as a changed session. The driver also discarded the remote monitor's known failure reason. **Expected behavior** An authenticated upload of the exact requested bytes can satisfy normal completion. Stop can fence the remaining assignment of the selected failed run. An old Stop action cannot stop newer work. Known remote lease and reconnect errors remain visible without raw diagnostics or credentials. **Steps to reproduce** 1. Connect an experimental Grok Bot agent and verify its routine. 2. Assign a task that requires a document and a file. 3. Upload the file through its accepted assignment and cite the attachment in semantic completion. 4. Observe the missing publication receipt error on the previous implementation. 5. For recovery, select Stop after that run fails but its assignment remains active. **Paperclip version or commit** The live defects were observed after #16274. The current candidate merges master `739f9e40a4` and is `26c5b9601d1cb2b8f5bb7fcf730f019523763f5b`. ## What Changed - Record a server-owned publication receipt in the verified upload transaction. Bind it to the real native Grok run and exact stored file metadata. - Validate external publication against the actual upload and admitted assignment. Keep existing native and person upload rules. - Recover the committed receipt after a lost response. Reverify the same bytes when an older upload lacks proof. - Hold assignment authority through publication with grant-before-assignment and task-before-run lock order. Disconnect does not wait for remote file storage. - Allow Stop to fence the exact lingering assignment of a terminal run. Reject stale Stop calls for successor work. - Preserve only known lease/reconnect failure codes from the remote monitor. Do not expose raw stderr. - Extend real Rust, overlapping document/upload and disconnect/upload, file retry, completion and Stop regressions. Document setup, files and recovery. ## Verification - The real Rust Grok regression requests a file, checks an overlapping native document save, concurrent retries and lost-response recovery, rejects missing delivery evidence, then uses normal semantic completion and finalization. - Focused tests cover person MCP, native file handoff, completion, Stop and both external driver providers. - Local repository typecheck and production build pass on the current candidate. The five focused protocol suites pass (264 tests); all 17 external Runner driver tests pass, with the driver unchanged since the master merge. - Token gates and Storybook build pass. Nine existing Grok fixture journeys pass their browser interaction assertions; the static probe separately records five fixture API routes returning 404. This is UI fixture evidence only. - Full repository coverage passed in CI. The duplicate local `pnpm test:run` was interrupted after CI completed; its partial log is retained and is not reported as a local full-suite pass. The earlier aggregate invocation was interrupted for the second review fix. - All CI gates pass on `26c5b9601d1cb2b8f5bb7fcf730f019523763f5b`: 55 successful checks and two skipped Storybook jobs (no UI diff). This includes the complete sharded repository suites, native Runner, release checks, build and security scans. - Current-head Greptile is 5/5 with zero unresolved findings. The PR has no merge conflicts. - The prior live attempt is failed acceptance evidence. Real Grok requalification, proactive work, controller restart and active-session Stop remain release gates. This PR does not claim those passed. ## Risks - Upload and completion now share a durable run receipt. Lock order and current assignment authority must remain consistent with native mutations. - Publication fails closed if the file, run or grant changes. A retry must keep the same upload identity and bytes. - Person connections gain no agent authority. No database migration is required. - Known exit diagnostics do not prove global Grok termination. Grok spending remains unreported. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution and computer use. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ad0bd26bb7 |
perf(heartbeat): avoid Git inspection for stored workspace reads (#16306)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat selects and restores a saved execution workspace before dispatch. > - These reads used the workspace detail reader, which also scans Git and resolves delivery state. > - Configuration reads do not use those live display fields. Each extra process can delay other controller work. > - This pull request adds a stored-record reader and uses it in configuration, environment selection, and finalization. > - It also derives ancestry from the existing ahead/behind result and skips delivery HEAD reads for unfinished tasks. > - The benefit is fewer Git processes while live execution and cleanup guards remain in place. ## Linked Issues or Issue Description Related: #16282, #16302, #13619. Those changes reduce background or display scan demand. This change removes live display inspection from execution configuration reads. **What existing behavior does this improve?** Heartbeat workspace configuration reads and workspace close-readiness checks. **Current behavior** Reading saved workspace configuration also loads runtime health, Git status, ancestry, and delivery state. Close readiness walks history twice to determine ancestry. **Proposed behavior** Execution reads stored configuration through a company-checked method. Live branch validation and destructive cleanup still inspect current Git state. Close readiness uses its ahead/behind result to determine ancestry. **Reason and benefit** A synthetic benchmark of the actual service returned identical stored fields for 20 workspaces. Detail reads launched 60 Git commands and took 689 ms. Stored reads launched zero Git commands and took 8 ms. This was a local macOS measurement, not an end-to-end cloud latency claim. **Breaking changes** None to public APIs or saved data. Workspace detail responses keep their live fields. ## What Changed - Add `getStoredById(companyId, id)` and use it in three heartbeat reads. - Preserve the access-denied error for a foreign-company workspace. - Remove the extra `merge-base` process and the unused HEAD read for unfinished tasks. - Add recovery, company-boundary, ancestry, and cleanup regressions. - Add a repeatable database/Git benchmark and document the read boundary. ## Verification - The new environment-selection regression failed before the fix with `Live Git inspection is unavailable` and passes after it. All eight environment-selection tests pass. - On the final rebased commit, workspace service, environment selection, and branch finalization: 89 tests passed. Branch-containment, runtime-execution, and display-cache suites also passed during development. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: started locally, then stopped after all full-suite CI test shards passed on the final commit. The complete local run was not finished; CI provides the full-suite result. - Final commit `9a13ef0481b7bf6a60b2b7ac13f35f78a14e232c`: 55 successful checks and 2 expected skips. Greptile: 5/5, no actionable findings. - `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-record-reads.ts`: identical stored fields, 60 to zero Git commands, 689 ms to 8 ms for 20 reads. ## Risks - Execution callers must use saved fields only. Detail readers and live branch and cleanup guards remain separate. - A failed ancestry comparison stays unknown. Automatic cleanup still rejects unknown delivery state and rechecks Git before deletion. - This patch has not been deployed. It does not prove the full slow-run latency is resolved. ## Model Used OpenAI Codex, GPT-6. The exact model identifier and context window are not exposed in this session. Used reasoning, repository inspection, code editing, and local command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4460785534 |
fix(skills): preserve referenced files in large repository imports (#16308)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use skills imported from project repositories.
> - Large repositories fall back to isolated skill folders.
> - That fallback drops shared files and turns advisory links into
import failures.
> - This pull request retains bounded referenced files and keeps missing
references advisory.
> - Users can import valid skills such as test-audit without editing the
source repository.
## Linked Issues or Issue Description
Refs #16066. Refs #16256. Refs #16288. Related draft #15730 adds
optional reference selection to individual skill packages. This fix
applies automatic repository fallback and does not add that selection
UI.
**What happened?**
A project source showed 86 warnings. It rejected 23 skills solely
because references were absent from the isolated package. It also
treated web routes and inline Markdown examples as files. For
test-audit, repository scripts were resolved inside the skill folder.
**Expected behavior**
Retain referenced files that fit the package limits. Missing references
must not block an otherwise valid selected skill. Preserve content
validation and installed shared files.
**Steps to reproduce**
Import paperclipai/paperclip at commit
|
||
|
Devin FoleyandPaperclip
|
5b29b9062e |
fix(server): compact finished task worktrees and shorten the reaper cooldown (#16304)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each task can run in its own isolated Git worktree, and each worktree installs its own dependencies > - The terminal-workspace reaper removes a worktree only when the task tree is finished, the delivery is merged, the files are clean, and a 7-day cooldown has passed > - A recurring task that runs every hour keeps a week of worktrees, and each worktree holds gigabytes of `node_modules`. A finished task with no merged pull request, or with leftover agent files, keeps its worktree forever > - On a busy instance this fills the disk in days, and a larger volume only delays the next failure > - This pull request adds compaction: one hour after a task tree finishes, the reaper removes regenerable directories from the worktree and keeps the workspace, the branch, and all source changes > - The benefit is that finished tasks stop holding gigabytes of disk, and no work is lost because a resumed task regenerates the files when its workspace is provisioned again ## Linked Issues or Issue Description Refs #2022 (retention for workspace cleanup). Related: #11936 and #14608 add external reclaim scripts (a scheduled tool and a disk-guard step). This pull request does the reclaim inside the server's terminal-workspace reaper instead. That way it uses the reaper's lifecycle lock, issue-tree checks, and live-run checks, and needs no extra scheduler. **What existing behavior does this improve?** Disk retention of execution workspaces after their tasks finish. **Current behavior** A finished task keeps its full worktree, including installed dependencies and build output, for at least 7 days. If the delivery is not merged, or the worktree has uncommitted or untracked files, the reaper never removes it. On one instance, an hourly recurring task created about 22 worktrees per day at about 3 GB each (about 2.8 GB of that was `node_modules`). Together with finished tasks that had no merged pull request, they filled a 100 GB volume in about two days. **Proposed behavior** One hour after a task tree becomes terminal, the reaper removes regenerable directories (installed dependencies, build output, tool caches) from the worktree. It keeps the worktree, the branch, the commits, and every tracked, untracked, or other ignored file. When the task resumes, the existing reuse path runs the provision step, which reinstalls the dependencies. Clean, merged worktrees are archived after 1 day instead of 7. **Reason and benefit** Most of a finished worktree's size is regenerable. Compaction removes that part without the risk of deleting work, so it can apply to workspaces that the reaper must keep (no merged pull request, leftover files). The shorter cooldown limits the cost of the source checkouts that stay. **Breaking changes** None. The default archive cooldown changes from 7 days to 1 day. `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS` still overrides it. `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. ## What Changed - New `server/src/services/workspace-compaction.ts`: - Finds directories with a regenerable name (`node_modules`, `dist`, `build`, `coverage`, `storybook-static`, `target`, `.cache`, `.next`, `.nuxt`, `.parcel-cache`, `.svelte-kit`, `.turbo`, `.vite`). - Keeps only the directories that the repository's own `.gitignore` files ignore and that hold no tracked file. A negated pattern (`!build/`), a global excludes file, and `.git/info/exclude` do not count. Each candidate is checked per path, so dependencies below an ignored parent (`vendor/tool/node_modules`) are found. - Searches inside a regenerable-named directory that does not qualify (for example a tracked `build/`), so its ignored dependencies still go. - Never enters `.git` or `.paperclip` (the worktree instance data), and never follows a symlink. - Each Git read has a 60-second deadline and stops when the sweep is aborted. - Moves the directories into `.paperclip-compaction-trash` beside the worktree with a same-filesystem rename, and can restore them. - `sweepTerminalWorkspaces` compacts each terminal workspace once per terminal cycle, after the compaction delay and before the archive cooldown check: - Only isolated linked Git worktrees qualify. A shared workspace, a project's main checkout, or a row whose `.git` is a directory is never compacted. - The move runs inside a transaction that holds the workspace lifecycle lock. The guard checks a fresh row and the same lifecycle generation. It also checks that the issue tree is terminal, that no queued or running run works on the tree or the workspace (by checkout or execution run, or by run context), and that no runtime service runs from the workspace. - The guard runs before the move and again after it. If the second check fails (for example, the task reopened or a run started), the directories move back. Otherwise the trash is deleted. - The worker wakes at the compaction deadline, not only at the archive deadline. - Stale trash from an interrupted compaction or a failed delete is purged by later sweeps. The sweep checks each worktree parent at most every 30 minutes, whether or not anything is left to compact. - New counters `compacted` and `compactionFailed` in the sweep result. The activity log gets an `execution_workspace.regenerable_files_removed` entry with the removed paths. - Config: new `PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS` (default `1`) and `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED`. The default for `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS` changes from `7` to `1`. Both new keys are added to the run-failure diagnostics allowlists. - Docs: `doc/DEVELOPING.md` documents the new variables and has a new "Terminal workspace compaction" section. `doc/SPEC-implementation.md` describes compaction. - `execution-workspaces.ts` was already above 1,000 lines. The file-system logic is in the new module. Only the guarded orchestration, which uses the existing lifecycle lock and run predicates, is in the service. ## Verification - `pnpm exec vitest run src/__tests__/workspace-compaction.test.ts src/__tests__/execution-workspace-compaction.test.ts` (in `server/`): 27 passed. `workspace-reaper-cooldown-config.test.ts`: passed. - Unit tests: directory selection (ignored, untracked, regenerable name only, `.env` and local data kept, tracked `coverage` kept, `.paperclip` skipped, symlinks not followed), nested candidates under a source `build/` and under an ignored `vendor/`, a negated `!build/` kept, global and `info/exclude` rules ignored, glob characters in names, abort, main checkout vs linked worktree, move and restore, restore that keeps a recreated directory, and stale-trash purge. - Embedded Postgres tests: - A no-PR, unmerged, dirty workspace is compacted. HEAD, the branch ref, `git status --porcelain --untracked-files=all`, untracked notes, `.env`, local data, and `.paperclip` data stay the same, and the row stays active. - A delivered workspace inside the cooldown is compacted. - The delay is respected, and the worker wakes at the delay. - An active checkout run, a queued run found through its context, an unfinished child issue, and a running runtime service all block compaction. - The task reopens during the move: the files are restored and unchanged. - A run starts during the move: the files are restored. - A shared workspace and a main checkout are never compacted. - Compaction runs once per terminal cycle and again after a later cycle. - Disabled compaction does nothing. - End to end: `realizeExecutionWorkspace` creates a worktree, the sweep compacts it, and a second `realizeExecutionWorkspace` reuses the same worktree. The provision command regenerates `node_modules`, and the untracked notes and the unmerged commit are still present. - Leftover trash is purged when the workspace has nothing left to compact. - Mutation checks: when the post-move guard is disabled, both race tests fail. Without the negation fix or the repository-only source fix, their tests fail. - `pnpm exec vitest run` for `execution-workspaces-service.test.ts`, `terminal-workspace-work.test.ts`, `execution-workspace-reopen.test.ts`, `server-startup-feedback-export.test.ts`, and `workspace-runtime.test.ts`: 335 passed. Two `workspace-runtime.test.ts` cases fail locally with and without this change (`does not diagnose a missing remote branch as a rejected credential`, `rejects explicit main on a master-only remote ...`). They also fail on unmodified `master` on the same machine, so the cause is the local Git environment. - `pnpm typecheck` in `server/`: passed. `pnpm check:module-boundaries`: passed. ## Risks - A repository can keep something that is not regenerable in an ignored `dist`, `build`, `target`, or `.cache` directory. Compaction removes it. The fixed name list, the ignore check, and the tracked-file check limit this. `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. - A repository without a provision command does not get dependencies back automatically. This is the same state as a new worktree for that repository, where the agent already installs them. - A resumed task pays for a dependency install again. This happens only for tasks that finished more than an hour earlier. - The shorter default cooldown removes clean, merged worktrees after 1 day instead of 7. An operator who wants the old hold can set `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS=7`. - Self-hosted instances get the same behavior. There is no schema change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). A separate Claude agent read the reaper code during the diagnosis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
adb69e8f09 |
feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683)
## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto | paperclip | legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. **Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass.** The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board **Try again** and **Resume** recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal **Approve completion** reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
21ee5c9c85 |
feat(sentry): add opt-in run failure reporting with saved run references (#16300)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs retain execution results and output in the instance. > - Operators can report run failures to Sentry for investigation. > - A backend Sentry DSN currently enables those reports with no separate choice. > - A repair bot also needs the exact instance, company, agent, and saved run. > - This pull request adds an explicit reporting mode and authenticated run references. > - The benefit is that operators can investigate every failure on selected deployments while keeping other deployments quiet. ## Linked Issues or Issue Description **What existing behavior does this improve?** Optional server reporting of terminal agent run failures to Sentry. **Current behavior** A backend DSN enables run failure reports. Known account, quota, and connection failures are excluded. Reports omit company and agent references. Runs without a task are skipped. Tool-name collision errors omit the conflicting bindings. **Proposed behavior** Set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain the existing filters. Set it to `all` to report every failed or timed-out run, including known setup and quota failures. The default is `off`. Expected stops stay excluded. Each report supplies identifiers and API paths that a bot can use with its own authorized Paperclip connection. **Reason and benefit** Operators can enable broad failure reporting on one staging deployment. A bot can fetch the saved run, events, and logs instead of relying on a short error label or copying complete transcripts into Sentry. **Breaking changes** A backend DSN alone no longer enables run failure reports. Existing operators must set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected` to retain the previous policy. Ordinary server exception capture remains enabled by the DSN. Failure grouping gains the phase and original cause code when available. Refs #16281, #16297, #16298. These cover related tool discovery and failure diagnostics. This change retains their existing boundaries and does not depend on their open changes. ## What Changed - Add `off`, `unexpected`, and `all` modes. Missing and invalid values keep reporting off before diagnostic reads. - Keep the Sentry backend DSN and audited optional SDK requirements. - Include company, agent, run, and optional task identifiers. Include versioned paths for the saved run, events, log, and task. Add absolute run URLs only for a valid public HTTP(S) origin. - Report company runs without a task. Prefer the first-class issue binding when selecting the task reference. - Select saved native provider/model settings, timestamps, and original failure codes. Preserve gateway reason codes through redacted exception projection. - Group by error code, adapter, and available phase/cause. Keep run IDs and error text out of grouping. - Include the tool names and connection bindings in collision errors without copying tool schemas or provider metadata. - Document configuration, upgrade behavior, bot access, redaction, retention, and best-effort delivery. ## Verification - All 56 checks passed for `cccb47a721eaa1cc36e63278fc177b9101477541`. Two unrelated Storybook jobs were skipped by their path filters. No checks remain pending. - Complete [Paperclip CI run](https://github.com/paperclipai/paperclip/actions/runs/38591422284): passed. This covers general and serialized tests, Runner checks, browser E2E shards, typecheck, build, release-registry checks, canary dry run, and Docker checks. - Greptile reviewed that exact head: 5/5, with no unresolved threads or actionable findings. Superagent, Socket, and Snyk checks passed. - Local focused reporting, diagnostics, collision, and real SDK contract tests: 410 passed across five suites. This includes the external-wake filter in both reporting modes. - The audited `@sentry/node@10.71.0` tests use an in-memory transport. No events were sent to Sentry. - `pnpm -r typecheck` and `pnpm build`: passed locally after the final rebase. - `node scripts/check-module-boundaries.mjs` and `git diff --check`: passed. - The full local `pnpm test:run` command was stopped to avoid duplicating the complete CI shards. It did not finish locally. The full suite passed in CI. ## Risks - The new default disables existing run failure exports until the operator selects a mode. The upgrade setting is documented. - `all` can increase event volume and include known user-action failures. Secret redaction and diagnostic limits still apply. - Reporting remains asynchronous and best effort. This PR adds no durable delivery queue and does not install a bot or Sentry alert rule. - API references grant no authority. Existing company, private-task, and run-output checks still apply. Complete transcripts stay in Paperclip. - This changes optional Sentry error reporting. It changes neither first-party Telemetry nor OpenTelemetry span exports. It adds no database migration. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, shell, file editing, and test execution. The exact backend deployment ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
74ba530b9e |
fix(workspaces): keep cleanup Git scans fresh and dedupe page reads (#16302)
## Thinking Path > - Paperclip manages execution workspaces for agent tasks. > - It must preserve work when it closes a workspace. > - The terminal-workspace worker checks Git before it decides whether a workspace can close. > - A request for fresh Git status could join a scan that started before a file changed. > - Several records for one directory can also repeat the same Git work in a bounded page. > - The worker already reads the task tree before Git, then reads it again for delivery assessment. > - This pull request fixes the fresh-read boundary and removes those repeated reads. ## Linked Issues or Issue Description - Refs #14759 and #14423 for the background Git scan load. - Refs #16282 for the merged event-driven terminal-workspace worker. - Refs #13619 for the original community investigation and tests. Thanks to @fayonation for identifying the shared-directory and in-flight scan cases. This implementation starts from current master and adapts those ideas to the new worker. ## What Changed - Give correctness-sensitive Git status requests their own scheduler scan. They skip completed cache entries and older scans in progress. - Share compatible Git readiness results across workspace records in one bounded worker page. The key includes company, path, branch, base ref, and runtime ownership. Each record keeps its own task and delivery assessment. - Reuse the task tree already read by the worker during delivery assessment. - Add regression tests for a file created during an older scan, compatible scan sharing, and company and branch separation. Update developer and workspace map documentation. ## Verification - `pnpm exec vitest run server/src/services/workspace-git-operation-scheduler.test.ts server/src/__tests__/execution-workspaces-service.test.ts` — 92 tests passed. - `pnpm -r typecheck` — passed with the repository's pnpm 9.15.4. - `node scripts/check-maps.mjs` — passed. - `pnpm build` — passed. - The full local `pnpm test:run` was interrupted when its original worktree disappeared. All PR test shards passed in GitHub CI. - Latest commit `e7622127c7d978a9a90857112675834725965d72`: 53 successful PR checks, 2 expected skips, and no failures. Greptile reviewed this commit at 5/5 with no open findings. ## Risks - A shared page inspection can become stale if the directory changes before the next record is assessed. The final cleanup still starts a separate Git scan and checks current task, ownership, and lifecycle state under the existing guards. - The new scan key changes request coalescing for correctness-sensitive callers. They still use the scheduler's concurrency and queue limits. There is no database migration or API change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code editing, and shell tools. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd64eb27b0 |
perf(recovery): use indexed execution identity lookups (#16291)
## Thinking Path > - Paperclip manages AI agents and their execution. > - Recovery must keep old execution ownership visible before new work starts. > - Ownership reads filter large run contexts through a combined text expression. > - Recovery and status delivery also cast indexed UUID columns to text. > - These reads can scan history before finding the requested execution. > - This pull request preserves identity and ownership rules while using existing indexes. > - The benefit is less database work during recovery and task polling. ## Linked Issues or Issue Description Refs #14406. Related community report: #15275. This is an independent implementation. It does not use that pull request's code or commits. It covers the reported execution joins and the additional ownership query hotspot. **What happened?** Accumulated staging query statistics showed 11,782 ownership reads at 51 ms each. A paired recovery-action read averaged 22 ms. Both queried large run JSON. Text casts also prevented indexed UUID lookups in recovery and status delivery. **What did you expect to happen?** Recovery must find the same execution evidence through existing indexes. Malformed identifiers must match nothing without a cast error. Native issue bindings must retain precedence. Missing termination evidence must continue to block new work. **Steps to reproduce** Run the disposable benchmark below. It loads the original service modules from the baseline revision. It compares their exact ownership, blocker action and status-delivery SQL with the current modules on 1,400 synthetic runs, 350 historical recovery actions and 192 KiB contexts. ## What Changed - Convert saved text to UUID only after checking its canonical lowercase form. - Keep indexed run and task UUID columns unchanged in execution joins. - Split native and legacy issue predicates so existing indexes can filter ownership candidates. - Preserve company scope, native precedence, process and lease holds, recovery eligibility and delivery receipts. - Add real PostgreSQL identity and ownership regressions, a reproducible benchmark and database documentation. ## Verification - Local service SQL benchmark: ownership 78.30 ms to 1.14 ms, with the same two complete rows. Blocker action read 76.94 ms to 22.55 ms, with the same complete row. These are synthetic SQL timings, excluding process probes, provider calls and HTTP. - Status delivery returned the same 100 rows. Timing was neutral on the small-task fixture: 7.85 ms before and 8.30 ms after. No speedup is claimed for that case. - 231 local tests passed across six server suites. They cover malformed identifiers, canonical spelling, company scope, native precedence, retained processes, workspace restore, native replacement, conversations and delivery. - `pnpm -r typecheck` and `pnpm build` passed. - `git diff --check` passed. - All current-head CI and security checks passed, including the full general and serialized suites, browser shards, Runner verification and build. Greptile reviewed the exact commit at 5/5 with no actionable findings. - The older full local sweep in the companion worktree was stopped after final-commit complete CI passed. It is not claimed as a completed local pass. The 231 focused tests above passed locally on this branch. ```sh pnpm --filter @paperclipai/server exec vitest run src/services/conversation-continuation.test.ts src/services/legacy-execution-recovery.test.ts src/__tests__/legacy-workspace-restore-recovery.test.ts src/__tests__/chat-completion-delivery.test.ts src/__tests__/agent-conversations.test.ts src/services/native-runtime/native-safe-replacement.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-recovery-queries.ts --baseline-ref=1e731520bb ``` ## Risks - PostgreSQL accepts more UUID spellings than the old text joins. The explicit guard preserves the old lowercase spelling requirement. - Query plans depend on the data distribution. Small task sets can show neutral status-delivery timing. The benchmark is not a production latency claim. - Recovery ownership checks remain conservative. Process, lease, workspace and no-replay rules are unchanged. - No migration, new index, cache, cleanup change or lock removal is included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, shell tools and local code execution. The runtime does not expose a more specific backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ebeac833a6 |
perf(runs): bound history and reduce repeated context reads (#16286)
## Thinking Path > - Paperclip helps operators manage AI agents and inspect their work. > - Run history supplies recent execution records to the board and API clients. > - The list repeats eight reads of a large stored JSON context. > - A caller can also omit the limit and read the complete history. > - This pull request expands context fields once and bounds every list read. > - The benefit is lower database cost and bounded redaction work. ## Linked Issues or Issue Description Refs #958. Related community reports: #13962 and #10631. This is an independent implementation. It does not use their commits or code. It covers the history bounds reported in both. Process memory monitoring from #10631 is outside this change. **What happened?** The run-history query repeatedly extracts fields from large context JSON. Requests without a limit also load every historical run before authorization and redaction. Accumulated staging query statistics showed 271 ms for summary history and another 58 ms for its redaction lookup. **What did you expect to happen?** Normal reads must use a bounded recent history. Context projection must preserve the existing values and privacy behavior with less repeated database work. **Steps to reproduce** Run the disposable benchmark below against the baseline revision. It creates 1,400 synthetic runs with 192 KiB contexts. It compares the exact service SQL and every returned column for both summary and full history. ## What Changed - Expand context fields with one lateral record projection. Preserve non-object legacy contexts and the first-class issue binding. - Enforce a 200-row default and a 1,000-row maximum in the route and service. - Apply the default in the shared UI client, including both agent detail implementations. - Preserve response arrays, authorization, result projection and secret redaction. - Fetch older linked runs by ID in both agent pages. Check company and agent scope, navigation races and visible retry errors. - Update OpenAPI, data-path documentation and capability maps. - Add real PostgreSQL and HTTP regressions plus a reproducible benchmark. ## Verification - Local benchmark: summary history 37.72 ms to 19.34 ms; full history 37.84 ms to 19.01 ms. All 200 complete positional rows match in each comparison. These are synthetic local SQL timings, excluding HTTP and redaction. - History, route, privacy and redaction suites: 160 server tests passed across six files. Twenty-two UI API, selection, run-panel and log-visibility tests passed. Both agent-page variants render an older linked run with an empty history page on desktop and mobile. - `pnpm -r typecheck` and `pnpm build` passed. - Map validation and `git diff --check` passed. - All current-head CI and security checks passed, including the full general and serialized suites, browser shards, Runner verification and build. Greptile reviewed the exact commit at 5/5 with no remaining findings. - The full local sweep began before the final compatibility fixes. It was stopped after the final commits passed complete CI. The focused local suites listed above passed on the final code. Before interruption, the broader local sweep passed 31,743 tests across the general server, UI and several package groups, with no reported failures. The full local command is not claimed as a completed pass. ```sh pnpm --filter @paperclipai/server exec vitest run src/services/heartbeat/run-state.test.ts src/__tests__/heartbeat-list.test.ts src/__tests__/agent-live-run-routes.test.ts src/__tests__/run-secret-redaction.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts pnpm --filter @paperclipai/ui exec vitest run src/api/heartbeats.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-run-history.ts --baseline-ref=1e731520bb ``` ## Risks - Clients that relied on an unbounded array now receive 200 rows by default. Explicit limits allow up to 1,000. This change does not add pagination. - Legacy context conversion must stay identical. Real database regressions cover null, array, scalar, nested values and native issue precedence. - No schema migration or persistent cache is introduced. Redaction and access checks remain active. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, shell tools and local code execution. The runtime does not expose a more specific backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
03fcd3c1fb |
fix(native-runtime): report which close barrier failed (#16298)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agent runs use the Paperclip runner. When a run ends, the controller closes the session. Close must prove that the runner durably suspended before the controller releases its checkpoint. > - If close cannot prove this, the run fails with `provider_transport_failed: runner did not durably suspend before checkpoint`. > - The runner knows which barrier was not proven. It writes that settlement only to the instance run log. > - The Sentry report for the failed run has no settlement evidence. Operators cannot tell a stuck suspend command from an undrained provider, an unreadable provider state, or unsettled tool results. > - This pull request sends that settlement to the run failure report as closed labels and counts. > - The benefit is that the next occurrence shows which barrier failed, so the root cause can be fixed instead of guessed. ## Linked Issues or Issue Description No public issue exists. This describes the bug. **What happened?** Hosted staging native Codex runs failed with `provider_transport_failed: runner did not durably suspend before checkpoint`. This happened on two runs, on 2026-09-30 and 2026-10-14. The second run used a release from 2026-10-14. The Sentry events contain only the error code, run phase, driver kind, and duration. They contain no stack and no settlement data. The native executor calls `reportRunFailure(db, run)` without the caught error, and the runner settlement is never read. **Expected behavior** A failed close report should tell an operator which barrier was not proven: runner suspension, provider drain, provider state, or semantic tool settlement. It must not expose tool identities, digests, provider turn IDs, or unredacted error text. **Steps to reproduce** 1. Run a native Codex session whose close cannot prove durable runner suspension. For example, the runner suspend command does not complete before the close deadline. 2. Observe the Sentry run failure event. Before this change, `run_execution` has no close settlement fields. ## What Changed - `collectRunFailureDiagnostics` accepts a new `nativeCloseFailure` option. It reads only a `native_session_close_unrecoverable` error, directly or up to four `cause` levels deep. - It adds closed labels and counts to `run_execution`: `closeRunnerSuspended`, `closeProviderDrained`, `closeSuspendCommandStatus`, `closeRunnerLifecycle`, `closeRunnerIdentityMatches`, `closeProviderState`, `closeProviderPendingEventCount`, `closeProviderTurnActive`, `closeSemanticResultPersistenceFailed`, `closeSemanticToolsPending`, `closeSemanticToolDeliveriesOpen`, and `closeSemanticResultFailures`. - Values outside the known vocabularies are dropped. Getters that throw are contained. - The native session executor passes the caught error as `nativeCloseFailure` when it reports a terminal run failure. It does not pass the error as `error`, because the raw error text and stack bypass the executor's identity redaction. - `doc/observability.md` documents the new fields and the privacy boundary. ## Verification - `cd server && npx vitest run src/services/__tests__/run-failure-diagnostics.test.ts src/__tests__/run-failure-sentry.test.ts`: 130 tests pass. - New tests check the full label set, a wrapped cause, null suspension and provider states, unrelated error codes, hostile and unknown values, and a missing settlement. They also check that no tool identity, digest, turn ID, or error text leaves the module. - Negative control: with the reader disabled, the new main test fails. - A scoped `tsc` run on `run-failure-diagnostics.ts`, its test, and `run-failure-report.ts` passes. The full server `tsc` and `pnpm build` were not run locally. The sandbox has a 4 GiB memory limit and no `cargo` for the runner binary. CI runs the full checks. ## Risks - Low risk. The change only adds Sentry context fields for one error code. Close, quarantine, retry, and recovery behavior do not change. - Sentry grouping does not change. The fingerprint stays `[errorCode, agentAdapter]`, and no exception chain is added. - The data path is Observability (Sentry error reporting, gated by the operator DSN). It does not change Telemetry. The fields contain only booleans, fixed labels, and counts. The DSN gate is unchanged. - These fields are observations. They do not prove a root cause by themselves. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), run through Claude Code as an agent with tool use (shell, file edits, test execution, Sentry and GitHub connectors). Context window and reasoning mode are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
df4ee06e2f |
feat(skills): select and validate tracked GitHub branches (#16288)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Skills can track a branch in an authorized GitHub repository. > - Sources previously offered only a default-branch repair action. > - People need to select a listed branch or enter a branch that GitHub has not listed. > - The server must validate the selected ref with the caller's authorized connection. > - This pull request connects the approved branch picker to production APIs. > - Installed skills remain usable while the source changes its tracked ref. ## Linked Issues or Issue Description **Subsystem affected** Skills UI, shared API contracts, GitHub source services, and agent API discovery. **Problem or motivation** The default-branch repair action does not let people choose another branch. The settings service also validates the old ref when a caller supplies a new ref other than HEAD. A missing tracked branch can prevent connection repair even when the selected account has repository access. **Proposed solution** Load a bounded branch list with the authorized connection. Mark the default branch. Search listed branches or accept a typed ref. Validate the selected ref before saving. Keep the chosen value after a load error, invalid ref, or concurrent update. Save the selected connection and branch together during connection repair. **Alternatives considered** A default-branch button does not support a named branch. A text field alone does not help people find the repository default or available branches. **Roadmap alignment** This improves the existing Skills source manager. It does not add a new skill import system. **Additional context** Refs #16066 and #16256. This is the production implementation of the approved branch-selection Storybook journeys. ## What Changed - Add an authorized branch-list endpoint and shared response/query types. Register it in agent API discovery. - Validate the requested ref, including the returned commit SHA. Preserve source leases, revision checks, project ownership, skill identities, selections, and installed versions. - Add the searchable branch picker to Sources and connection repair. Allow typed refs during branch discovery and after discovery fails. - Keep save errors in the dialog. Retain the selected branch through a concurrent source update. - Use the production components and API client in Storybook. Add keyboard, touch, loading, retry, invalid-ref, connection-repair, and concurrent-update browser journeys. - Update product documentation and the feature/access maps. ## Verification - Red/green: two new database regressions fail against the previous service and pass against the implementation. - Linux database/service suite: 85 tests passed. This includes source visibility, human/agent authorization, selected connections, immutable versions, and invalid refs. - Rebased branch and review fixes: 94 focused UI, route, branch-validation, and fixture tests passed. Two conflict-message regressions and the stale-retry fixture regression fail before the fix and pass after it. - Storybook build passed. All 26 focused browser journeys passed. - Repository typecheck and build passed. Final UI typecheck passed after the review fixes. - The full local Vitest run was interrupted when the checkout and its dependencies disappeared. It reported missing working-directory/Postgres files and worker startup errors. Its two timed-out tests pass after the checkout was restored. The complete CI suite passed on the final head, `cf5d4c8f0a`: [CI run](https://github.com/paperclipai/paperclip/actions/runs/38571068332). One unchanged workspace timestamp test required a single rerun. Its release-minus-one-millisecond value can equal the original confirmation time. The rerun passed. - Live acceptance passed on implementation head `5dd935bf14` in an isolated macOS arm64 instance with Node 24.21.0 and an authorized private GitHub repository. The UI loaded the actual default branch, saved branch and selected connection together, and preserved the installed skill/version IDs after reload. An invalid typed ref retained the draft and showed the provider error. A valid typed HEAD ref then saved successfully. The owned runtime and database were stopped after the test. Focused unit, Linux database, and browser suites passed again after the error-handling review fixes. - Greptile reviewed final head `cf5d4c8f0a` at 5/5. Both review threads are resolved; the fresh review has no remaining findings. - Token gates and map/catalog checks passed. - Review entry points: `Skills / Project repository skills / 03 Branch selection components` and `04 Branch selection journeys`. Run `pnpm storybook` to inspect them. - Browser command: `pnpm exec playwright test --config tests/storybook-visual/playwright.config.ts tests/storybook-visual/skill-source-branch-selection.spec.ts tests/storybook-visual/project-repository-skills.spec.ts`. - Manual check: open an independent GitHub source, choose a new branch, select the default or type a ref, save, and reload. Try an invalid ref and confirm the selection remains in the dialog. For a project-bound source, change its ref in project settings. ## Risks - GitHub discovery is bounded to 100 listed branches. The default branch is always included. Typed refs remain available. - A source can change while the dialog is open. The API rejects a stale revision and the UI keeps the draft for retry. - A ref change schedules the existing refresh pipeline. Installed versions remain available until a successful refresh imports new content. - No database migration or credential storage change is required. Existing HEAD requests remain supported. ## Model Used OpenAI Codex, GPT-6 family. The serving model's exact identifier and context-window size are not exposed in this session. Used reasoning, repository tools, code execution, and browser tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d4892d4761 |
fix(observability): sample native runs independently of triggers (#16285)
Start native background run traces under the operator's root sampling policy and link them to valid trigger contexts. Preserve child parenting and silent unconfigured tracing. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f69c6a8a35 |
fix(github): retry and recover review publication (#16284)
## Thinking Path
> - Paperclip manages agent work and its published results.
> - GitHub review bots publish task-bound assessments through governed
tools.
> - A review can survive a caller timeout because its publication intent
is durable.
> - A transient provider failure still returned a failed receipt before
recovery.
> - Gateway replay could return an empty cached result, and publication
retries had no dedicated scheduler deadline.
> - This pull request adds bounded retries, receipt recovery, and an
independent durable worker.
> - A temporary GitHub failure can recover without another review or
duplicate publication.
## Linked Issues or Issue Description
Refs #16275. That change increased the GitHub tool deadline. This change
repairs retry and recovery behavior.
**What happened?**
A one-off GitHub HTTP 503 or 429 left the caller with a failed
publication receipt. The saved retry did not have its own scheduler
worker. An identical gateway call after a timeout could return an empty
cached result instead of the saved publication receipt.
**Expected behavior**
Temporary failures should retry under current authority. Pending work
should resume at its saved deadline after restart. Exact retries should
return the current receipt. Lost responses must not create duplicate
comments, findings, or checks.
**Steps to reproduce**
1. Admit an advisory PR review request and begin its assessment.
2. Return HTTP 503 or 429 for the first summary-comment PATCH, then let
GitHub succeed.
3. Submit the assessment through the run-scoped tool gateway.
4. On master, the result says publication failed. With this change, the
same call returns the processed receipt and neutral check.
5. For receipt recovery, use a shorter caller deadline and repeat the
exact call after publication finishes.
**Paperclip version or commit**
Reproduced on master at `6cb6ac3b8e40cb90d734ba115a2f22f850ff9477`.
## What Changed
- Classify transient transport, server, and rate-limit failures. Honor
Retry-After and primary reset deadlines. Stop retries for permanent
permission and validation errors.
- Retry safe reads and installation-token issuance within a bounded call
window. Retry publication writes through the existing outbox and
provider receipt checks.
- Wait for up to three publication attempts within a 45-second retry
window. Keep the durable eight-failure cap and enforce saved deadlines
on explicit retries.
- Register an independent GitHub publication worker with
transaction-owned wake notifications, startup recovery, and per-PR lease
deadlines.
- Recover exact comment, progress, and assessment gateway calls from
their stored publication. Require the same company, agent, task, run,
connection, tool, arguments, and current policy. Clear stale timeout
errors after recovery. Approved exact retries read the current receipt
through the existing signed replay path; the worker retains the verified
approval for deferred writes and still rejects revocation.
- Add regression coverage and update the connector guide and
feature/access maps.
## Verification
- Red: both one-off HTTP 503 and 429 regressions failed on master with a
failed publication receipt.
- Green: the same tests passed after the change.
- GitHub review workflows: 133 covered against isolated PostgreSQL 18.
The final combined run passed 132 and hit one fifteen-second timeout
under concurrent build load. All three variants of that workflow passed
in isolation. The earlier combined run passed all 131 workflows before
the two approval regressions were added.
- Retry policy and delivery coordinator tests: 35 passed.
- Additional focused checks passed for the actual eight-failure cap,
exact-argument conflicts, policy revocation, and timeout receipt
recovery. The approved deferred-publication and revocation regressions
failed before the review repair and passed after it, with one approval
request preserved.
- `pnpm -r typecheck`: passed. `pnpm build`: passed. `node
scripts/check-maps.mjs`: passed.
- The broad local `pnpm test:run` path remains blocked by the host's
exhausted PostgreSQL shared-memory slots. The changed database-backed
workflows use an isolated Docker database instead. All Paperclip CI jobs
passed at
|
||
|
DottaandPaperclip
|
7c016c049b |
perf(workspaces): wake terminal cleanup from committed changes (#16282)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use execution workspaces to retain their files and Git branches. > - A reaper closes eligible workspaces after their task tree is terminal. > - The reaper scanned every heartbeat tick and read Git before cheap eligibility checks. > - Permanent scans also query idle databases when no cleanup work exists. > - This pull request wakes cleanup after transaction settlement and restores bounded work at startup. > - The benefit is fewer scans while fresh cleanup guards continue to protect retained work. ## Linked Issues or Issue Description Related work: Refs #13619, Refs #11753. Those PRs bound scans or back off unchanged candidates. This PR uses the transaction work coordinator to remove the permanent reaper tick. It overlaps their early terminal-state filtering but does not change their runtime restart behavior or introduce a second Git scheduler cache. **What existing behavior does this improve?** Terminal workspace cleanup and Git inspection for workspace detail reads. **Subsystem affected** Server orchestration, execution workspaces, and idle sleep admission. **Current behavior** The heartbeat tick scans workspace candidates every thirty seconds. It runs Git before rejecting nonterminal trees, cooldowns, and active runs. Display inspections expire after five seconds. **Proposed behavior** Recover workspace rows in bounded startup pages. Wake after relevant issue, workspace, and pull-request work-product transactions settle. Schedule the terminal tree cooldown or a five-minute eligibility retry. Back off unchanged candidates across unrelated wakes, and invalidate their backoff after relevant committed changes. Park empty and nonterminal-only lanes. Reuse display inspections for thirty seconds. Read fresh Git before destructive cleanup. **Reason and benefit** Avoid recurring database and Git work when no cleanup is possible. Retain transaction admission holds, restart recovery, and fresh deletion checks. Retention eligibility can wait for the next host wake; active and failed recovery work still blocks sleep. **Breaking changes** No schema or API shape changes. Display Git state can lag by thirty seconds. Cleanup eligibility retries can take five minutes. ## What Changed - Add a terminal-workspace worker to the existing transaction work coordinator and scheduler. - Signal relevant writes before mutation on the owning transaction. Outer transaction settlement wakes cleanup. - Derive cooldown deadlines from the full company-scoped terminal task tree. Continue bounded recovery pages without a permanent sweep. - Bound unchanged candidate inspections to five minutes, retain at most 256 entries, and invalidate affected entries with scoped transaction hints. Repeated wakes preserve the earliest inspection deadline. - Collect deadlines from task trees already read by bounded pages. Restore the cursor after a failed page and rescan commits received during a rotation. - Check terminal state, cooldown, and active runs before Git. Preserve lifecycle locks, generation fences, and fresh final checks. - Keep failed or incomplete recovery visible to idle admission. Preserve the passive retention exception. - Extend the bounded display cache to thirty seconds and document the scheduling and wake limits. ## Verification - Focused workspace, coordinator, cache, and local sleep suites: 111 tests passed with real local PostgreSQL and Git worktrees where applicable. - Final worker, display cache, and local sleep suites: 28 tests passed. - The regressions prove three ineligible workspaces invoke zero Git scans, cached clean display state cannot remove new untracked files, and real producer notifications wait for outer transaction settlement. - `pnpm -r typecheck` and `pnpm build` passed locally. Map and module-boundary checks passed. - Work-product and sleep-safety suites: 221 tests passed. Reopen and issue-tree control suites: 46 tests passed. Final backoff, workspace, worker, and work-product suites: 87 tests passed. - Final workspace and worker suites: 81 tests passed. Deadline selection performs zero database reads. A failed full page retries its unread terminal workspace before recovery can park. - Full CI coverage passed on head `2c397970f7`: general tests, serialized server suites, runner verification, E2E shards, typecheck, build, and the final verification gate. The duplicate serial local `pnpm test:run` was stopped after full CI coverage passed; it is not reported as a completed local pass. - Greptile: 5/5 with both findings resolved. The security scan also passed. No failing or pending PR checks remain. ## Risks - Display state can be stale for thirty seconds. Destructive guards bypass the display cache. - Undelivered or otherwise ineligible terminal work retries after five minutes instead of each heartbeat tick. - Transaction signals cover the owning `createDb` scope. Direct SQL and independent server processes require startup recovery. - Local cooldown timers are maintenance hints. They do not establish an external wake alarm or whole-instance sleep safety. Existing blockers remain in force; failed bounded recovery keeps the host awake. ## Model Used OpenAI GPT-6, model ID `gpt-6`, through Codex. Reasoning, repository inspection, shell execution, and browser tools assisted the change. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1e731520bb |
fix(mcp): deduplicate discovered tool descriptors (#16281)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive governed app tools through the MCP gateway. > - GitHub review tasks can contain a PR discussion and inline finding threads. > - Discovery repeated each tool for every conversation linked to the task. > - Native tool registration treated these repeated descriptors as a fatal collision. > - This pull request lists GitHub tools once and deduplicates identical descriptors across providers. > - Agents can continue work while conflicting definitions and normal tool permissions remain enforced. ## Linked Issues or Issue Description **What happened?** A follow-up run on a GitHub review task failed with `assigned_mcp_tool_name_collision` before the provider started. The task had a main conversation and an inline finding thread. Discovery returned the same seven tools twice. Repeated descriptors from other providers could cause the same failure. **Expected behavior** Discovery exposes each identical tool once. Separate connection names remain separate tools. Conflicting definitions remain an error. Every call still passes the gateway's policy, approval, revocation, and audit checks. **Steps to reproduce** 1. Run a GitHub review and publish an inline finding. 2. Send a follow-up request on the same PR task. 3. Inspect the task-bound bot catalog and the next native startup. 4. Separately, return two identical descriptors from an assigned app or plugin catalog. Related: #13717 added the GitHub review workflow. ## What Changed - Use a distinct GitHub discovery query across conversations that share an endpoint and task. Keep both thread mappings. - Deduplicate identical gateway descriptors by their full gateway name across providers. Use this rule for MCP listings, plugin listings, assigned-tool startup, and fresh assigned-tool discovery. - Reject conflicting descriptors with `tool_name_collision`. Keep the projected-name collision guard. - Cover GitHub inline-thread discovery, repeated plugin/app tools, direct and on-demand catalogs, revocation, conflicting bindings, and separate connection names. - Update the execution contract, GitHub documentation, and agent access map. ## Verification - Regression baseline: six new assertions failed on the original implementation. GitHub returned fourteen entries, plugin discovery repeated a tool, and native startup failed on repeated app tools. - Focused verification passed: 272 GitHub cases, all 29 assigned-tool cases, and all 88 gateway acceptance cases. - `pnpm -r typecheck` passed. - `pnpm build` passed. - `node scripts/check-maps.mjs` passed. - Current-head CI passed: 55 checks passed, two inapplicable checks skipped. Greptile scored the current head 5/5 with no unresolved threads. - Full local `pnpm test:run` encountered a 15-second timeout in the existing bearer-auth throttling test, before tool discovery. That test passed in the separate 88-case gateway run and on one isolated rerun. The duplicate broad local run was then stopped; the complete current-head CI matrix passed. ## Risks - Deduplication keeps the first identical descriptor and preserves listing order. Same-name descriptors with different definitions fail with HTTP 409 rather than selecting an ambiguous binding. - The GitHub query changes result cardinality. It retains the existing task/company/agent filters and leaves conversation rows intact. - No schema migration or credential change. Live staging recovery requires deployment and a new authorized run. ## Model Used OpenAI Codex, based on GPT-6, with code inspection, editing, browser investigation, and test execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e9b6c4d44c |
perf: remove history scans from task and chat queries (#16278)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task reads and external chat notices must remain responsive as run history grows. > - The chat notice sweep casts indexed UUID columns to text. > - Run redaction and task provenance filter large JSON contexts with expressions that lack matching indexes. > - These queries scan history and read large stored contexts on common requests. > - This pull request adapts two existing fixes to current master and adds the missing provenance index. > - The benefit is less database work with the same query results and access rules. ## Linked Issues or Issue Description Refs #15927. This change adapts that chat sweep fix to the extracted inbound notice module and retains the contributor credit. Refs #12499. This change adds that missing nested-context index through a current, concurrent migration. It retains the contributor credit. This combined change replaces those older layouts for the measured query paths. It also covers task creation provenance, which those pull requests do not address. **What happened?** The chat notice sweep scans wakeup history. Issue redaction scans run contexts because only one branch of its JSON condition has an index. The task creation-source filter also scans contexts because its native and legacy ID expression lacks an index. **Expected behavior** The queries use indexed lookups and return the same rows. Company access, task origin precedence, secret redaction, and chat notice ordering remain the same. **Steps to reproduce** Run `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-slow-queries.ts --baseline-ref=f8af817f8d`. The benchmark creates disposable PostgreSQL data and compares the old query with the current query on the same fixture. ## What Changed - Compare chat UUID columns directly with safely parsed payload IDs. Reject malformed and noncanonical IDs without a cast failure. - Add the nested `paperclipIssue.id` run-context index and the exact native/legacy task-provenance expression index. - Build both indexes concurrently. Register the migration for top-level execution and invalid-index retry repair. - Extend chat payload and migration regression coverage. - Add a repeatable synthetic SQL benchmark and document its scope and command. - Keep the five newest schema snapshots. ## Verification - Focused local checks: 76 selected tests passed in five files. These cover FIFO notices, malformed IDs, conversation state, redaction, provenance, and company boundaries. - Disposable PostgreSQL 18.1 fixture: 335 tasks, 1,400 runs, 13,000 wakeups, 377 actions, and 192 KiB synthetic contexts. - Five warm samples per query, after two warmups. Every returned SQL column matched before and after, compared as positional tuples so duplicate unnamed projections are retained. | Query | Before, median | After, median | Improvement | | --- | ---: | ---: | ---: | | Chat notice candidates | 26.21 ms | 6.43 ms | 4.1× | | Task creation-source list | 159.69 ms | 6.51 ms | 24.5× | | Issue redaction contexts | 131.88 ms | 0.34 ms | 389.1× | The run-history context projection returned the same 200 rows and kept the same index plan and buffer counts. Its timings were noisy (37.30 ms before and 56.78 ms after in the last run; 34.25 ms and 34.14 ms in the previous run). No improvement is claimed for this control. Its repeated JSON projection remains a separate optimization opportunity. The populated plans use the wakeup primary key, the provenance expression index, and a BitmapOr over both redaction indexes. These are local SQL measurements. They do not prove an equivalent change in hosted API latency. - `pnpm -r typecheck` and `pnpm build` passed locally. - The full local `pnpm test:run` was started and then interrupted (exit 130) when the Mac reached its PostgreSQL shared-memory limit. It did not complete and is not reported as passed. The final SQL benchmark passed after its disposable database resources were released. The full general and serialized test suites passed in CI. - All CI checks for final head `193acca6abaa992acad12f4b631bd31dd3234641` passed, including full Vitest coverage, E2E checks, typecheck, build, migration order, and lifecycle boundaries. - Greptile reviewed that exact head at 5/5 with no actionable comments. ## Risks - Both indexes add storage and write cost. Initial builds read stored run contexts and can take time on large histories. - Concurrent index builds avoid holding a write-blocking table lock for the whole build. The existing migration retry path removes invalid partial indexes before retrying. - The chat parser accepts only canonical lowercase UUID text. This preserves the prior text comparison, including rejection of uppercase and malformed values. - Staging has not been migrated or deployed. Local PostgreSQL and synthetic data do not reproduce hosted load or network latency. ## Model Used - OpenAI Codex, based on GPT-6. The exact serving model ID and context window were not exposed in this session. Used reasoning, repository tools, browser inspection, and local code execution. The adapted fixes retain their original contributor credits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c80d93e14e |
fix(observability): time execution configuration steps (#16236)
Record optional execution-configuration timings before provider dispatch, with success, failure, sampling, cleanup, and silent-unconfigured controls. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6cb6ac3b8e |
fix(github): allow review publication to finish before tool timeout (#16275)
## Thinking Path > - Paperclip manages agent work and its published results. > - GitHub review bots publish task-scoped assessments through the tool gateway. > - A submission saves its assessment and publication intent before it waits for GitHub. > - The gateway's ten-second default can expire while those provider requests continue. > - GitHub can then show a completed review while the agent receives a timeout and blocks its task. > - This pull request gives GitHub bot tools the existing sixty-second maximum by default. > - The caller can receive the confirmed publication result without manual task repair for ordinary slower responses. ## Linked Issues or Issue Description **What happened?** A GitHub bot's `submit_review` call returned `504 tool_timeout`. Its saved assessment continued to publish a summary and completed check after the caller stopped waiting. The agent treated publication as unconfirmed and blocked the task. **Expected behavior** A valid GitHub response that takes longer than ten seconds should return its confirmed publication receipt within the gateway's supported deadline. **Steps to reproduce** 1. Configure a report-only GitHub review bot and admit a PR review request. 2. Delay the summary-comment PATCH response by eleven seconds. 3. Submit a complete assessment through a real run-scoped tool gateway. 4. Before this fix, the caller gets a timeout while the durable action publishes one summary and a neutral check. 5. With this fix, the caller gets `completed`, `processed`, and the assessment score. Retrying does not add another comment. **Paperclip version or commit** Reproduced from master at `2c17d3ca52a7840f280f1dd87975a2ad77de3b35`. Related changes: Refs #16259 and #16263. Those changes repaired connection recovery. This change addresses the subsequent submission deadline. ## What Changed - Use a sixty-second default for `paperclip_github_chat` tools in run execution and board tool tests. - Preserve explicit shorter deadlines and the existing maximum. - Add PostgreSQL-backed regressions through the real gateway for delayed publication, explicit deadlines, neutral checks, and duplicate prevention. - Document the deadline and the durable publication behavior. ## Verification - Red: the eleven-second publication regression failed with `ToolGatewayHttpError(504, tool_timeout)` after the summary and neutral check were published. - Green: the same regression passed after the repair. - GitHub review workflow suite: 122 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm test:run`: running; results will be recorded before handoff. - Live local verification is running against a dedicated test App on [the disposable test PR](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/pull/9). A test-only preload delays one real GitHub PATCH response by eleven seconds. It is not part of this change. ## Risks GitHub bot calls can occupy their execution slot for up to sixty seconds by default. Calls that exceed that limit can still time out. Durable publication may continue after any caller timeout, so a timeout alone is not proof of a failed write. This change does not add automatic task reconciliation for arbitrarily slow or unavailable providers. No schema or credential changes are required. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f8af817f8d |
fix(ui): make external agent pairing codes copyable (#16274)
## Thinking Path > - Paperclip manages AI agents and their work. > - External agents connect through the shared MCP agent gateway. > - Grok setup gives the operator a one-use code for the authorization page. > - The password field hides the code but does not provide a working copy action. > - The pending approval button names Dot before the agent preview loads. > - This pull request adds explicit code copying and a neutral approval label. > - The operator can review the fixed agent without sending credentials to chat. ## Linked Issues or Issue Description Refs: #16219, #15581, #15414. **What happened?** The Grok setup code is in a read-only password field. Ordinary browser copying does not copy that field. The shared authorization page says “Connect Dot with pairing code” before it knows the invited provider. These defects blocked a watched Grok desktop setup before approval. **Expected behavior** An operator can copy the one-use code with a visible control. The code stays hidden and outside the Bot prompt. The authorization page shows the registered client and the fixed company, agent, permissions, and authorizing person before approval. **Steps to reproduce** 1. Choose New agent, Invite an external agent, and Grok Bot. 2. Save the Bot identity and open Connect access. 3. Try to copy the masked code into the Bot's browser authorization page. 4. Inspect the pending approval action before a valid preview loads. **Paperclip version or commit** The fix starts from master commit |
||
|
DottaandPaperclip
|
2c17d3ca52 |
fix(github): recover webhook verification in attention state (#16263)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review bots require signed webhook delivery before activation. > - Cloud can deduplicate an old ping after a connection repair. > - A fresh event could verify active connections, but an attention-state connection discarded it before authentication. > - This pull request allows signed delivery verification while the connection needs attention. > - It discards ordinary work until the manager completes the existing reconnect and activation checks. ## Linked Issues or Issue Description Refs #16259. Related: #16138 addresses the separate classification of local authorization failures. **What happened?** Live verification of #16259 found that a GitHub App already marked as needing attention remained stuck waiting for a signed ping. The Cloud relay had already delivered that ping. New signed events were discarded by the attention-state ingress gate before they could verify delivery. **Expected behavior** A fresh valid event from the bound installation verifies delivery. It does not activate the connection, change repository access, or start work. The manager can then complete the normal reconnect flow. **Steps to reproduce** 1. Put an existing App connection into attention state with webhook verification cleared after repair. 2. Deliver a fresh signed comment event for its bound installation. 3. Continue setup. Before this fix, webhook verification remains missing and reconnect cannot proceed. ## What Changed - Use a content-free verification path for signed ordinary GitHub events while the connection needs attention. - Allow a fresh signed event from the same installation to verify webhook delivery. - Verify delivery even from disabled or unavailable repositories. Discard the event before content storage, resource reconciliation, or runtime creation. - Keep newly verified connections in the provider-setup step. Later installation events cannot activate them before reconnect. Preserve normal installation recovery for existing qualified connections. - Extend recovery regressions to cover attention state, foreign installations, invalid signatures, missing installation IDs, unchanged resources, no wakeups, and successful reconnect after verification. - Document the recovery behavior. ## Verification - The targeted regression reproduced the attention-state failure before the change. - All ten recovery cases pass after the change. The tests cover disabled and unavailable repositories, content-free verification, and installation events after verification. - Server typecheck passed. - All 260 GitHub integration tests passed after the review fixes. Final server typecheck also passed. - `pnpm -r typecheck` and `pnpm build` passed. - The full local suite is running. One unrelated workspace-runtime conflict-reference test failed and then passed on its isolated rerun. Final totals and CI/review outcomes will be updated before merge. - Live verification will use the existing advisory review bot after deployment. ## Risks - Attention-state ordinary events authenticate under the current credential lease without creating a durable content record. They return before resource reconciliation or runtime creation. - Verification is still bound to the current credential lease, runtime generation, valid signature, and installation ID. It does not activate the bot or replace normal setup checks. - Paused, revoked, archived, and unconfigured connections retain their existing ingress rules. No schema changes or permission expansion. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c36281d639 |
fix(skills): repair missing branches and show grouped import progress (#16256)
## Thinking Path > - Paperclip lets people manage AI agents and their reusable skills. > - GitHub skill sources must keep their installed content usable during updates. > - A source can track a branch that no longer exists. > - The UI treats that GitHub error as a connection problem, so saving a valid account does not resolve it. > - Project workspaces also show the same repository several times, and long imports give little feedback. > - This pull request adds explicit branch repair, groups repository imports, and shows import progress. > - People can repair the actual problem and see when their import is working. ## Linked Issues or Issue Description Refs #16066 and #14713. This fixes the Sources experience introduced by those changes. Related adapter-source PRs #14340 and #14341 cover runtime mounting, not this source-management flow. No overlapping source-management fix was found. **What happened?** An installed source tracked `main` while its GitHub repository used `master`. GitHub returned HTTP 422. The UI showed a general read error and asked for a new connection after a valid account was saved. The project repository list repeated the same repository/ref for each project. Import had no spinner or explanation of queued work. **Expected behavior** Identify an unavailable branch and offer an explicit repair. Retain the selected connection and installed versions. Show one repository/ref row with its projects. Give immediate feedback and show the queued and active import states. **Steps to reproduce** 1. Install a source with an immutable commit and a saved tracked branch that does not exist. 2. Refresh it, or try to save a valid GitHub connection. 3. Add the same repository/ref to two legacy project workspaces and open Sources. 4. Import that repository and observe the row until installation completes. ## What Changed - Distinguish a missing GitHub revision from access denial and temporary read errors. - Validate both repository identity and the tracked revision when saving a connection. - Offer Use default branch on independent sources and in connection repair. Keep `HEAD` as the default-branch alias after refresh. Preserve skill IDs, versions and selections. - Keep active project refs under project configuration. Reject source-level changes that would alter their tracking. Recheck the ref under the source lock when project attachment races with branch repair. - Group matching repository/ref workspaces. Adopt eligible workspaces and skip pending workspaces. Keep remaining project links inside the existing source card. Allow remaining imports while another workspace is pending. - Show a spinner, busy state and queued/importing text. Keep the saved authorized account selected in connection repair. - Update shared contracts, feature/access maps, database notes and production-component Storybook journeys. ## Verification - Red/green: three local regressions failed on the original code: missing-ref classification, duplicate repository rows, and branch repair instead of another connection prompt. The changed UI/reader tests pass after the fixes. - Live local app with a real authorized GitHub credential: reproduced HTTP 422 for the unavailable branch; used the UI to restore default-branch tracking; confirmed the same connection, skill ID and single immutable version remained. - Live local grouped Paperclip import: one source, two project associations and 52 installed skills. Sources survived reload. No staging data was changed. - All 16 project repository Storybook browser journeys pass on the final head, including keyboard focus recovery after branch repair. These include branch repair, grouped imports, connection selection, recovery, removal and responsive layouts. - Repository typecheck, production build, Storybook build, token gates and 28 map/catalog tests passed. Final UI typecheck and build also passed after grouping remaining project links. - Linux qualification: all 94 focused source, project import, GitHub reader and UI tests pass with supported Node 24. No database tests were skipped. Both review findings failed before their fixes and pass after them. - The full local `pnpm test:run` attempt was stopped after macOS database suites exhausted the host shared-memory limit. The complete Linux CI matrix is the full-suite gate. The complete Linux CI matrix passes for head `f886e3ac073ec1c6f051fd6429496234f5143694`. - Greptile rates head `f886e3ac073ec1c6f051fd6429496234f5143694` 5/5. Both findings are fixed and all review threads are resolved. - Storybook: `skills-project-repository-skills-02-journeys--missing-branch` and `--duplicate-projects`. ## Risks - Default-branch repair changes the tracked branch only after an explicit user action. Existing execution version pins stay intact. - A selected connection now needs access to the tracked revision, not only repository metadata. - Grouping is by repository and ref. Project access and provenance remain separate and are checked by the existing services. - Legacy source errors contain safe text. The repair-action classifier maps known errors and treats other failures as retryable. No credentials or provider response bodies are returned. - No schema migration, workflow change, lockfile change or staging deployment. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model identifier and context-window size are not exposed in this session. Tool-assisted implementation, code execution and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a8b652b085 |
fix(github): preserve review delivery through reconnect (#16259)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review bots turn authorized repository events into agent tasks and publish the result on the pull request. > - Cloud Apps send callbacks through a relay, but the direct callback recovery worker reported that URL as broken. > - Reconnecting the same App then prevented a new authorized publication from updating its existing working comment. > - Report-only runs could also produce an action-required check when no complete assessment was available. > - This pull request preserves comment identity across reconnects, verifies fresh signed delivery after repair, and keeps report-only results neutral. > - Users can receive the review without changing repository access or making an advisory bot block their pull request. ## Linked Issues or Issue Description **What happened?** A review could be saved in Paperclip while GitHub kept showing “Working on this…”. A same-App reconnect changed the runtime generation and the existing response receipt then failed with “GitHub response binding changed”. Cloud callback recovery also compared the relay URL with a direct tenant URL. An incomplete or failed report-only run could produce an action-required check. **Expected behavior** A fresh authorized attempt updates the same App-owned comment after reconnect. Stale attempts and revoked people remain denied. Cloud relay delivery does not produce a direct-callback mismatch. Report-only checks remain neutral and explain incomplete work honestly. **Steps to reproduce** 1. Accept a GitHub request and publish its working comment. 2. Reconnect the same App before the agent publishes its result. 3. Submit the result and inspect the working comment and check. 4. Separately, let a report-only run end without an assessment and inspect its check conclusion. Related: #16138 addresses the separate classification of local authorization errors as provider credential failures. This change fixes the response binding and report-only paths themselves. ## What Changed - Bind durable response comments to the App and agent. Keep per-attempt runtime generation checks and provider comment ownership checks, including legacy receipts. - Exclude Cloud-managed Apps from the direct webhook recovery scan and its stale callback-mismatch warning. The Cloud inbox owns relay retries. - Let a fresh authenticated event from the bound installation verify webhook delivery after repair. Keep signature, runtime epoch, installation, and credential lease checks. - Keep report-only assessment and terminal-run checks neutral. Preserve incomplete/error text and gated-review behavior. - Add integration regressions for reconnect races, legacy comments, failed runs, Cloud recovery, and foreign or forged webhook traffic. Update connector documentation. - Correct the assigned-tool search guidance to name the actual input fields (`offset` and `schemaOffset`) and show valid arguments. The failed automatic review had an input-schema rejection before tool execution; this clarifies recovery without expanding tool access. ## Verification - `pnpm -r typecheck` passed. - GitHub policy and integration suite: 291 tests passed (`vitest run server/src/services/chat-github-review-policy.test.ts server/src/__tests__/chat-channels.integration.test.ts -t 'GitHub|github'`). - The report-only policy regression failed before the fix and passed after it. - `pnpm build` passed. - Assigned tool discovery: 24 tests passed. Its guidance now maps returned pagination offsets to the correct input fields and tells the model to omit unused optional fields. - Full `pnpm test:run` is running. It encountered a 15-second timeout in the unrelated authenticated Workspace calendar-write test. That test passed on one isolated rerun. - After the review-requested module extraction, server typecheck and both direct/Cloud recovery activity integration checks passed. - All 55 non-skipped CI checks passed for `32e690279d847e44a6946b6c7cfb110fb11aff79`. Greptile scored that commit 5/5 with no unresolved findings. Live staging publication verification follows deployment. ## Risks - A durable response can now survive a same-App reconnect. Fresh authorization, runtime generation checks for each publication attempt, App ownership, and the publication lease still fence writes. - Report-only failures intentionally become neutral. The check still reports the failure; it does not claim a complete assessment. - Cloud-managed callbacks rely on their existing durable relay inbox. The direct callback recovery worker remains unchanged for direct installations. - No schema migration or permission expansion. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40ecacff4c |
fix(voice): accept authenticated Cloud callers (#16254)
## Thinking Path > - Paperclip manages agent work through company-scoped tasks. > - Speko voice connections bind a caller to that work. > - Private voice routes must recognize an authenticated human caller. > - Cloud sign-in uses a trusted Cloud actor, rather than a local session actor. > - The voice caller check rejected that existing authentication type. > - This pull request accepts authenticated Cloud callers and retains the current access checks. > - Cloud users can load call history and start private voice sessions. ## Linked Issues or Issue Description Refs #15727, #15728, #15729. **What happened?** A signed-in Cloud user sees “Sign in to start a private voice conversation” when the Speko connection loads call activity. The same check prevents the user from starting a private voice session. **Expected behavior** The trusted Cloud user identity should have the same voice caller access as a local authenticated session. The current company and user identity checks must still apply. **Steps to reproduce** 1. Sign in to a Cloud workspace with experimental chat connectors enabled. 2. Connect a Speko persona to an existing agent. 3. Open its call activity, or start a browser voice session. 4. Observe the sign-in rejection despite the authenticated workspace session. **Paperclip version or commit** Reproduced on the voice route in master at |
||
|
DottaandPaperclip
|
b8ffeb973a |
feat(runner): add experimental Grok Bot agent participation (#16219)
## Thinking Path > - Paperclip manages AI agents that perform work for a company. > - The shared Runner owns task admission, authority, receipts and finalization. > - An existing Grok Bot needs to receive assignments and return results through that Runner. > - The Bot also needs governed access when it starts work from its own conversation. > - This PR adds agent-purpose MCP access and routine webhook dispatch. > - It reuses the external-agent Rust Runner and keeps Grok capabilities explicit. > - The provider stays experimental until the real-client managed journeys pass. ## Linked Issues or Issue Description **Agent or provider** Grok Bot, distinct from the Grok Build coding runtime in #13882. **Why this adapter is useful** Users can assign work to an existing Bot, receive progress, questions, documents and files in Paperclip, and let the Bot initiate work under its agent permissions. **How the agent is invoked** A webhook-only routine delivers bounded references. Dedicated named-agent OAuth and MCP provide inbound calls. Setup follows [Grok routines](https://cursor.com/help/grok-bot/routines) and [custom plugins](https://cursor.com/help/grok-bot/connect-plugins). The current desktop app needs its Bot-generated Webhook URL navigation link when routine-row clicks fail. The GitHub search found related Grok Build runtime work, but no existing Grok Bot agent provider. This extends the roadmap’s existing agent, Runner, governance and artifact foundations. ## What Changed - Add default-off Grok Bot participation through the shared real Rust external backend, preserving native provider identity guarantees and distinguishing session closure from remote termination. - Bind agent OAuth to company, agent, client, gateway and authorizer; recheck permissions and task-bound execution authority without manufacturing a native identity. - Persist routine configuration, dispatch intents, authenticated acknowledgement and recovery holds. Store URL/key together in encrypted secret storage, use guarded public egress, and never automatically replay ambiguous HTTP delivery. - Reuse native tools, durable waiting, completion assessment and review. Add task-bound file grants through existing artifact storage; suppress proactive echo dispatch and synchronous result wake loops. - Enforce the current native deliverable work-mode policy at file-ticket issuance and use. Add operator retries for confirmed rejected wakes, with retained receipts, stable retry intent and current-authority checks. Recovery locks bindings before routine rows. Thread Stop resolves the selected run and its agent. - Cancel queued and preparing Stop targets before assignment creation, restore saved setup after a failed request and successful retry, and publish the Grok provider descriptor schema. Advance the versioned Copilot source declaration for the shared schema change; its qualification remains pending. - Filter task summaries with the shared privacy policy, preserve feedback admission when wake delivery is blocked, reconcile settled delivery receipts without restoring work authority, and bind every Stop control to its displayed run. Recovery records distinguish dispatch uncertainty from a revoked session. - Add resumable guided setup, credential replacement, consent, management, observed work status and stop/recovery controls, plus an interactive Storybook journey using the production task thread, question form and report card. Grok’s own apps remain available; Paperclip workspace and third-party tools are deferred. ## Verification - Five focused protocol, authorization, person MCP and Runner files: 192 tests pass. The Grok assigned-work case executes the actual Rust Runner, saves a document, uploads/deduplicates a file and reaches ordinary finalization. - Proactive intake/deduplication and renewed-session recovery pass. These use scripted clients and synthetic storage; they are not real-provider acceptance. - Repository typecheck, production build, token gates, map checks and Storybook build/catalog checks pass. The local full-suite campaign was stopped after macOS shared-memory exhaustion repeatedly prevented embedded PostgreSQL startup. No local aggregate pass is claimed. The first clean-worker CI run identified avatar fixture/configuration, flag defaults, feature-card ordering and OpenAPI route-coverage gaps; all four focused regressions pass after correction. Those regressions pass. The latest clean-worker CI run also found a Copilot source-hash declaration mismatch after the shared descriptor update; the corrected declaration passes all 28 integrity tests. The next review corrections pass 144 rendered UI tests, 21 OpenAPI tests and 82 Runner contract/integrity/history tests. The latest database fixtures were blocked at startup by host shared-memory exhaustion; clean-worker CI must execute their new regressions. Full current-head CI now passes: 54 successful checks and four intentional skips. The database regression uses the PostgreSQL driver result shape; file-policy, rejected-feedback and lock-order cases pass on clean workers. Current-head Greptile is 5/5 with no unresolved findings. - Rendered setup steps and outbound-only verification were walked through in Storybook. Simulation cannot enable Done without an authenticated-reply state. - Actual client OAuth, continued access, reconnect and one routine wake with an exact chat acknowledgement were observed separately. This candidate’s live authenticated challenge, assigned/proactive workflows, Linux managed artifact, restart and cleanup are not yet qualified. The provider stays disabled by default. - The separate Cloud ingress change forwards only exact task-bound upload/download paths and preserves host identity and header stripping. - Current Cloud policy reserves Test for pre-merge app/harness experiments; Staging accepts only master-derived revisions. Real-client qualification needs a supported, authorized deployment target and remains outstanding. ## Risks One append-only migration adds routine/dispatch state and extends shared bindings and file tickets. The unshipped migrations were regenerated after upstream consumed their original numbers. Rolling back code retains the durable rows; revoke grants and inspect unknown remote activity before resuming dispatch. Grok cannot confirm remote termination, stream its complete transcript or report spending. Plugins are account-wide and do not isolate credentials between Bots. Real-client managed acceptance remains required before enabling the experimental provider. ## Model Used OpenAI Codex based on GPT-6, with reasoning, code execution and browser/native UI tools. The exact deployment model ID and context-window size are not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |