mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 09:41:04 +01:00
7bd449dbb66473ccb3fc9965186eed495d7cc623
2592
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin FoleyandPaperclip
|
7bd449dbb6 |
fix(recovery): project context evidence for latest-run checks (#17463)
## Thinking Path > - Paperclip manages AI agents and their tasks. > - Recovery sweeps decide whether tasks have a valid next step. > - Those decisions inspect saved run metadata. > - Latest-run queries also return full prompts and execution setup data. > - This repeats large responses when a sweep leaves a task unchanged. > - This change projects the context evidence recovery actually reads. > - A row-version check keeps stale evidence from changing a task. ## Linked Issues or Issue Description **What happened?** The latest-run recovery queries retrieve large stored contexts for each candidate. The decision usually consumes small recovery, message, or workspace-mode fields. **Expected behavior** Keep all decision evidence while selecting a small response. Preserve complete stored contexts and full execution reads. Detect concurrent source changes. **Steps to reproduce** 1. Store a terminal run with large task Markdown, wake, workspace, and continuation bodies. 2. Call `reconcileStrandedAssignedIssues` for its assigned task. 3. Observe the full context returned by the latest-run query on each sweep. **Paperclip version or commit** Baseline: `410e5c6eedba98588a87e4104153898fb0437532`. Related read optimization: Refs #17405. Complementary fix: Refs #17460. These PRs have no code dependency. Searched existing PRs and issues for run projections, recovery reads, and database transfer. No duplicate was found. **Deployment mode** External PostgreSQL. Verification uses disposable real PostgreSQL and synthetic data. ## What Changed - Project typed context evidence in all three latest-run recovery queries. - Keep only the used workspace mode and external-interaction flag from nested setup objects. - Preserve original JSON types, message references, retry evidence, and existing query scopes and order. - Read PostgreSQL `xmin` with the run. Require the same row version during native blocked-wait repair. - Retain the original full-context comparison for callers that supply a full run without a row version. - Test large payloads, malformed JSON shapes, review-message recovery, and concurrent source edits. ## Verification - Final-source integration checks: **450 passed** across context projections, native finalization recovery, and the assembled heartbeat recovery service. - Focused issue-lock race check: **9 passed** (401 outside the filter), including omitted-context and late-output edits and successful repair on a fresh sweep. - Final-source recursive typecheck and full build pass. The full local stable suite is still running; current-head CI must also pass before merge. - Real-service SQL capture against the baseline: one returned row falls from 699,724 to 573 serialized JSON bytes (99.918% smaller). - The fixture uses four 174,764-byte incompressible execution fields. All unaffected returned columns compare equal. Regression tests check the retained decision evidence and unchanged stored records. - Five alternating warm samples after two warmups: baseline median 1.479 ms, treatment 1.084 ms. Ranges overlap; no request-latency speedup is claimed. Plans were captured outside the timing interval. - An initial full-context hash projection cut bytes but raised the local median to 7.457 ms. The final row-version check removes that cost. - One integration run overlapped the hash-export rename and failed seven tests on the removed export. A fresh final-source run passed all 450 tests. The earlier mixed-source failure remains recorded and is not counted as a pass. - All current-head CI checks pass on `17add20e391c30ed0496f4b39eb9c59779efa6b4`. Greptile: **5/5, medium impact**, no findings or unresolved threads on that head. - Hosted transfer and product behavior remain to be verified on staging after merge. ## Risks The context selection must track its recovery consumers. Tests preserve typed flags, retry budgets, external-chat references, and malformed-input behavior. The row-version guard is conservative: any intervening run update defers native blocker repair to a fresh sweep. Concurrent omitted-context and output edits are covered through the real service and issue-lock wait; a fresh sweep must still repair. There is no migration, prompt truncation, history deletion, or authorization cache. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
60ca3a903a | Keep warm sessions across project revision transitions (#17461) | ||
|
Devin FoleyandPaperclip
|
14e1f59fbf |
fix(heartbeat): avoid large run reads during deferred-message recovery (#17460)
## Thinking Path > - Paperclip manages AI agents and their tasks. > - Deferred messages must survive restarts and execution cleanup. > - Periodic recovery checks the latest run before it attempts release. > - That check and two release checks load large execution records. > - These checks use only admission evidence and completion identity. > - This change selects those fields while keeping the release policy and fresh reads. > - Full execution records remain available for execution and detail reads. ## Linked Issues or Issue Description **What happened?** Deferred-message recovery repeatedly transfers stored context, Runner profiles, and output excerpts to make small admission and release decisions. **Expected behavior** Select the fields each decision consumes. Preserve queued messages, current admission checks, cleanup ownership, and fresh completion reads. **Steps to reproduce** 1. Save a terminal run with large context, profile, and output fields. 2. Leave an accepted message deferred after the issue lock is released. 3. Call `resumeQueuedRuns` and follow execution release. 4. Observe full-row reads before the admission decision. **Paperclip version or commit** Baseline: `410e5c6eedb`. Related read optimization: Refs #17405. Searched open and closed PRs for run projections and stranded-queue reads. Searched open issues for database transfer. No matching fix was found. **Deployment mode** External PostgreSQL. The regression tests use disposable real PostgreSQL. ## What Changed - Select seven admission fields for the stranded queue's latest-run lookup. - Select authentication classification evidence before execution release. - Read completion status and typed issue identifiers after post-commit effects. - Keep company scoping on both release reads. - Add large-payload and fresh-state regressions. Document the read contract. ## Verification - Focused PostgreSQL tests: 68 passed across queue, run control, and the new release-read suite. - Release reads return less than 2 KiB with three large synthetic execution fields. - Queue recovery passes less than 512 bytes to release, retaining the full stored message. - Coverage includes legacy/native owners, paused recipients, live successors, authentication repair, and fresh completion identity after post-commit effects. - Root recursive typecheck and full build pass. The full local stable suite remains in progress; no local full-suite pass is claimed. - All current-head CI checks pass on `c819fbb6b46bd5480a197407f06e02f245667792`. Greptile: **5/5, medium impact**, zero comments and zero unresolved threads on that head. - The first focused attempt found a syntax error in the new test; it was corrected before the 68-test passing run. A redundant standalone compiler was stopped while the required recursive check continued. - Hosted impact is not measured yet. Staging verification follows merge. ## Risks The projection must retain all admission evidence and preserve JSON types. The release policy still owns transactions, locks, cleanup, and promotion. There is no schema migration, polling change, message deletion, or API change. Configuration repair evidence is read directly, including when unrelated result output is large. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
410e5c6eed | fix(native): use current authority for warm plugin completion (#17444) | ||
|
Nicky Leach
|
988fe0180b | fix(heartbeat): keep source-free project policy stable across turns (#17447) | ||
|
DottaandPaperclip
|
0eb2c964a7 |
fix(agents): allow runner qualification with agent configuration permission (#17448)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent owners configure the harness and computer used for that work. > - A legacy runner upgrade must first qualify an unverified sandbox. > - The upgrade called an instance-admin probe, so a company owner with agent configuration permission received HTTP 403. > - This pull request qualifies the saved computer through the agent's existing configuration authorization. > - Company owners can complete the upgrade while general instance environment management stays restricted. ## Linked Issues or Issue Description Refs #17361. Related: #17122. **What happened?** Upgrade runner displayed for an existing Claude agent on a Daytona Paperclip Computer. Clicking it returned “Instance admin access required” for a cloud company owner. The agent remained on the legacy runner. **Expected behavior** A caller permitted to update the agent can qualify that agent's saved computer and upgrade. Instance-admin access is not required for this agent-scoped operation. **Steps to reproduce** 1. Sign in as a cloud company owner with `agents:configure` and no instance-admin role. 2. Open an existing legacy Claude agent on an unqualified Daytona snapshot. 3. Select Upgrade runner. ## What Changed - Add `POST /api/agents/{id}/qualify-runner` with the existing `agent_config:update` guard. - Reuse the server's effective environment resolver. The submitted environment ID only detects stale state; it cannot select a different computer. - Preserve company binding checks, check binding again at lease acquisition, and record agent-scoped audit activity. - Return qualification status and the lease receipt without instance-level provider metadata or raw provider errors. - Use the new endpoint for the existing-agent upgrade. Keep the generic environment probe restricted to instance admins. - Disable Upgrade runner when an unsaved computer change resolves to a different target, with a save-first explanation. Other draft settings still accompany an upgrade. - Add real-database authorization regression tests and update the API contract and access maps. ## Verification - Red: the existing UI called the instance-wide probe and failed the new agent-scoped call assertions in eight upgrade cases. - Green: 349 related route, provider qualification, runner selection, and UI tests passed. - Green: 12 authorization cases passed with real PostgreSQL, real HTTP routes, real saved role grants, and a simulated provider boot. They cover the cloud owner, removed grants, foreign agent/computer, inherited targets, stale target, invalid body, archived/local target, redacted failure, audit scope, and unchanged agent settings. - Review regression: two cases first reproduced the enabled upgrade button after an unsaved computer change; both now pass, along with all 168 related UI tests. Qualified/unqualified computers and restoring the saved selection are covered. - All 26 OpenAPI cases passed; the 12 real-database authorization cases also passed after rebase. - The host exhausted PostgreSQL semaphores. For the 12 cases, a temporary local-only setup used disposable PostgreSQL 18 in Docker instead of the embedded database launcher. The committed tests use the normal embedded database helper. No authorization decisions were mocked. - `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, map validation, module-boundary validation, and `git diff --check` passed. - `pnpm test:run` was attempted but did not establish a full local pass: the host could not start embedded PostgreSQL clusters. All current-head CI gates passed, including the full suite in CI. The commitperclip review workflow timed out before reporting any executed steps on its first attempt and passed on one retry. - Fresh staging acceptance remains pending deployment. Verify persisted native configuration and a successful runtime test on the existing Daytona computer after merge. Earlier live Daytona proof from #17361 is separate evidence. ## Risks - Qualification can start a fresh, temporary sandbox. It uses the existing probe lifecycle and cleanup contract. - The new path must remain scoped to the saved agent and its company. Denial and redaction tests cover that boundary. General instance probe authorization is unchanged. - No migration, role change, credential change, or automatic agent upgrade is included. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model variant and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7ae81c9fa7 |
fix(slack): support rich questions and complete review delivery (#17255)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Questions and review decisions must work in the conversation where a task started. > - Slack omitted question context and did not publish several review kinds. > - Some answers saved in Paperclip also lost their authorized continuation or their final Slack reply. > - A settled card and a successful run did not prove that the user received the result. > - This pull request preserves complete questions and follows the existing answer and delivery receipts through continuation. > - Users can read the request, submit an authorized decision, and receive the resumed task's result in the same conversation. ## Linked Issues or Issue Description Refs #17045 and #17218. **What happened?** Slack question cards could omit instructions and editable starting text. Checkbox reviews, item verdicts, task suggestions, and connection requests had no publication path. Complex question answers submitted in Paperclip could fail continuation authorization. Several review continuations completed without delivering their result to Slack. **Expected behavior** Show the actual question and context. Use native Slack controls when they preserve the form contract. Otherwise provide an explicit Paperclip review link. Persist one authorized response, resume through the existing delivery path, settle the card, and deliver the useful result to its originating conversation. **Steps to reproduce** 1. Start a task in Slack and ask a text or choice question with instructions and help text. 2. Submit native text and choice responses, then submit a complex form through Paperclip. 3. Accept a checklist, submit item decisions, select suggested tasks, or decline a connection request. 4. Inspect the saved answer, continuation run, original Slack card, and final Slack reply independently. ## What Changed - Preserve full prompts, context, descriptions, recommendations, help, and editable starting text. Use direct buttons for short choices, inline searchable selects for longer lists, and native checkboxes with explicit submission for multiple choices. - Add native custom-answer and validated-text modals, ordinary confirmation buttons, checklist decisions, per-item verdicts with reasons, suggested-task creation/dismissal, and connection decline. Keep governed approvals, permission grants, and forms that cannot be represented faithfully in the full Paperclip review surface. - Keep Slack-specific canonical-validation feedback in its owning platform module. Shared callbacks delegate without interpreting provider presentation. - Reuse the existing canonical interaction transactions, callback receipts, publication queue, and continuation delivery. Share ordinary confirmation handling across Telegram and Slack. Recheck current authorization and commit response receipts atomically. - Settle cards with the saved answer and remove stale controls. Ignore passive Slack link callbacks so opening a task does not display a false answer error. - Validate exact multi-select receipt mappings during native continuation. Recover automation-origin connection replies from their published prompt after validating the durable connection receipt. - Keep oversized multi-select handoffs free of unused native tokens, and retain the complete Paperclip form for 101–128 choices plus Other. Make the original-conversation audience explicit before submitting a Slack modal. - Add rollback, provider serialization, callback, continuation, and negative authorization regressions. Update the connector playbook and feature/access maps. No React or application UI changes are included. - Add a durable Q01–Q11 qualification index and provider coverage matrix. Require distinct functional, good-experience, and best-available-control assessments; real unanswered/answering/settled evidence; saved results, continuation and navigation proof; and retained failures. Mark other providers unaudited rather than assuming Slack parity. ## Verification Candidate: `e0017d446b1016a38d65382366cd1408cce33039`. The eleven-core-format live Slack gallery is pinned to `c3d458b2a06b660448940fd8cf33c5906f0058d5`; subsequent boundary fixes and the sharing notice have automated verification, not new live screenshots. - Real Slack UI testing covered all 11 formats: text, closed choice, long single select, multiple choices, choices with a custom answer, pattern-validated text, confirmation, checklist, item verdicts, suggested tasks, and connection requests. Each has unanswered/answering/answered screenshots as applicable, an inspected saved result, one continuation, and an individual experience review in the local HTML gallery. - The dedicated private QA app/channel used signed callbacks to the local candidate. A separately approved password-protected, read-only preview proved connection-review, suggested-task review, and created-task navigation. Anonymous access, writes, and credential routes were denied. No OAuth grant was attempted. - Real testing exposed and repaired a multi-select continuation failure and a misleading error on link-button clicks. Failed attempts remain in the evidence. The final navigation replay opened the exact pending connection request and declined it in Slack without granting access. Suggested-task creation produced the selected child with the correct parent and description. - Final review-fix verification: **32 form tests**, **28 provider/form-rendering tests**, **30 publication tests**, and **171 native continuation/authorization tests** passed. Earlier **3 atomic-resolution rollback**, **9 targeted callback/publication/Telegram**, and **28 review-continuation** cases passed. The final Slack/Teams modal subset also passed. The automation-origin connection cases, 101/128 custom-choice cases, and oversized multi-select continuation reproduced before their fixes and passed afterward; failing logs remain retained. These counts describe relevant runs, not a reliability percentage. - CI on `b5885b85fd` exposed a test fixture that assumed every Slack modal block was an input; the new sharing notice correctly renders a non-input block. The fixture now selects input blocks safely. The exact signed-modal correction/one-continuation test passed locally, together with 14 Slack form/modal tests. The failing CI log and its specific cause remain recorded. - `pnpm -r typecheck` and `pnpm build` passed. Server typechecking was repeated after the final source changes. Map and module-boundary checks and `git diff --check` passed. - The full `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL semaphore exhaustion on this host. Its 1,395-test chat suite found two stale native-control assertions; both were corrected and re-tested in a focused run. Focused database suites used separate disposable databases on existing local PostgreSQL. The full suite is not claimed green; new-head CI must complete before merge. - Final-head CI passed on `e0017d446b1016a38d65382366cd1408cce33039`: all test/build jobs and required `ci / verify` and `ci / e2e` gates are green ([run](https://github.com/paperclipai/paperclip/actions/runs/40115430194)). Greptile and Ponytail both reviewed this exact head at 5/5, the security scan passed, and all review threads are resolved. - All three QA app callbacks were restored to staging. Both temporary tunnels and local proxies were closed. The local endpoint was archived and its local credential copies removed. The candidate app was stopped. Limits: provider execution used a deterministic Claude protocol fixture, not an LLM. This is local real-provider UX proof, not deployed staging or broader Codie/Claude qualification. The public preview intentionally blocked ancillary artifact routes, so its destination pages could show a partial-history warning. The complete permission-grant flow and every oversized/mixed-form branch were not driven manually. Earlier fixture, test, and provider failures remain recorded. ## Risks - New reply routes require durable origin proof. Incorrect receipt matching could lose a reply or route it incorrectly. Tests cover missing, forged, mismatched, and revoked authority. - Governed forms keep their Paperclip resolution checks. This change does not grant new approval authority to Slack. - Existing completed review results can become deliverable through the repaired publication path without another provider run. - No schema, lockfile, workflow, public task-status model, or runtime-instruction change is included. This branch is not deployed. ## Model Used OpenAI Codex, GPT-6 family, with code execution and tool use. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the new head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the new head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b7506671b9 |
fix: preserve qualified snapshots during native setup validation (#17445)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agent setup can validate a selected sandbox before the agent runs. > - A qualified snapshot binds the saved environment and resolved runtime to an immutable provider snapshot. > - Setup validation changes lease retention so each test starts fresh and leaves an archived sandbox for inspection. > - The test passed that changed config as the saved environment, which made valid snapshot proof appear stale. > - This pull request keeps saved qualification separate from the two test retention settings. > - Native setup can use the qualified snapshot and still clean up its test lease. ## Linked Issues or Issue Description Related: #15997, #17262. **What happened?** Native setup validation can return `environment_lease_acquire_failed` for a sandbox whose saved snapshot qualification is valid. The test changes `reuseLease` and `archiveOnRelease` before the runtime checks the saved configuration fingerprint. **Expected behavior** The test must qualify the original saved environment, acquire a fresh lease from its immutable snapshot, and preserve archive-on-release in the lease metadata. A changed image, provider, runtime config, or stale proof must still fail. **Steps to reproduce** 1. Save a Daytona snapshot environment with `reuseLease: true` and `archiveOnRelease: false`. 2. Store valid company-scoped snapshot qualification for the saved and resolved runtime configurations. 3. Run native agent setup validation against that environment. 4. Observe that the validation retention overrides invalidate the saved fingerprint before a provider lease is acquired. **Paperclip version or commit** Reproduced from master at `785d54636e1c8513ee5503d602daf6a5ee1970d6`. The fix is based on `62f60bf158746c5807763778326b1eba4f2ed18a`. **Deployment mode** Any server with a qualified native Daytona sandbox environment. ## What Changed - Pass the original saved environment to native setup lease acquisition. - Allow only `reuseLease` and `archiveOnRelease` to differ in the run configuration. Apply those fields after saved snapshot qualification. - Keep both retention fields in the acquired lease metadata so provider release still archives the test sandbox. - Add service-to-runtime cases for valid and stale proof, immutable snapshot selection, fresh lease retention, release behavior, and rejected non-retention overrides. - Extend the route regression to require the original saved configuration. ## Verification - The focused route regression failed before the fix because acquisition had no `configuredEnvironment`. - All 101 focused route, snapshot, and orchestration tests passed on the exact rebased head. - Full repository typecheck and build passed on exact head `15e13e119941090e85638005de54b2d14ea52c00`. - The local database regression could not start PostgreSQL because macOS could not allocate system semaphores. The full `pnpm test:run` attempt also reported database startup failures and was stopped as incomplete. Neither attempt is counted as passing. - Exact-head Linux CI passed all 179 environment runtime integration tests, including the seven new regression cases. Its server shard passed 3,635 tests with 16 existing skips. All exact-head CI gates passed. The final repository review must also complete before merge. - No live provider or model call is part of this verification. This PR does not deploy the fix. ## Risks The acquisition boundary now accepts two explicit retention-only differences. All other config fields, environment identity, company-scoped proof, resolved runtime fingerprint, and template binding remain subject to the existing checks. Regression cases reject changed snapshot, image, provider, and environment variables before provider dispatch. No schema or API contract changes. No user-facing command or documentation contract changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, code execution, and test tools. The session does not expose an exact backend model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
62f60bf158 | Measure first deliverable text in native run traces (#17434) | ||
|
DottaandPaperclip
|
5db5734774 |
fix(plugins): scope operational reads to the caller's companies (#17435)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One instance hosts many companies, and company boundaries are the main isolation contract > - Plugin worker logs, job runs and webhook deliveries store a `company_id` attribution (NULL for instance scope) > - The plugin diagnostics routes checked board org access, but they did not filter those rows by the caller's companies > - A board member of one company could therefore read another company's log messages, run errors and delivery errors > - This pull request scopes those reads to the caller's companies and keeps the instance-wide view for instance administrators > - The benefit is that plugin diagnostics obey the same tenant boundary as the rest of the plugin routes ## Linked Issues or Issue Description **What happened?** A board user who is a member of only company A could read plugin operational rows attributed to company B. `GET /api/plugins/:pluginId/logs` returned log rows with `company_id` values outside the caller's companies. `GET /api/plugins/:pluginId/jobs/:jobId/runs` returned other companies' run rows, including their error text and run logs. `GET /api/plugins/:pluginId/dashboard` returned other companies' recent job runs and webhook deliveries. The routes only required board org access and filtered on plugin id or job id. **Expected behavior** A board member sees only operational rows for companies the member can access. An instance administrator (and the local implicit board) keeps the instance-wide view, which includes instance-scope rows where `company_id` is NULL. **Steps to reproduce** 1. Create two companies, A and B, on an authenticated instance. 2. Install a plugin and produce a log row, a job run and a webhook delivery for company B. 3. Sign in as a board user who is a member of company A only. 4. Call `GET /api/plugins/:pluginId/logs`, `GET /api/plugins/:pluginId/jobs/:jobId/runs` and `GET /api/plugins/:pluginId/dashboard`. Before this change, the responses included company B rows. After this change, they do not. **Paperclip version or commit** Reproduced on `2e6d3f7` (`master`, 2026-10-24). The report came from a scoped security audit of `52a66af29ddccf1fcac25f1f6efa9e9958e11fbd`. **Deployment mode** Self-hosted server, authenticated mode. The defect does not depend on the deployment mode, but `local_trusted` always yields an instance-admin actor and hides it. ## What Changed - Added `pluginOperationalReadCompanyIds()` in `server/src/routes/plugins.ts`. It returns `null` for instance admins and the local implicit board, and the actor's company ids for every other board actor. - Scoped `GET /api/plugins/:pluginId/logs` with `inArray(pluginLogs.companyId, companyIds)`. - Scoped `GET /api/plugins/:pluginId/jobs/:jobId/runs` through a new optional `companyIds` argument on `pluginJobStore.listRunsByJob()`. - Scoped the dashboard's recent job runs through the same optional argument on `pluginJobStore.listRunsByPlugin()`, and scoped its webhook delivery query. - Added `server/src/__tests__/plugin-operational-reads-tenant-scope.test.ts`. It starts embedded Postgres, seeds two companies plus instance-scope rows, and asserts the single-company view and the instance-admin view for all three routes. ## Verification - Red first: with the test added and the route unchanged, 3 of 4 cases failed and the response contained the company B rows. - Green after the fix: `pnpm exec vitest run server/src/__tests__/plugin-operational-reads-tenant-scope.test.ts` passed (4 tests). - Related suites: `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-tenant-isolation.test.ts server/src/__tests__/plugin-install-route-security.test.ts` passed (59 tests). - Typecheck: `pnpm --filter @paperclipai/server typecheck` passed. ## Risks - Behavior change for non-admin board members: they no longer see instance-scope rows (`company_id` NULL) or rows for companies they cannot access. Instance administrators keep the full view. For a single-company instance the visible rows are the same, except instance-scope rows. - No schema or migration change. No API shape change. Row-level filtering only. - A plugin that writes diagnostics without a `company_id` will show fewer rows to non-admin users. That is the intended tenant boundary. ## Model Used - Provider: OpenRouter - Model: `deepseek/deepseek-v4.1-flash-1017` (DeepSeek V4.1 Flash) - Harness: OpenCode, with tool use, code execution, and multi-agent delegation - Reasoning: enabled (thinking mode) - Context window: not exposed by the harness ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
785d54636e |
Delete re-homed isolated clones when their contents are reproducible (#17410)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Isolated task workspaces are Git worktrees on the server disk, and the terminal workspace reaper keeps that disk usage bounded > - #17369 moves old workspaces off their private per-issue clones onto the project's shared clone and then deletes the private clone > - On one instance, the first live sweep moved 12 workspaces but deleted none of their clones (about 600 MB each), and 3 workspaces could not move because their restore point was "too large" > - The kept clones are not referenced by any workspace, so no other cleanup removes them > - This pull request replaces the clone check with a real reproducibility check, sweeps the orphaned clones, lifts the restore point limits for local transfers, and fixes a false `source_identity_changed` > - The benefit is that the disk space #17369 was meant to reclaim is reclaimed, without deleting unique work ## Linked Issues or Issue Description Refs #17369. Bug report described in-PR (bug report template). **What happened?** On one instance, with `PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED=true` and dry run off, the first sweep re-homed 12 workspaces, skipped 5 with `live_or_unresolved_consumer` and 3 with `restore_point_too_large`. No clone was removed: 11 logged `cloneKeptReason: "clone_checkout_has_files"` and 1 logged `"source_identity_changed"`. Each clone's own checkout was clean (`git status --porcelain --untracked-files=all` empty). Its only history that the shared clone lacked was remote branches and release tags that the shared clone never fetched, which exist on the remote. An operator removed 11 clones by hand after verifying them. **Expected behavior** A re-homed clone whose checkout is clean and whose local history is in the shared clone or on the remote is deleted. A clone with real unique work is kept. Workspaces whose changes exceed the upload limits still move, because the transfer is local. **Steps to reproduce** 1. Use a project repository whose `.gitignore` (or the clone's `info/exclude`) contains `.paperclip/`, with a release tag on a commit that is not on any branch. 2. Re-home an isolated workspace of that repository with #17369. 3. The old worktree is removed, the empty `.paperclip/worktrees` directory stays, and `git ls-files --others --ignored --directory` lists `.paperclip/`. The clone is kept with `clone_checkout_has_files`. Without the ignore line, the local release tag keeps it with `unique_clone_history`. 4. Add a file larger than 64 MiB to a workspace. The move skips with `restore_point_too_large`. **Paperclip version or commit:** `master` at |
||
|
DottaandPaperclip
|
71c04c0bfb |
fix: stop idle environment cleanup polling (#17145)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent environments can leave pending teardown or unverified provider resources. > - Recovery must preserve those resources until ownership and workspace safety allow cleanup. > - Empty maintenance scans keep an idle instance and its database active. > - The existing rows, transaction signals, and scheduler can represent the actual recovery obligation. > - This pull request schedules pending cleanup and historical verification only while work remains. > - An empty or fully verified instance stops these scans without losing cleanup recovery. ## Linked Issues or Issue Description **What happened?** The server scanned pending environment cleanup on its maintenance tick. The run reaper also scanned it even when it recovered no runs or active leases. Historical resource verification scanned every five minutes, including when all history already had valid cleanup proof. **What did you expect?** Writes should wake cleanup. Pending rows, deferred orphan buffers, and unverified history should retain recovery deadlines. When none remain, these workers should make no periodic database or provider calls. **Steps to reproduce** Start a configured instance with no pending leases and fully verified history. Leave it idle. Observe repeated pending-cleanup and historical-verification reads from the maintenance paths. Related: #17106 supplies the preceding login-cleanup migration. #13523 improves lease-reaper query indexes. #16809 addresses reusable-sandbox recovery authority. This PR keeps those authority checks in their existing services. ## What Changed - Register two workers with the existing delivery coordinator and shared scheduler. Remove their unconditional maintenance-tick calls. Run the reaper's inline pending cleanup only after it recovers actual work. - Add commit-aware signals to the audited lease and custom-image writers. Existing transactions notify directly; other writers use one small transaction helper. No tables or migrations are added. - Derive pending cleanup deadlines from the existing row backoff and retry or claim metadata. Retain a historical retry only while an unverified candidate exists. - Register unwritten orphan buffers across runtimes that share the database handle. Buffer registration wakes recovery without a database connection. Load the existing spool at startup and retry failed or capacity-limited loads while needed. - Isolate spool failures per runtime. Attempt all buffers and persisted pending rows, then retain a retry for any failed spool. - Document producer coverage, idle behavior, and remaining blockers. Update the sleep inventory. - Repair the inherited CSS-recovery E2E test: the current build no longer emits task-page CSS. Exercise the actual lazy environment stylesheet, preserving real network failure, recovery, and bounded reload assertions. ## Verification October 24 integration update: merged current master and resolved the runtime/spool conflicts. Preserve serialized ingestion of records appended after startup. Plugin replacement failures now register deferred recovery with an idle hold; concurrent callers share one persistence attempt. The producer audit includes this new writer. - Focused cleanup selection: 270 tests now pass across five suites. The initial run lacked a built plugin SDK and executed no tests; building the SDK fixed setup. The new concurrent-flush assertion then reproduced duplicate rows; a single-flight guard fixed it and all 18 plugin integration tests passed on rerun. - Server typecheck passed. After review fixes, all 211 tests across the cleanup worker, real orphan spool, environment runtime, and plugin integration suites passed. These overlap the earlier focused selection. The rate-limit test initially expected a write after twenty idle minutes to wait; corrected it to assert that subsequent writes cannot accelerate the five-minute cadence. - Review fixes preserve retries for unreadable spool directories/records, keep historical provider checks off the startup wait while joining them at shutdown, and restore their independent five-minute minimum spacing. The complete [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/40108905033) passed on final commit `79ae1589c3c7b924db4bedb337d0c7ed3d95020b`, including all E2E shards, server/workspace tests, runner checks, typecheck, build/public-install verification, and security checks. - CI found a stale E2E asset matcher: `issue-detail-*.css` is no longer emitted. Both corrected browser tests passed against a fresh production UI build and disposable local instance. Fresh Greptile review is 5/5 on that same final commit, with all review threads resolved. Previous implementation verification: - `pnpm -r typecheck` and `pnpm build` passed. - New PostgreSQL worker suite: 12 tests passed. Covers quiet configured state, verified history, writes, outer commit and rollback, retry deadlines, restart, drain, deferred orphan buffers, spool failure, pagination, provider uncertainty, shutdown, and the idle run-reaper path. - Worker and environment runtime suites after review fix: 183 tests passed. A new assertion reproduced buffer starvation before the fix. Updated transaction fault injection and isolated simulated-instance database handles. Server typecheck also passed after the fix. - Run-recovery suites: 58 tests passed. Pending-cleanup suite: 38 tests passed after updating the logging fixture to seed real orphan recovery; an idle reaper no longer invokes cleanup. - Stop and workspace-recovery integration selection: 268 tests passed; two live-provider tests skipped by their environment gate. This overlaps the worker suite and is not an additional unique-test count. - Started the full local `pnpm test:run`, then stopped that older duplicate run after CI found the fixture mismatch. The corrected 38-test suite passed locally. The full sharded [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39678174479) passed on final commit `fc0ff3c4554ca4190a8b0b81ec06ef7065b8ad2c`, including build, typecheck, runner, server, workspace, and E2E checks. This is not a claim that the local full run completed. ## Risks - Missing a future producer signal could delay discovery until restart. The producer audit is documented in `doc/idle-environment-cleanup.md`. Independent SQL writers and another server process are outside the existing process-local notification contract. - Historical candidates blocked by ownership or provider uncertainty keep a five-minute retry until they can be verified. Provider destruction, claim fencing, Stop retention, and workspace safety remain controlled by their existing services. - This does not remove orphan-active discovery or legacy shutdown settlement from run recovery. Active custom-image expiry is also still periodic. External wake coverage does not change. - Local tests prove these workers become quiet and recover. They do not establish deployed instance suspension or fleet cost savings. ## Model Used OpenAI Codex (GPT-6), with code execution and reasoning. The exact serving model ID and context size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
e5c9e60c5a | Avoid session resets when reusing default task directories (#17419) | ||
|
Nicky Leach
|
f7716ec65b | Allow exact warm-state reads during retained task retirement (#17418) | ||
|
Nicky Leach
|
6b4bdc515b | Fix GitHub ownership checks for warm plugin tasks (#17417) | ||
|
DottaandPaperclip
|
87bf89756e |
fix: support Daytona runner upgrades and ignore obsolete Claude options (#17361)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Existing Claude and Codex agents can upgrade to Paperclip Runner. > - A Daytona computer needs platform qualification before Runner becomes its default. > - The upgrade notice required that default, so it hid the action that could start qualification. > - Saved Claude turn limits also caused conversion to fail even though Runner can run without them. > - This pull request verifies the computer on click and ignores obsolete Claude options with a warning. > - Agents retain their model, account, environment, instructions, and permission policy. ## Linked Issues or Issue Description Related: #17122. **What happened?** A legacy Claude agent on a Daytona snapshot had no upgrade button until its environment had platform qualification. A saved `maxTurnsPerRun: 1000` then caused conversion to return HTTP 422. **Expected behavior** The upgrade action verifies a supported Daytona computer and converts the agent. Unsupported tuning options produce a visible warning and do not block execution. **Steps to reproduce** 1. Create a legacy Claude agent with `maxTurnsPerRun: 1000` on an active Daytona snapshot. 2. Open Runtime before the environment has platform qualification. 3. Try to upgrade to Paperclip Runner. **Paperclip version or commit** After #17122, reproduced against `9c442cf48e`. ## What Changed - Offer upgrade on probe-capable Daytona snapshots and immutable sandbox images. Perform the public environment probe only after a click. Save after fresh adapter discovery confirms support. - Resolve the actual computer for explicit, instance-default, and hidden managed environments. Preserve pending edits and reject stale probe results. - Ignore Claude `maxTurnsPerRun`, `chrome`, `warmHandleIdleMs`, and `acpWarmHandleIdleMs`. Return structured warnings from create, hire, and update. Show warnings in Runtime and after a save. - Keep validation for commands, credentials, filesystem, network, and permission settings. - Retain the legacy turn limit in the existing live conversion fixture. Update regression tests, upgrade stories, and capability maps. ## Verification - Red: the old route returns HTTP 422 for the saved turn limit; the old form hides the Daytona upgrade notice. - Green: 417 focused server and form tests pass. A further 188 tests pass after the review fixes. Qualification coverage includes failed probes, unsupported targets, changed settings, managed defaults, unsaved environment variables, and forced Kubernetes targets. - `pnpm -r typecheck` and `pnpm build` pass. Token gates, map checks, map tests, and Storybook catalog checks pass. - Real local Claude Opus 5.5 conversion and task pass, with the downloaded artifact checked against seeded source bytes, SHA-256, and Git commit. - Real browser upgrade and full Claude Opus 5.5 task on a disposable named Daytona snapshot pass at `b81cec7d8c`. The saved agent retains its model, high effort, credentials, timeout, and grace settings while dropping the turn limit with a visible warning. Exactly one successful run completed the task. The downloaded artifact matched independently seeded source bytes, SHA-256, and Git commit; the source was unchanged, no interactions remained pending, and the delivery link survived a browser reload. The snapshot used current runtime assets built from public source. The supplied Daytona account could not access the original staging snapshot, so this verifies an equivalent local fixture rather than that exact snapshot. Disposable test resources were cleaned up. - Earlier live attempts remain recorded: an image-based sandbox creation failed, and the first named-snapshot probe returned failure with unresolved provisional cleanup identity. Later named-snapshot probing and the task succeeded; the initial creation failures are not attributed to this PR or declared resolved. A separate attempt using the Oct 18 cached runtime failed the current provider-pack compatibility check; rebuilding the runtime from matching public source removed that mismatch. Owned resources were identified and cleaned up, with diagnostic databases retained. - 54 checks passed at `b81cec7d8c`, with no failed checks; Greptile and Paperclip Review both scored 5/5, with both review threads resolved. Master advanced during the live proof, so the branch was rebased with both sides of additive import/test conflicts retained. On the rebased head `2b74222b94`, 472 focused tests, UI typecheck, and UI build pass. All required CI gates pass; Greptile and Paperclip Review both score 5/5 on this head, with no unresolved review threads. The PR is mergeable. The serial local `pnpm test:run` was stopped after the CI test lanes passed; that local run is not claimed as a pass. ## Risks - An explicit upgrade can allocate a temporary probe sandbox and take several minutes. Probe failures leave the agent on its existing runner. - Ignored options no longer enforce a turn cap, Chrome integration, or legacy idle-time tuning. The UI warns about each active ignored option. - Unsafe or ambiguous execution settings still fail validation. This change does not weaken environment qualification or runtime artifact checks. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
515ff0f284 |
fix(server): report which S3 call failed in storage error events (#17424)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server keeps attachments and other files in a storage provider. Hosted instances use the S3 provider. > - Sentry received `SignatureDoesNotMatch` errors from the S3 provider on two staging hosts. > - The events have only AWS SDK stack frames. They do not show which S3 call failed, the HTTP status, or the S3 request ID. > - Without these facts, we cannot find the cause or make a safe fix. > - This pull request records a small, fixed-shape failure record on the S3 error and sends it with the Sentry event. > - The benefit is that the next event shows the failed operation and the provider request ID, and no private data goes to Sentry. ## Linked Issues or Issue Description **What happened?** The S3 storage provider threw `SignatureDoesNotMatch` four times on staging. The Sentry events do not identify the S3 operation or the caller. **Expected behavior** A storage provider failure in Sentry shows which storage call failed and the provider's correlation fields. **Steps to reproduce** 1. Configure the S3 storage provider against an endpoint that rejects the request signature. 2. Read, write, check, or delete a stored file. 3. Look at the Sentry event. It has no first-party stack frame and no operation name. No public GitHub issue exists. I searched open and closed pull requests for `SignatureDoesNotMatch` and storage failure diagnostics. No pull request covers this. ## What Changed - Add `server/src/storage/failure-diagnostics.ts`. It keeps a frozen, fixed-shape record for each failed storage error in a `WeakMap`. - Add `server/src/storage/s3-failure.ts`. It wraps each S3 call and records the operation, error code, HTTP status, request ID, and SDK attempt count. It reads only data properties. A value that does not match its bounded format becomes `unknown` or `null`. - Wrap `PutObject`, `GetObject`, `HeadObject`, `DeleteObject`, `CreateMultipartUpload`, `UploadPart`, and `CompleteMultipartUpload`. - In `captureException`, add the `storage_operation` and `storage_error_code` tags and a `storage_failure` context to the event of that error only. The original error and its default grouping do not change. - Document the new context in `doc/observability.md`. ## Verification - `pnpm exec vitest run src/__tests__/storage-s3-failure-diagnostics.test.ts src/__tests__/sentry.test.ts` (in `server/`): 37 passed, 5 skipped. The skipped tests are the existing real-SDK tests, which need an environment flag. - `pnpm exec vitest run src/__tests__/storage-s3-multipart.test.ts src/__tests__/storage-local-provider.test.ts`: 8 passed. - The new tests use a local HTTP server that returns a real S3 `SignatureDoesNotMatch` XML body with a `StringToSign` field. The tests make sure that the record has the operation, code, status and request ID, and that it does not contain the private string. - I typechecked the changed source files with a scoped `tsc` project. They have no errors. I could not run the full `pnpm -r typecheck` and `pnpm build` in my sandbox: it has a 4 GiB memory limit and no `cargo` for the runner binary. CI runs the full checks. ## Risks - Low risk. The change adds a `try`/`catch` that records data and throws the same error again. Storage behavior, not-found handling, and HTTP responses do not change. - Secret-sensitive routes replace the error before reporting, so those events do not get the record. This is intended. - The record never contains the bucket, object key, endpoint, credentials, error message, or provider response body. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude Code with tool use (shell, file edit) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5e26b733b9 |
fix(server): finish workspace operations when the log write fails (#17415)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Before a run starts, Paperclip prepares the execution workspace (for example `git worktree add`). Each step is a workspace operation. The workspace operation recorder writes it as a row and as a log file on disk. > - When an operation fails, the recorder writes the error to the log file, then sets the row to `failed`. > - If that log write also fails (for example ENOSPC when the disk is full), the recorder throws before it updates the row. > - The row stays `running` with no `finishedAt`. The log write error also replaces the real error of the operation. > - This pull request makes the failure-path log write best-effort, so the row always reaches `failed` and callers get the correct error. > - The benefit: on a full disk, the operator sees a terminal operation with the real failure reason. Operations do not stay "running" for ever. ## Linked Issues or Issue Description No public issue exists. Bug description: **What happened?** A run failed during workspace setup on a host with a full disk (`ENOSPC: no space left on device, write`). The stack shows the error in the workspace operation log append inside `recordOperation`. In the failure path, the recorder calls the log append before it updates the row. Because the append threw again, the row stayed `running` with no `finishedAt`. When the operation itself failed (for example a git error), callers got the ENOSPC log error and not the git error. **Expected behavior** A failed workspace operation always reaches `failed` with `finishedAt` set. Callers get the operation's own error. A log write failure must not hide that error. **Steps to reproduce** 1. Make the operation log append throw ENOSPC (the new tests stub `fs.appendFile` for the log root). 2. Record an operation whose `run` throws, or whose output append throws. 3. Before this fix: the row stays `running`, and the caller gets the ENOSPC error and not the operation's error. **Paperclip version or commit** `ee29e28` (current `master`). **Agent adapter(s) involved** Not adapter-specific (core bug). Related open PRs with disk-full handling in other places (no overlap with this code path): #5882 (run-log rotation), #16773 (backup disk use). ## What Changed - `server/src/services/workspace-operations.ts`: in the `recordOperation` failure path, the stderr log append is now best-effort (`.catch(() => undefined)`). The stderr excerpt still records the message, because the excerpt updates before the file write. `logStore.finalize` was already best-effort in this path. - `server/src/__tests__/workspace-operations-reconciliation.test.ts`: two regression tests with log appends that fail with ENOSPC: - The output append fails after a successful run. The row reaches `failed` with `finishedAt` and the ENOSPC message in the stderr excerpt. The caller gets the ENOSPC error. - The operation itself throws. The row reaches `failed` and the caller gets the operation's own error, not the log write error. ## Verification - Red before the fix: both new tests failed on `master`. The row stayed `running`, and the caller got the ENOSPC error and not the git error. - Green after the fix: `pnpm exec vitest run src/__tests__/workspace-operations-reconciliation.test.ts src/__tests__/workspace-runtime-control-recovery.test.ts` passes (17 tests). - Sibling suites that use the recorder (26 files, 945 tests): all pass except 4 tests in `workspace-runtime.test.ts`. Those 4 tests also fail on unchanged `master` in the same sandbox, because they need the local pnpm and worktree CLI. This change does not cause them. - Typecheck: `tsc --noEmit` on `workspace-operations.ts` passes. The full server `tsc` was not run locally because the 4 GB sandbox ran out of memory. CI runs the full typecheck. ## Risks - Low risk. The change affects only the failure path, and only the log file write. The success path does not change: if a log write fails after a successful run, the operation still fails with the write error. - The log file for a failed operation can lose its last stderr line on a full disk. The row's stderr excerpt still has the message. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ee29e2850d |
fix: keep GitHub guest provenance from failing tool discovery (#17340)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents reach their tools through the MCP tool gateway. Every run starts with `tools/list`. > - Tool discovery runs a GitHub guest sponsor probe (`githubGuestBotConnectionForSession`) before it lists any connection. > - The probe walked the run's identity chain before it knew whether the task had a guest sponsored GitHub message at all. > - When agent A works a task and the task is reassigned to agent B, B's run chain continues A's run. The chain walk refuses with a plain `HttpError`. The gateway returns 403 for `tools/list` with no audit record. B loses every tool, built-in and connected, even when the company has no GitHub endpoint. > - This pull request makes the probe look up guest sponsored GitHub messages for the task and agent first. When there are none, the probe answers `null` and does not consult provenance. > - The benefit is that a reassigned or continued task keeps its tools, while a guest sponsored task still refuses a broken chain. ## Linked Issues or Issue Description Fixes #17307 **What happened?** `tools/list` on the MCP gateway returned `403 {"error":"Chat tools cannot be inherited by another task"}` for every run whose identity chain contained a run of another agent or another task. No audit event was recorded. The agent received no tools at all, including unrelated MCP connections. The company had no GitHub chat endpoint. **Expected behavior** A run on a task with no GitHub guest sponsor lists its tools. The guest probe only decides whether the run must be limited to the sponsoring bot's connection. When the task has no guest sponsored GitHub message, the answer is "no sponsor" and discovery continues. **Steps to reproduce** 1. Let agent A run on a task. The run gets an accepted identity context. 2. Reassign the task to agent B and let B run. B's identity context has A's context as parent. 3. Call `tools/list` for B's run. Before this change the gateway returns 403 with the message above. The same happens when the parent context belongs to a run on another task (see #17307). **Call path** `handleMcpGatewayProtocol` → `listToolsForNamedGateway` → `listToolsForContext` → `buildToolsForContext` → `connectionToolsForContext` → `githubGuestBotConnectionForSession` → `chatRunSourceMessageId`, which throws `forbidden(...)`. The error is an `HttpError`, not a `ToolGatewayHttpError`, so the route sends it as a bare 403 and the discovery audit never runs. **Paperclip version or commit** `d69631b01b9c658f80bf8bed389863dce82643e1` (image `sha-d69631b`), Docker, authenticated mode with PostgreSQL. ## What Changed - `server/src/services/chat-github-tools.ts`: `githubGuestBotConnectionForSession` first selects the inbound GitHub message links on the task whose delivery carries `githubAuthority.guest = true`, for endpoints assigned to the session agent. When there are none it returns `null` without reading the run or its provenance chain. When there are some, it verifies provenance as before and returns the connection of the link that matches the verified source message. - `server/src/services/chat-channels/run-origin.ts` is unchanged. The cross-agent and cross-task refusals stay in place for Slack authority, GitHub reviews and response comments, and for guest sponsored tasks. - New test `server/src/services/chat-github-tools.test.ts` (embedded PostgreSQL, same pattern as `run-origin.test.ts`): - a run whose chain continues another agent, or another task, with no guest message → `null` (fails before this change with the 403 above, passes after); - a guest sponsored task whose chain continues another agent → still rejects with status 403; - a run sponsored by a GitHub guest → returns the bot connection; the same run with `guest: false` → `null`. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/services/chat-github-tools.test.ts`: 2 failed / 3 passed before the fix, 5 passed after. - `pnpm --filter @paperclipai/server exec vitest run src/services/chat-channels/run-origin.test.ts src/services/chat-github-tools.test.ts`: all passed. - `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && pnpm --filter @paperclipai/server exec tsc --noEmit`: clean. - Manual: with a patched image, a run on a task that was reassigned between two agents receives its full tool list through `tools/list` and a `discovery` audit event is written. Before the patch the same kind of run received the 403 and no audit event. ## Risks - Low risk. The probe's result is unchanged for every session where it returned a value before. The only behavior change is for sessions that previously threw: when the task has no guest sponsored GitHub message, they now get `null`, which is the value the old code could only have returned for them anyway, because the final match also required a guest link on this task for this agent. - One extra query per discovery on guest sponsored tasks. On every other session the run and provenance queries are skipped. ## Model Used - Claude (Anthropic), Claude Code agent session; model id `claude-fable-5-1`, extended thinking, tool use (shell, git, vitest with embedded PostgreSQL). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LoF4rrmx35JJqbhEpaLQ9Q --------- Co-authored-by: Ingvar Petrov <ip@protrener.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
1a79522a4e |
perf: narrow run-to-issue authorization reads (#17405)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Run pages show links to issues associated with a run. > - The issue-link route checks the run's company and issue visibility before it reads those links. > - That check needs four binding fields, but the route reads the full run. > - Full runs can include large prompt, output, and result fields. > - This pull request uses the existing narrow authorization read. > - The route transfers less database data and keeps its permission checks and response contract. ## Linked Issues or Issue Description **Problem or motivation** `GET /api/heartbeat-runs/:runId/issues` calls `heartbeat.getRun` for a permission check. Its consumers only need the run ID, company ID, scope kind, and issue ID. The full read also fetches context, output excerpts, result data, and runtime metadata. Repeated link reads pay for that unused data. **Proposed solution** Use `getRunReadAccess`, the existing four-field read from #17348. Keep authentication, company access, company-scope authorization, run issue visibility, and linked-issue filtering in their current owners. Keep the existing empty response for missing and inaccessible runs. **Alternatives considered** A new query contract is unnecessary because the existing read provides all required fields. A permission cache could become stale. Changing the full run reader could affect consumers that need run details. **Roadmap alignment** This is a focused improvement to an existing route. It adds no feature, route, provider, cache, or background work. Searches for run-issue links, narrow run authorization, and `getRunReadAccess` found no open duplicate. Related work: #17348. ## What Changed - Change the issue-link route to use the existing narrow run read. - Reject full run reads in the route tests. Cover readable company and issue scopes, denied issue access, missing bindings, anonymous requests, and the missing/cross-company empty response. - Add a developer note about the read contract. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/activity-routes.test.ts src/services/heartbeat/run-state.test.ts src/__tests__/agent-live-run-routes.test.ts`: 126 tests passed. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - The route tests use the real run visibility helper with only the four binding fields. The full reader throws if a route calls it. The existing run-state suite checks the exact four-column selection and covers database wiring with disposable Postgres. - `git diff --check`: passed. Reviewed the committed diff for secrets and private identifiers before push. - Apex review: 5/5 on exact head `888a4ce9ea88bb2b284eb5e28fafc01b6f48b386`, completed successfully, with zero review threads. - Full current-head CI passed: 54 successful checks and two expected skips. [Workflow attempt 2](https://github.com/paperclipai/paperclip/actions/runs/40051399577/attempts/2). [Initial serialized shard failure](https://github.com/paperclipai/paperclip/actions/runs/40051399577/job/120744710081): the first company-history case in unchanged `agent-live-run-routes.test.ts` hit its 15-second timeout; the shard had 391 passing tests and one failure. That file, its agent route, the heartbeat state reader, and the service export are unchanged from base. The five company-history cases pass on unchanged base `9fb28d06ffebbb615f413765078aeeb7c4291311` (9.12s suite) and this head (8.29s suite). They also passed in the 126-test focused run. The timeout was not reproduced locally; its cause is not established. Only the failed shard was retried on the same head. It passed in attempt 2. This establishes recovery; the initial timeout remains in this record and its cause is unproven. - The prior local full-suite failures and baseline triage remain documented in #17384. No new local full-suite pass is claimed here. This change uses focused local tests and the full current-head CI gate. - No hosted user journey, staging deployment, production deployment, or billed savings is claimed for this follow-up. ## Risks - Run visibility depends on the canonical `scopeKind` and `issueId`. The existing helper returns those same fields. It does not use prompt or result data for access decisions. - Linked issues still pass the existing SQL visibility filter. Missing and cross-company runs still return `200 []`. Anonymous requests still fail before the run read. - The database query count stays the same. The change reduces data selected for this authorization read. - No schema or API response changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
70de7d49e9 |
Migrate standalone nested clones onto the shared project clone (#17407)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Each task runs in a git worktree. Extra project repositories sit in
the task under `.paperclip-repositories/`.
> - #17382 made new nested checkouts git worktrees of one shared clone
per project repository. Existing standalone nested clones stayed as full
clones.
> - Long-lived tasks keep those clones for a long time. Many tasks hold
the same large repository, so most of the disk is duplicate `.git` data.
> - #17369 added a locked and verified move of isolated workspaces onto
the shared clone, with a reaper sweep behind a setting.
> - This pull request adds the same kind of locked and verified
conversion for standalone nested clones in open tasks, and runs it from
the same reaper path behind a new setting.
> - The benefit is that the duplicate clones go away without losing any
state or local history, and the disk stops filling.
## Linked Issues or Issue Description
Refs #17382 #17369. Bug report described in-PR (bug report template).
**What happened?**
On one instance, 44 open task worktrees in one project each hold a
standalone nested clone of the same large repository. Together they use
about 25 GB. About 16 GB of that is duplicate `.git` data. Another
project has 33 smaller clones. The clones are removed only when their
tasks expire, and many tasks stay open for a long time. The disk keeps
filling.
**Expected behavior**
Existing standalone nested clones in open tasks become worktrees of the
project's shared clone, with the same state and local history, and the
standalone clones are deleted.
**Steps to reproduce**
1. Prepare a task with a nested project repository before #17382, or
without a shared clone, so that `.paperclip-repositories/<name>` is a
standalone clone.
2. Create the project's shared clone, as #17382 does for new tasks.
3. Observe that the old checkout stays a full clone until the task
expires.
**Paperclip version or commit:** `master` at
|
||
|
Nicky Leach
|
aa0037a137 | Support Codex warm plugin task reuse and recovery (#17390) | ||
|
Devin FoleyandPaperclip
|
0c3b5ce273 |
perf: filter sandbox provider lookups in Postgres (#17395)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox execution selects a provider from installed plugin manifests. > - Driver-key lookup reads every plugin row before it finds a matching declaration. > - Repeated lookups transfer manifests for unrelated plugins. > - The registry owns this database read and can filter candidates before returning rows. > - This pull request adds that filter and keeps the existing provider selection checks. > - The returned data now grows with matching providers instead of the full catalog. ## Linked Issues or Issue Description **Problem or motivation** `resolvePluginSandboxProviderDriverByKey` calls the unfiltered registry catalog on each lookup. The caller only needs plugins with a sandbox-provider declaration for one driver key. Unrelated plugin manifests still cross the database connection and are decoded in the server. **Proposed solution** Add an optional sandbox-provider driver-key filter to `pluginRegistryService().list`. Match the key and kind on the same JSON declaration in Postgres. Pass the key from the existing resolver. Keep full matching rows, install order, status checks and worker checks. **Alternatives considered** A catalog cache would need an invalidation contract for plugin updates and independent writers. This change keeps fresh database reads. Returning only one SQL row could skip a later eligible provider when an earlier worker is unavailable. The filter therefore keeps all matching candidates. **Roadmap alignment** This is a small performance fix to existing plugin execution. It adds no product feature or provider. Searches for sandbox/driver lookup and plugin catalog/filter work found no duplicate PR or issue. ## What Changed - Add an optional JSONB containment filter to the existing registry list read. - Use it in sandbox-provider lookup by driver key. The unfiltered catalog call keeps its existing behavior. - Add real-Postgres regression coverage and a short developer note. ## Verification - Four focused provider-selection/recovery/seam suites passed: 27 tests. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - The new Postgres tests seed large unrelated manifests and verify the actual resolver SQL filters rows in the database. They cover key/kind matching within one declaration, unfiltered reads, install order, disabled and unavailable workers, live status/declaration changes, unknown keys and bound unusual input. - The filter keeps the existing SQL call count. It does not add a network call, cache, timer, or background worker. - `git diff --check`: passed. Reviewed the staged diff for secrets and private account data before push. - Full PR CI passed on `6f97c984a931416b6347e9885d840557c54d8aab`: 54 successful checks and two expected skips. [Workflow evidence](https://github.com/paperclipai/paperclip/actions/runs/40041294925). - Apex review: 5/5 on that exact head, completed cleanly, with no comments or unresolved threads. Merge state is clean. - The prior local full run on parent work is documented in #17384. It did not pass because of test setup, host Git-default assumptions and transient failures. Those results were preserved and triaged against unchanged code. This change uses focused local checks and full current-head CI; no new local full-suite pass is claimed. - No hosted user journey, live provider throughput, production deployment or billed savings is claimed. ## Risks - JSONB containment checks the stored manifest before returning rows. It still scans candidate metadata in the database; no index or SQL-call reduction is claimed. - Both key and kind must match the same declaration. The regression fixture includes a plugin with those values on different declarations. - Selection still uses the existing install-order and live readiness/worker checks. Matching disabled records remain visible to callers that do not require a running worker. - No schema, API response, provider protocol, file-copy behavior or scheduling changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
8549ecaf9a |
Re-home isolated repository workspaces onto the shared project clone (#17369)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Each isolated task workspace is a Git worktree on the server disk,
and the terminal workspace reaper and git cache expiration keep that
disk usage bounded
> - For some days, workspace preparation gave each isolated
`git_worktree` workspace its own full clone under
`isolated-repository-sources/`, with the worktree inside it. #16963
removed that creation path, but the existing clones stay, and restoring
an expired workspace still recreates its private clone
> - On one instance, 30 such clones use about 23 GB: 12.1 GB of `.git`
data, 5.2 GB of unused default-branch checkouts and 5.75 GB of real
worktrees. 28 of them back open workspaces, and the disk is nearly full
> - This pull request moves each such open workspace onto the project's
shared managed clone with its exact state, deletes the private clone,
and makes restore use the shared clone
> - The benefit is that disk use drops to one shared clone per project
repository, with no lost work and no change for workspaces that cannot
move safely
## Linked Issues or Issue Description
Refs #16963. Bug report described in-PR (bug report template).
**What happened?**
On one instance, 30 per-issue clones under
`isolated-repository-sources/<companyId>/<issueId>/checkout` use about
23 GB. 28 of them back open workspaces. No process moves these
workspaces to the shared project clone that new workspaces use, and git
cache expiration does not remove a clone while its workspace is open.
Reopening an expired workspace whose recipe has `sourceKind:
"isolated_remote"` clones the full repository again. Separately, a
restore point with a non-default file mode cannot be restored: the
reference read back from jsonb has reordered `modes` keys, so
`loadGitRestorePoint` fails with `restore_point_manifest_mismatch`.
**Expected behavior**
An open workspace with a private clone moves to the shared project clone
with the same branch, HEAD, index, worktree, untracked and ignored
files, and file modes. The private clone is then deleted. Reopening an
expired workspace uses the shared clone when one exists. A restore point
with non-default file modes restores.
**Steps to reproduce**
1. Create an isolated `git_worktree` workspace with
`materializeIsolatedRemoteWorkspaceSource` and record
`isolatedRemoteSource` metadata, as preparation did before #16963.
2. Keep its issue open. The clone stays, because no sweep moves or
removes it.
3. Expire the workspace with a restore point that has a file with mode
`0600`, then reopen it. The private clone is created again, and the
restore fails with `restore_point_manifest_mismatch`.
**Paperclip version or commit:** `master` at
|
||
|
Nicky Leach
|
a978ecadef | Honor negated attachment requests in task completion (#17399) | ||
|
Nicky Leach
|
100477e887 | Avoid live workspace inspection in task context reads (#17396) | ||
|
Devin FoleyandPaperclip
|
5a7e740d82 |
fix: validate managed AI accounts before verification allocation (#17384)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A saved agent must pass execution configuration checks before it can run. > - Managed AI account selection belongs to the existing company-scoped account service. > - Verification allocated its execution environment before it checked that account. > - A missing default or revoked account could create a sandbox on each retry. > - This pull request checks account selection before allocation and keeps the final check. > - Invalid account settings now fail before they cause avoidable provider work. ## Linked Issues or Issue Description **What happened?** Saved-agent lifecycle verification allocates a test execution target before `withManagedAiProbe` selects the managed AI account. An unavailable account can cause sandbox creation and cleanup on every retry. **Expected behavior** Check account compatibility, responsible-user access, default selection, grant health and agent eligibility before target allocation. Check current account access again before the real probe. **Steps to reproduce** 1. Configure a saved OpenCode agent with an OpenRouter personal-default binding. 2. Leave that user's OpenRouter default empty, or revoke the selected grant. 3. Let lifecycle verification use a sandbox execution environment. 4. Before this change, verification can allocate the target and then reject the account. After this change, account selection rejects the configuration before allocation. **Paperclip version or commit** Source baseline `b5bf86ea09042a864b60910f80c8fdf7e64e9ef0`. **Deployment mode** Server with Postgres and a remote execution environment. Tests use disposable local Postgres and a controlled target allocation boundary. Searched public PRs and issues for verification, connection preflight, probes and lifecycle work. No duplicate fix found. Related public issue #16128 concerns credential probing and reauthorization. This change does not alter reauthorization or credential refresh. ## What Changed - Call the existing managed account selector with the resolved adapter and model before allocating a verification execution target. - Keep router resolution, secret handling, final account selection and target release with their existing owners. - Add six Postgres tests and document the saved-agent ordering contract. ## Verification - Focused verification suites: 30 tests passed across two files. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - Full PR CI passed on `ddc0879d46b60310c533e6d3c4b8acdd178d979c`: 54 successful checks and two expected skips. [Workflow evidence](https://github.com/paperclipai/paperclip/actions/runs/40033261909). This includes typecheck, tests, build and browser checks. - Apex review: 5/5 on that exact head, clean check, no comments or unresolved threads. Merge state is clean. - Additional local `pnpm test:run` is still running. The complete local suite is not yet a pass claim. - The first focused run passed the missing-default case but failed five setup hooks because fixtures reused a unique company prefix. Each fixture now has a distinct prefix. The complete focused rerun passed. - Tests use real account selection, memberships, grants, personal defaults, encrypted fixture credentials and managed runtime preparation. Only execution-target allocation and the provider probe are controlled boundaries. No live provider request is required. - Tests prove that missing defaults, revoked grants and incompatible models do not allocate a target. A valid account still reaches the managed probe. Revocation during allocation still fails the final account check and releases the target. Probe failure also releases it. - `git diff --check`: passed. Reviewed the diff for secrets and private account data before push. - No hosted login or production deployment is claimed for this follow-up. ## Risks - Adds one read-only account selection to each managed saved-agent verification attempt. The selector can read company accounts and grants. It makes no provider request and creates no execution target. - Final selection stays fresh. A successful preflight does not reserve an account or cache authorization. - Missing configuration still follows the existing durable lifecycle retry schedule. This change prevents needless allocation; it does not remove those database retries. - No schema, scheduler, provider-account choice, capacity or public API changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
ec2e0d6407 |
Share one clone across nested project repository checkouts (#17382)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A project can attach extra repositories, and every task worktree
gets a managed checkout of each one under
`.paperclip-repositories/<name>`
> - Each of these nested checkouts is a full clone. The clone uses
`--dissociate`, so no objects are shared between tasks
> - On one instance, 44 task worktrees held nested checkouts of the same
repository: 24 GB of 27 GB of worktrees. Each new task added about 0.5
GB, and the disk filled twice in one day
> - Main task worktrees already avoid this, because they are git
worktrees of one managed project clone
> - This pull request makes nested checkouts git worktrees of one shared
clone per project and repository, and adapts expiration, restore points,
restore and cleanup to them without weaker safety checks
> - The benefit is that repository objects are stored once, and a new
task adds only its working files
## Linked Issues or Issue Description
No public issue exists. This is the in-PR bug report.
**What happened?**
Every task in a project with extra repositories got a full, independent
clone of each extra repository under `.paperclip-repositories/`. On one
instance this used 24 GB of 27 GB of task worktrees (16 GB of it `.git`
data) and filled the disk twice in one day.
**Expected behavior**
Nested checkouts of the same repository share one object store, like
main task worktrees share the managed project clone. Disk use per new
task is only its working files.
**Steps to reproduce**
1. Create a project with a primary repository and a second repository
that has a repository URL and no local folder.
2. Run several tasks in isolated git worktrees in that project.
3. Look at `<task worktree>/.paperclip-repositories/<name>/.git`: each
one is a full directory with its own copy of every object.
**Paperclip version or commit:** `master` at
|
||
|
DottaandPaperclip
|
bf223596ab |
fix(runner): bootstrap fresh sessions in replacement sandboxes (#17320)
Reuse the untouched-session proof and atomic directory claim when a sandbox replacement carries a provider session that has never run. Retain verified restore for saved backups and all prior or ambiguous session activity. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
e0d90339bf |
fix(runner): make task-capable plugin environments Runner-only (#17380)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An environment plugin can run Paperclip Runner tasks through a `driver: "plugin"` environment. > - Core lets only the `paperclip_runner` adapter use a plugin environment. > - A Claude agent cannot get onto the Runner there. Runner selection needs a reported platform for Claude, and a plugin target reports none. > - The adapter Test route also checks the driver against the requested harness before it resolves the runner. > - This pull request makes task-capable plugin environments Runner-only and checks the Test driver after runner selection. > - The benefit is that Claude and Codex agents can use a Runner task environment, and the UI cannot produce a legacy agent there. ## Linked Issues or Issue Description Refs #17029. Refs #16978. **What happened?** A Claude Code agent with a plugin environment that runs Runner tasks fails the connection test. The error is: `Environment driver "plugin" is not allowed here. Allowed drivers: local, ssh, sandbox`. The Runner selector in Advanced shows Paperclip Runner as unavailable, so the user cannot fix the agent in the form. **Expected behavior** A Claude or Codex agent on that environment uses Paperclip Runner. The connection test passes the driver check. Legacy is not offered, because a legacy adapter cannot run on a plugin environment. **Steps to reproduce** 1. Install an environment plugin whose driver declares `supportsTasks: true`. 2. Create a plugin environment for that driver. 3. Open an agent with the Claude Code harness and select that environment. 4. Click Test. The test fails with the driver error above. **Agent adapter(s) involved** Claude Code, Codex (through Paperclip Runner). **Additional context** Two gates caused the failure: - `agentRunnerAvailability` requires a platform for `claude_local`. A plugin target is `{ driver: "plugin" }`, so Claude resolves to Legacy. - The Test route ran `assertAdapterTestEnvironmentForCompany` with the URL adapter type before `resolveNewAgentRunnerForCompany`. The plugin driver is allowed only for `paperclip_runner`, so the check failed before selection could change the adapter. ## What Changed - Runner target resolution marks a plugin environment as `runnerTasks` when its plugin manifest declares `supportsTasks` for the driver key and has the `environment.drivers.register` capability. The check reads the manifest only. Run admission still verifies a ready plugin and a live worker. - `agentRunnerAvailability` returns Paperclip Runner as the only runner for a `runnerTasks` target. It skips the platform check, because the plugin provisions the host and launches its own pinned Runner. Harnesses that the Runner does not support keep the Legacy result. - The adapter Test route runs the company-binding check first, as before. It runs the driver check after runner selection, with the resolved adapter type. The driver check still runs before secret resolution, leases, and the adapter probe. - A saved adapter that cannot run on the requested environment no longer keeps its saved execution in the Test route. It re-resolves its runner. - `assertAdapterTestEnvironmentBinding` is a new helper that holds the binding half of the guard. `assertAdapterTestEnvironmentForCompany` keeps its behavior and calls it. - The agent form switches a Legacy runner choice to Paperclip Runner when the selected environment offers no Legacy runner. The test and the save then send the Runner choice. - `docs/specs/agent-config-ui.md` records the Runner-only rule and the Run test order. ## Verification - `pnpm --filter @paperclipai/server exec tsc --noEmit` passes. - `pnpm --filter @paperclipai/ui exec tsc -b` passes. - Server suites pass: `agent-runner-selection.test.ts`, `agent-test-environment-routes.test.ts`, `agent-permissions-routes.test.ts`, `environment-selection-route-guards.test.ts`, `agent-harness-verification.test.ts`, `plugin-runner-task.integration.test.ts`, `adapter-routes.test.ts`, `adapter-auth-signal-routes.test.ts`, `agent-adapter-validation-routes.test.ts`, `agent-hire-ai-connections.test.ts`. - UI suites pass: `AgentConfigForm.render.test.tsx` and `NewAgent.test.tsx`. - New tests cover these cases: - A task-capable plugin target offers only Paperclip Runner for Codex and Claude. - Claude saves as `paperclip_runner` with ACPX on that target. - A driver without `supportsTasks`, a different driver key, a missing capability, an unknown plugin, or a malformed config keeps the existing plugin target. - The Test route checks the driver against the resolved adapter. - The Test route still rejects a plugin environment for a legacy adapter, before any secret resolution. - A saved adapter re-resolves only when it cannot run on the requested environment. - The form switches a saved or new Legacy choice to Paperclip Runner on a Runner-only environment. - I ran the new route and form tests against the old code. They fail without the change. - The full server `typecheck` script did not complete locally. Its Runner vendor step needs `cargo`, which this machine does not have. CI runs the full script. - I did not run a live plugin environment end to end. ## Risks - Behavior change: Codex on a task-capable plugin environment now offers only Paperclip Runner. Before, the selector also showed Legacy, but Core rejected a legacy save on that environment. No working setup loses an option. - The `runnerTasks` decision reads the plugin manifest. If the plugin is down, the agent still selects the Runner, and the run fails at admission with the existing plugin error. - The adapter auth-signal route still checks the driver with the requested harness. This PR does not change that route. - An explicit Legacy save through the API on a Runner-only environment still fails with the existing driver error. The UI no longer offers that choice. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) wrote the change and the tests in Claude Code, with shell, file editing, and local test runs. - Claude Fable 5.1 (`claude-fable-5-1`) did the earlier diagnosis in the same session. - The session does not expose the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
b5bf86ea09 |
Clear process identity when settled native runs stop (#17374)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Each heartbeat run stores the process identity of its runner in
`heartbeat_runs.process_pid`, `process_group_id` and
`process_started_at`
> - Only cancellations (`release_run_resources`), restart recovery and
heartbeat recovery cleared these fields
> - Failed native runs take the `schedule_retry` path, so they kept a
PID after the run finished, for weeks
> - Liveness checks for old runs then depend on PID-reuse defenses, and
code that reads a set PID as "running" is misled
> - This pull request releases the process identity of settled native
runs when the process is confirmed gone, and backfills old sandbox rows
> - The benefit is that a finished run no longer looks like it owns a
live process, while live, warm and retrying processes keep their
identity
## Linked Issues or Issue Description
Refs #17358. Bug report described in-PR (bug report template).
**What happened?**
On one instance, 16 of 23 old native workspace sync folders belonged to
failed or cancelled runs that finished more than a week earlier and
still had `process_pid` set. The PID is written when the run's process
starts (for remote targets it is the sandbox PID). The status arbiter
clears it only through `release_run_resources`, which only cancellations
emit. Failed native runs take `schedule_retry`, and other terminal paths
(finalization reconciler, restart recovery terminal failures, watchdog
timeouts) do not clear it either.
**Expected behavior**
When a run is terminal and its process is confirmed gone (or no longer
owned), the process identity is cleared. A process that may still be
alive keeps its identity.
**Steps to reproduce**
1. Run a native task whose provider turn fails, so the arbiter decides
`run_failed_partial_evidence_preserved` with a `schedule_retry` effect.
2. Wait for the run to finish and the runner to exit.
3. Read the run row: `status = 'failed'` and `process_pid` /
`process_group_id` / `process_started_at` are still set.
**Paperclip version or commit:** `master` at
|
||
|
Devin FoleyandPaperclip
|
4660b22f6b |
perf: narrow run bindings during event ingestion (#17378)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs append native events and retry-exhaustion receipts. > - These writes lock the run row and verify its company and agent bindings. > - The binding check fetched the full run, including prompts, configuration, and output. > - This pull request reads only the two fields that the check uses. > - The benefit is less database transfer while the same row lock protects sequence allocation and replay checks. ## Linked Issues or Issue Description Related: #17355 and #17348 reduce other run-read payloads. GitHub searches for event-read and narrow-run PRs found no duplicate of this event-ingestion change. **What existing behavior does this improve?** The run binding read during native event and retry-exhaustion ingestion. **Current behavior** The transaction uses a full-row `SELECT ... FOR UPDATE` to check only `companyId` and `agentId`. Large run context, runner configuration, and output cross the database connection for each such check. **Proposed behavior** Select only the company and agent IDs. Keep the same run ID predicate, row lock, transaction, sequence allocator, replay cursor, and binding checks. **Reason and benefit** The check does not use the other fields. Narrowing the query removes that repeated transfer without a cache or an authorization change. **Breaking changes** None. No API response, schema, event, or scheduling changes. ## What Changed - Narrow the existing run-row read inside `appendHeartbeatRunEvent`. The event service continues to own its transaction and binding validation. - Exercise native and retry-exhaustion writes and replays with large stored payloads. Record the executed SQL to prove the lock reads only the two binding fields. - Extend missing-run and company/agent mismatch coverage to both transaction paths. Keep the existing concurrent-writer and rollback tests. - Document the read boundary. ## Verification - Focused PostgreSQL event append and sequencing suites: 9 tests passed. They include concurrent writers, replay conflicts, rollback, cursor checks, large payloads, and company/agent/missing-run denials. - The first new retry-exhaustion fixture failed its duplicate assertion because it omitted the exhaustion fields from the event payload. The existing producer contract stores those fields in the payload. The fixture now does the same; the rerun passed. No production retry or replay logic changed. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - Greptile Apex: 5/5 on `76a6d3ee813ae47919b1ec8e8b88e01e58d5655d`, clean check and no review threads. The [full PR workflow](https://github.com/paperclipai/paperclip/actions/runs/40028758665) passed on that exact head: 54 checks passed and two conditional checks were skipped. The branch is conflict-free. - The full local all-in-one suite is not repeated for this follow-up. Full exact-head CI will supply the broader suite coverage. - No hosted deployment or bill reduction is claimed for this follow-up. ## Risks - Low scope: only the selected columns change. The `FOR UPDATE` lock, transaction boundaries, sequence reservation, duplicate checks, and failure handling remain in place. - Native replay and legacy event writers must still serialize on the same row. Existing real PostgreSQL concurrency tests cover that boundary. - No migration, provider change, new timer, or operator configuration is required. ## Model Used OpenAI Codex (GPT-6 family), with code execution and repository tools. The runtime does not expose the exact model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
a3e8fa568a |
Reduce repeated work during plugin Runner admission (#17377)
## Thinking Path > - Paperclip manages agents and their execution authority. > - A plugin can host Runner tasks on a persistent execution host. > - Admission checks must observe current lifecycle state and credentials. > - The current runtime reader also resolves files, skills, and connections for each check. > - Some providers also repeat the same project authentication check during submit. > - This pull request adds a compact fresh reader and an explicit submit-time validation contract. > - Providers can remove repeated work while Core keeps lease and scope checks. ## Linked Issues or Issue Description **What existing behavior does this improve?** Plugin Runner startup before task submission. **Current behavior** One measured startup made 30 full runtime reads during bootstrap. These reads took 5.759 seconds. Each guard can resolve instructions, skills, and connection assignments even when it only needs current identity and lifecycle state. Providers that already enforce project authentication in submit must also perform a separate Core preflight. **Proposed behavior** Use a lifecycle-owned runtime identity reader for guards. Keep the full runtime reader for synchronization inputs. A trusted provider can declare `projectAuthCheckAtSubmit: true` in its prepared result. Core pins the exact configuration, provider, and project set on the lease. The provider must reject authentication conflicts before credential installation or task start. Providers that omit the flag retain the existing preflight. **Reason and benefit** Estimated savings with a paired consumer: 4–5 seconds from compact runtime checks, plus roughly 0.5–0.9 seconds from removing the duplicate authentication operation. These estimates overlap and are not a measured deployed result. Refs #17367 for the separate run-log HEAD optimization; this PR does not duplicate it. **Breaking changes** No migration. The SDK adds a host method and an optional prepared-result field. Consumers that use them require this Core host version. Older providers retain their existing flow. ## What Changed - Add `agents.getRuntimeIdentity` across SDK, RPC, host service, and test fixtures. - Reuse the existing company, plugin ownership, readiness, and capability checks. Read lifecycle state and protected instance secrets fresh on every call. - Skip instruction, organization, skill, and connection hydration in the compact reader. - Allow explicit submit-time authentication enforcement, with prepared scope pinned and checked again at dispatch. - Emit fixed preparation, project-auth, submit, and connection timing phases. ## Verification - 72 targeted SDK capability, plugin task-owner, and environment-task mediator tests passed. - 33 real embedded-Postgres lifecycle and secret tests passed, including pause, content revision, company isolation, plugin disablement, and secret rotation. - SDK and server TypeScript checks passed. Local Runner/eval TypeScript outputs were built to satisfy server type resolution. - Local Postgres required missing library aliases in ignored dependencies; no product source was changed for that repair. - Full repository suites are delegated to GitHub CI. No live rollout or speedup claim. ## Risks The submit-time flag delegates enforcement to a trusted provider. It is not proof that prepare already checked authentication. The provider must check before VM credential installation or task admission. Core still checks exact prepared scope, active lease, run identity, credential type, and dispatch fencing. Existing providers keep the separate check. Roll back a paired consumer before rolling back this host. ## Model Used OpenAI GPT-6 (Codex), with code execution and repository tools. Exact deployment build and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3c4050e3ad |
fix(connections): keep Cloud refresh reauthorization diagnostics (#17362)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed connections (Gmail, Google Workspace, GitHub, Asana, Slack manager) get and refresh their OAuth tokens through the Paperclip Cloud connector broker. > - When a provider refuses a refresh grant (the user revoked it, it expired, it was already rotated, or it no longer matches the profile), the broker sent `502 PROVIDER_OPERATION_FAILED`. That is the same answer it sends for a provider outage. > - Paperclip then kept the grant `active`. Each health check, catalog refresh, or catalog read failed again with a 502, and the server reported each failure as a crash. The user did not get a reconnect prompt. > - The broker now sends `409 PROVIDER_REAUTHORIZATION_REQUIRED` for a dead grant. The connector already maps any 409 to `REAUTHORIZATION_REQUIRED`. Thus the grant becomes `needs_reauthorization` and the caller gets a 422 reconnect response. > - This pull request adds the new reason to the diagnostic allowlist, pins the producer contract, and tests the reconnect path from end to end. > - The benefit is that a dead managed grant asks the user to reconnect. It is not reported as a server crash. Real provider outages keep their 502 response. ## Linked Issues or Issue Description **What happened?** A managed Paperclip Cloud connection refresh failed with `Paperclip Cloud connector rejected the request (operation=refresh, status=502, reason=PROVIDER_OPERATION_FAILED)`. The connection health-check and catalog routes wrap that error as an HTTP 502. The error handler reports every status of 500 or more, so the failure went to error monitoring. The grant stayed `active`, so the same failure can occur again on each use. **Expected behavior** If the provider refuses the refresh grant, Paperclip marks the grant `needs_reauthorization` and shows the reconnect path. Only a real provider or Cloud outage gives a 502. **Steps to reproduce** 1. Connect a managed Gmail connection through Paperclip Cloud. 2. Revoke the app access in the Google account, or let the refresh grant expire. 3. Wait until the access token needs a refresh. Then run a connection health check. 4. Before the broker change, the health check returns 502 and the server reports it. The grant stays `active`. **Additional context** The broker-side classification ships separately in Paperclip Cloud. This pull request is safe before or after that change, because the connector already maps any broker 409 to `REAUTHORIZATION_REQUIRED`. Without this pull request, the 409 still works, but diagnostics show `reason=UNKNOWN_BROKER_ERROR`. ## What Changed - `server/src/services/paperclip-cloud-connector.ts`: Add `PROVIDER_REAUTHORIZATION_REQUIRED` to the allowlist of broker reasons that errors can keep. - `server/src/services/fixtures/cloud-connector-refresh-reauthorization.json`: Add the producer HTTP contract (status 409 and the exact JSON error body). - `server/src/services/paperclip-cloud-connector.test.ts`: Test that a refresh with this contract gives `REAUTHORIZATION_REQUIRED`, status 409, the retained reason, and no refresh-token text. - `server/src/__tests__/tool-access-service.test.ts`: Test that `refreshOAuthGrantCredentials` changes a reauthorization refresh failure into a 422 `oauth_reauthorization_required` response and sets the grant to `needs_reauthorization`. - `doc/connections/GMAIL.md`: Add a troubleshooting row that tells the two refresh reasons apart. ## Verification - `cd server && npx vitest run src/services/paperclip-cloud-connector.test.ts`: 48 passed. Without the allowlist change, the new test fails (`reason=UNKNOWN_BROKER_ERROR`). - `cd server && npx vitest run src/__tests__/tool-access-service.test.ts -t "managed grant for reconnection|orders managed reconnect"`: 2 passed, against the embedded test database. - I did not run the full `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` locally. The sandbox has a 4 GB memory limit, and the server `tsc` process was stopped because it used too much memory. CI runs these checks. ## Risks - Low risk. The status mapping does not change: a 409 already meant `REAUTHORIZATION_REQUIRED`. This change only keeps one more fixed reason code in the error message. - The reason allowlist accepts fixed protocol codes only. The broker message, request data, and tokens stay out of errors and logs. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code agent mode with tool use (shell, file edit, code search, and a sub-agent for the code-path trace). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky Leach
|
2e38649341 | perf(run-logs): skip durable log restore for a proven-new run (#17367) | ||
|
|
75fcb0b3fc |
fix(sentry): keep typed ACP provider access failures local (#17366)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The server sends unexpected run failures to Sentry through
`reportRunFailure`
> - A Claude ACP run can fail because the provider rejects the stored
credential ("Invalid API key"). The ACP agent then sends a typed
`access` terminal failure.
> - The Claude adapter already classifies this failure as
`claude_auth_required`. The run is blocked, and the responsible user
gets a reconnect request.
> - Sentry also records each of these runs as "ACP agent reported a
terminal access failure.". This duplicates a signal that the user
already gets, and it hides real defects.
> - This pull request adds one narrow exclusion for this proven provider
auth state, next to the existing quota and Codex capacity exclusions.
> - The benefit is less Sentry noise, and all other access-shaped
failures stay visible.
## Linked Issues or Issue Description
Refs #14886 (related: it routes ACP access failures in recovery, not in
Sentry reporting).
**What happened?**
Staging Sentry recorded a run failure "ACP agent reported a terminal
access failure." for 8 events. All events had
`agent_adapter=claude_local`, `error_code=claude_auth_required`, and a
typed provider access failure ("Invalid API key · Fix external API
key"). The failures stopped when the provider accepted the credential
again.
**Expected behavior**
A provider auth failure that the adapter types and classifies is a user
action state. The product already shows it to the user. It must not go
to Sentry in `unexpected` mode.
**Steps to reproduce**
1. Configure a `claude_local` agent with an ACP runtime and a credential
that the provider rejects.
2. Start a run.
3. The ACP agent ends the turn with `category: "access"`. The run fails
with `claude_auth_required`.
4. See a new Sentry run-failure event.
**Agent adapter(s) involved**
Claude Code (`claude_local`, ACP lane).
## What Changed
- `server/src/services/run-failure-report.ts`: add
`isAcpProviderAccessFailure`. It skips a report only when all of these
are true: the reporting mode is `unexpected`, the run status is
`failed`, the error code is `claude_auth_required` or
`acpx_auth_required`, `resultJson.terminalSessionFailure.category` is
`access`, the phase is not `setup`, and the run has no signal.
- `server/src/services/__tests__/run-failure-report.test.ts`: add
regression tests for the staging shape, eight near-miss shapes that must
still report, and the setup and `all` mode paths.
## Verification
- Red: without the predicate, the two new "keeps ... local" tests fail
(`2 failed | 266 passed`).
- Green: `cd server && pnpm exec vitest run
src/services/__tests__/run-failure-report.test.ts
src/services/__tests__/run-failure-diagnostics.test.ts` gives `383
passed`.
- `pnpm exec tsc --noEmit -p server` shows no errors in the changed
files.
## Risks
- Low risk. The change only removes Sentry events. It does not change
run status, recovery, or user notices.
- A real defect that the ACP agent wrongly types as `access` with a
Claude auth code is not reported in `unexpected` mode. Set
`PAPERCLIP_SENTRY_RUN_FAILURES=all` to see these events again.
- Codex and other ACP adapters do not map `access` to an auth code, so
their access failures still report.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), run through Claude Code
with tool use (shell, file edit, Sentry and GitHub connectors).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Devin FoleyandPaperclip
|
f73ace9f1a |
Clear stale native recovery state so closed work expires (#17359)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs keep a finalization row, and tasks can keep recovery
actions while work is stuck
> - Git cache expiration and native harness backup expiration read these
rows to decide if closed work is still live
> - Commit paths never cleared
`native_run_finalizations.recovery_state`, so committed rows kept
`resuming_session` forever
> - Recovery actions on done or cancelled tasks also stayed active when
the close did not go through the issue routes
> - So workspaces and backups of tasks that closed weeks ago stayed live
and used disk space
> - This pull request clears the stale state at commit, backfills old
rows, resolves stale recovery actions on closed tasks, and lets the two
expiration jobs ignore settled state on closed trees
> - The benefit is that closed work expires on schedule, while live
leases, unsettled phases, unverified owners, and in-flight system
recovery still keep it
## Linked Issues or Issue Description
Refs #17248
**What happened?**
On one instance, Git cache expiration and native harness backup
expiration kept the workspaces and backups of tasks that were done for
more than 7 days. Three kinds of stale state caused this:
1. 51 `native_run_finalizations` rows had `phase = 'committed'` and
`recovery_state = 'resuming_session'`. The commit writes (status
decision commit, attention replay, result persistence, and the finalizer
claim) never cleared `recovery_state`. No reader uses `recovery_state`
on a row that has a result, except the expiration predicates.
2. Rows with `phase = 'terminal_failure'` and `recovery_state =
'blocked'` (or NULL) are permanent states. Nothing clears them, and
continuations are refused for closed tasks.
3. `issue_recovery_actions` with status `active` or `escalated` stayed
open after the source task became done or cancelled. Only the issue
routes cancel them at close. The periodic sweep
`reconcileActiveRecoveryActions` skipped every action unless
`wakePolicy.type` was a bounded owner policy or the owner was the board.
Agent-owned `resume_native_run` actions (which use `wakePolicy.kind`)
and system-owned actions were never resolved.
**Expected behavior**
When a task tree is closed for the configured time, its Git cache and
native backup expire. Only live state keeps them: a held finalization
lease, an unsettled finalization phase, an unverified native workspace
owner, or system recovery that is scheduled, running, or awaiting
evidence.
**Steps to reproduce**
1. Run a native task. Let the runner restart so that restart recovery
sets `recovery_state = 'resuming_session'` on the finalization row.
2. Let the resumed run commit its result and close the task as done.
3. Wait past the expiration window (or run the sweep with a later
clock).
4. See that the finalization row is still `committed` +
`resuming_session`, and that Git cache expiration and native backup
expiration skip the closed workspace.
**Paperclip version or commit**
`master` at
|
||
|
Devin FoleyandPaperclip
|
dfe7df0015 |
fix(native): expire settled native workspace sync folders (#17358)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs sync their workspace with a remote sandbox through a
per-run folder under `<instance>/native-workspace-sync/<run>/`
> - The folder holds a descriptor, a manifest, and workspace and git
seed archives, and a seed can be more than a gigabyte
> - Only a few finalization commit paths delete the folder, so many runs
leave it on disk forever
> - Instance disks fill up with folders that no code will read again
> - This pull request adds a bounded, deadline-driven retention worker
that removes a folder only when no reader can still need it
> - The benefit is that instance disks stop growing with stale sync
folders, and recovery and export paths keep their files
## Linked Issues or Issue Description
**What happened?**
`cleanupNativeWorkspaceSync(runId)` is the only code that deletes
`native-workspace-sync/<run>/`. It runs only when finalization commits
through specific paths. Runs that end in `terminal_failure`, retained
recovery, reconstruction, superseded runs, commits through other paths,
and server crashes leave the folder forever. On one instance, 22 folders
older than 7 days held 6.8 GB. Single seed archives were up to 1.4 GB.
**Expected behavior**
A run's sync folder is removed after a retention window, once the run
and its finalization are settled and no recovery or export path can
still read it. The shared `generations` seed cache stays.
**Steps to reproduce**
1. Run a native agent with a remote sandbox and let the run end in
`terminal_failure` (or crash the server during finalization).
2. Wait more than 7 days.
3. Look in `<instance>/native-workspace-sync/`. The run's folder and its
seed archives are still there.
**Paperclip version or commit**
`master` at
|
||
|
DottaandPaperclip
|
4d90634f74 |
feat(connections): add 1Password service-account connector (#17249)
## Thinking Path > - Paperclip lets people manage AI agents and govern their access to outside services. > - Apps connections keep provider credentials in the instance vault and mediate every action through grants, policies, and audit. > - Agents currently have no Apps connection for reading or managing 1Password items with an operator-supplied service-account token. > - 1Password supports this through its official SDK; its documentation MCP server does not expose vault operations. > - This pull request adds a one-field service-account connection and registers eight reviewed SDK actions through the existing native-provider catalog. > - Operators can select vault permissions in 1Password and agent/action access in Paperclip without installing a CLI or handing durable credentials to agents. ## Linked Issues or Issue Description Related: #16144 adds 1Password as Paperclip's secret-storage backend. This PR adds the separate Apps/tool-access flow with a per-connection credential; it does not change secret providers or use a server-wide ambient token. **Subsystem affected** Apps catalog, native tool-provider registry, and shared connection setup. **Problem or motivation** People need to paste their own `OP_SERVICE_ACCOUNT_TOKEN`, find official service-account setup instructions, and authorize agents to use the selected vaults through ordinary Paperclip governance. **Proposed solution** Add `1password/service-account`, with a password field labeled **Paste your 1Password Service Account token** and a link to [1Password's creation instructions](https://www.1password.dev/service-accounts/get-started#create-a-service-account). Use the pinned official `@1password/sdk` 0.5.0 for vault/item listing, item and secret-reference reads, and item create/update/archive/delete. **Alternatives considered** A remote MCP card would not provide vault access. A local CLI or a third-party MCP package adds installation and credential surfaces that the official SDK avoids. ## What Changed - Added the source and generated catalog definition, permission review, official light/dark artwork, documentation, and capability-map references. - Added a provider-owned SDK bridge with strict schemas, account-domain validation, contract hashes, fixed errors, bounded output, and disposable workers with cancellation/deadlines. - Registered the provider through master's native-provider catalog; existing setup, health, discovery, gateway, policy, and audit owners remain in place. - Labeled the single password field **Paste your 1Password Service Account token**. The shared formatter preserves explicit credential prompts; ordinary key labels retain their existing presentation. - Added credential-ownership, gateway read/write, policy/revocation, worker, setup UI, and interactive Storybook coverage. ## Verification - Final review follow-up: added two deterministic worker-entry tests for nested exact-token redaction and oversized-output rejection. Removing each guard caused exactly its regression test to fail. The focused provider suite passed **15 tests**, and the official server typecheck passed after rebuilding stale workspace SDK artifacts. Current head `569cb7f9bd57db791fb13a6034ce69bf50878cb1` passed the full [CI run](https://github.com/paperclipai/paperclip/actions/runs/40016166260), including typecheck, build, unit/integration tests, and all eight browser shards. The required `ci / verify` and `ci / e2e` gates passed. [Greptile](https://github.com/paperclipai/paperclip/pull/17249#issuecomment-6353731805) and [Paperclip Review](https://github.com/paperclipai/paperclip/pull/17249#issuecomment-6372165895) each rated this exact head **5/5**, with no open findings. Paperclip Review completed with a neutral check because its scoped reviewer could not inspect CI; the full CI result was verified separately. Other neutral checks and intentionally skipped Storybook jobs contain no actionable findings. - CI recovery on the same commit: the first attempt lost six jobs. Five logged simultaneous runner shutdowns at 18:49 UTC on 2026-10-23; the sixth stopped logging before a test result and its root cause was not established. No assertion failure appeared in the retrieved logs. All six jobs and the aggregate gates passed one failed-job retry. [CI run](https://github.com/paperclipai/paperclip/actions/runs/40005754936). - Copy follow-up: the catalog, provider, and setup UI suites passed **275 tests**. The governed gateway lifecycle passed **3 tests** (one live-only test skipped), including create, update, archive, and delete. UI token gates and map checks passed. The branch was rebased onto `9c442cf48e` to resolve additive catalog conflicts while retaining the Vercel and Supabase entries. The existing live write proof below remains applicable; this follow-up changes presentation only. - Five affected suites, including the existing Daytona consumer, passed **290 tests** (one opt-in live test skipped in fixture mode). Full `pnpm -r typecheck` and `pnpm build` passed. The local broad-suite attempt could not complete because PostgreSQL semaphore exhaustion prevented a competing integration suite from starting; the complete affected-file retry passed after disposable-process cleanup. The prior commit `f208a8aff6dc28859cc3f343497e18d85f124579` passed the complete CI matrix. All CI build, typecheck, test, and browser gates also passed on the earlier commit `9e2f3d3140368a08da43b004b2e8d3a0f4c0fa4d`. The separate Paperclip Review reported a worker-result regression-test gap on that head; the follow-up above addresses it. Its review thread is resolved, and both fresh current-head reviews are clean as recorded above. - Token gates, module boundaries, brand validation, map checks, and Storybook catalog checks passed. - Direct SDK create/archive/delete and rejected-call coverage passed (13 provider tests); the Storybook Permissions/Test play completed successfully in Chromium. Server and UI typechecks passed after those fixes. Greptile passed prior commit `9e2f3d3140368a08da43b004b2e8d3a0f4c0fa4d` at **5/5**. Both reviewers also passed the worker-test follow-up at **5/5** as recorded above. - Live 1Password proof on 2026-10-22: connect, eight-action discovery, vault listing through an actual run-scoped gateway, refresh, reconnect preserving identity, another read, removal, and denial through the retained session passed. Credential absence was asserted in responses/audit; an exact-token scan of inspected logs and changed files passed. - Real browser: Connectors → 1Password → token entry → personal access limited to one disposable agent → completion → Permissions → Test `onepassword_list_vaults` succeeded. The disposable connection was removed afterward. - Compiled-worker smoke initially exposed inherited Node launch flags. Workers now use explicit empty `execArgv`; the compiled worker then authenticated successfully and listed one vault. - Live write proof on **2026-10-23**: the replacement Keychain token passed all eight actions through the actual run-scoped gateway for both personal and shared credentials. The test created, listed, read, updated, resolved a secret from, archived, and deleted two synthetic items. Both were absent from subsequent item lists. Refresh/reconnect, identity preservation, removal, retained-session denial, and credential absence passed. The live suite passed **3 tests** (one fixture-only test skipped) in 116.65 seconds. The expanded fixture suite passed **3 tests** (one live-only test skipped); the 13 provider tests and server typecheck passed. An exact-token scan passed over 64 source/log files. - Failure history: the original token denied creation on 2026-10-22 and created no item. The first 2026-10-23 attempt stopped before provider calls because host PostgreSQL semaphore limits prevented test-database startup. A temporary ten-connection test profile allowed the complete retry to pass without changing shared services or host limits. A new delete-result assertion also exposed an outdated fixture response (`success` instead of `deleted`); the fixture was corrected and the complete fixture suite passed. Official contract sources: [service accounts](https://www.1password.dev/service-accounts/get-started), [SDKs](https://www.1password.dev/sdks), [item management](https://www.1password.dev/sdks/manage-items), [official JavaScript SDK](https://github.com/1Password/onepassword-sdk-js). ## Risks - Authorized item/secret reads intentionally return secret values to the agent and ordinary tool-result history. Select trusted agents and use restricted service-account vaults; the connection token itself is redacted. - New connections retain existing Paperclip action defaults, including Allowed writes/destructive actions for selected agents. 1Password still enforces the token's permissions. - Service-account access cannot be changed in place. Create a replacement account and reconnect to change vault permissions; read-only keys can connect but cannot write. - SDK authentication happens per action in a disposable worker. Calls have a 30-second maximum; cancellation stops local work, while a mutation already accepted by 1Password can still finish. Writes are never automatically retried. - Additive dependency and catalog change; no database migration. Disable/remove the connection to withdraw access, or remove the provider from catalog visibility for rollback. ## Model Used OpenAI Codex, based on GPT-6, with code editing, shell/browser tools, and live provider verification. A more specific runtime model ID and context-window limit were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
ef9e17304c |
perf: narrow runtime progress database reads (#17355)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Active runs publish workspace and provider progress. > - Each update must check the current run status before it publishes. > - This check fetched the full run, including prompts and runner configuration. > - This pull request reads only the four fields that progress publication uses. > - The benefit is less database transfer while each update still checks durable state. ## Linked Issues or Issue Description Related: #17348 narrows authorization reads. This PR applies a separate projection to runtime progress. GitHub searches for runtime progress and run-status read PRs found no duplicate. **What existing behavior does this improve?** The database read before a runtime progress update. **Current behavior** Each update calls `getRun`. This reads large run context and runner configuration even though progress publication only uses the run ID, company ID, agent ID, and status. **Proposed behavior** Use a fresh four-column read for each update. Keep the existing active-status check, event fields, error propagation, and late-progress suppression. **Reason and benefit** Transfer progress can emit many updates for one run. A small status check avoids repeatedly transferring payloads that it does not use. **Breaking changes** None. No API response, schema, scheduling, or cancellation policy changes. ## What Changed - Add `getRunStatus` to the run-state owner. The full reader remains available to consumers that need execution data. - Wire the narrow reader into the lifecycle progress check. Remove the unused context field from that check's input contract. - Cover the exact selected fields, missing runs, database errors, large stored payloads, terminal transitions, and the public heartbeat service wiring. Document the read boundary. ## Verification - Run-state and lifecycle suites: 44 tests passed before the additional public-service regression. - Lifecycle, public-service wiring, and native cancellation suites: 36 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - The [full PR workflow](https://github.com/paperclipai/paperclip/actions/runs/40013092548) passed on `95bf496f689049812646b5ccddcdf2cca21efad0`. All 54 reported checks passed; two conditional checks were skipped. This includes test shards, browser checks, Runner verification, production bundle boot, and clean npm installation. - `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master pnpm test:run`: an additional local all-in-one run is still in progress; no full-local-pass claim. The process-scoped Git default matches existing master-only fixtures and does not change user configuration. - The initial description check failed because the dedup-search checkbox was missing. Searches are complete and the template checkbox is now included. The rerun passed on the same source commit. - Greptile Apex: 5/5 on `95bf496f689049812646b5ccddcdf2cca21efad0`, no findings or review threads. - No hosted deployment or bill reduction is claimed. CI and Apex are complete; this PR is merge-ready. ## Risks - Low scope: the query selects fewer columns and preserves the existing status read on every progress callback. It adds no worker, cache, timer, or idle query. - A run can still finish after the status read and before publication, as it could before this change. This PR does not change that concurrency boundary. - No migration or operator configuration is required. ## Model Used OpenAI Codex (GPT-6 family). The runtime does not expose the exact model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites above; full suite is still running) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
837add0caa |
fix(recovery): deliver review-path repair instructions to agent prompts (#17353)
## Thinking Path > - Paperclip keeps agent tasks moving through owned execution and durable waiting paths. > - A consumed review path gets one repair wake before board escalation. > - That wake records an instruction to save a new monitor or another real review path. > - The canonical wake builder omitted that instruction from the agent prompt. > - This pull request delivers it through the existing disposition-repair section. > - The repair limit and execution authority stay unchanged. ## Linked Issues or Issue Description **What happened?** An agent opened a pull request and promised to watch CI and external review. It ended without scheduling a monitor. Its bounded review repair also ended without a maintained path, so the task became blocked for a board decision. The saved repair context contained the instruction, but the canonical wake payload and rendered prompt did not. **Expected behavior** The repair prompt tells the agent to save a one-shot monitor before promising another check. Both fresh and resumed sessions receive that instruction and the one-attempt limit. **Steps to reproduce** 1. Create a review-path recovery with `decideIssueReviewPathRecovery` for a task whose review path was consumed. 2. Pass its recorded context through `buildPaperclipWakePayload` and wake serialization. 3. Render a fresh or resumed prompt. Before this fix, it has the wake reason but no repair instruction. **Paperclip version or commit** Base: `9c442cf48e6`. Current head: `f38d65891c8d9f2b38eb76a8578e870debc2b1e6`. **Deployment mode** Observed on managed sandbox execution with the legacy Claude ACP adapter. The corrected canonical payload serves the shared adapter prompt path. Refs #15764 introduced the bounded exhausted-repair disposition. This change preserves that limit. Open monitor-retry work such as #17171 addresses an already scheduled check; this defect occurs before a check exists. Searches found no duplicate for this missing prompt field. ## What Changed - Build the review-path repair disposition in its owning recovery module, using the server instruction and fixed one-attempt bound. - Project it into the existing canonical `dispositionRepair` field. No new adapter field or execution protocol is added. - Ignore arbitrary replacement instructions and attempt-limit values. A pending repair keeps its instruction across a connection-pool retry that changes the wake reason. Ordinary wakes without the typed repair marker ignore raw instruction text. Existing legacy disposition repairs retain their values. - Add a regression across the real recovery decision, wake builder, serializer and fresh/resumed prompt selectors. Update execution guidance and the affected recovery maps. ## Verification - Verified red before the source fix: two regression cases failed because the instruction was absent; the sibling repair case passed. The same cases pass after the fix. - Passed 194 focused and safety tests across wake preparation, review recovery, adapter prompt handling, and database-backed liveness escalation. The connection-pool retry case also failed before its correction and passes after it. - Replayed the recorded incident wake through the fixed boundary. The old canonical prompt lacked its saved instruction. The corrected serialized prompt includes it with a 1/1 bound. This replay did not execute a provider turn or change the source task. Private evidence stays outside the repository. - Passed `pnpm -r typecheck` after the pool-retry correction. Passed `pnpm build` on final head `f38d65891c8d9f2b38eb76a8578e870debc2b1e6`. Map checks and `git diff --check` pass. - [Full CI](https://github.com/paperclipai/paperclip/actions/runs/40005890857) passed on that head: 54 checks passed and two optional checks were skipped. This includes the full test shards, typecheck, build, and browser suites. The local `pnpm test:run` attempt was stopped after the CI test shards passed. No complete local full-suite pass is claimed. - [Apex review](https://github.com/paperclipai/paperclip/pull/17353#issuecomment-6372160694) is 5/5 on the final head, with no open findings. The first review found the pool-retry case. Its regression and correction are included. The branch has no merge conflicts. - Test setup first required the plugin SDK build, then an empty credential-history database boundary. Those setup errors are not the claimed red. The temporary recorded-wake probe was removed before final validation. - Hosted follow-up acceptance is not run. The source task remains blocked and needs a retry after rollout. This PR does not claim that prompt delivery itself schedules a monitor or that every model will follow the instruction. ## Risks - This fixes instruction delivery. The agent must still save a valid monitor or another waiting path. The existing bounded escalation remains if it does not. - The wake stays a normal-model repair. It does not become status-only recovery and can continue useful work. Existing ownership, approval, budget, pause, and monitor admission gates remain in place. - No schema, credential, telemetry, API, model, or retry-budget change. ## Model Used OpenAI Codex, based on GPT-6, assisted with diagnosis, code editing, shell execution and verification. The exact serving model ID and context-window limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (194 focused and safety cases; full coverage passed in CI) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9c442cf48e |
feat(connections): deploy and configure Supabase and Vercel (#17217)
Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations. Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior. Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
f406b32d08 |
perf: narrow run authorization reads (#17348)
## Thinking Path > - Paperclip lets people manage and observe AI agents at work. > - Live events and run history must enforce current task permissions. > - Some permission checks load the complete run record for each event or request. > - Run context, runner configuration, results, and output can make those records large. > - This pull request reads only the persisted fields each access check needs. > - The checks still read current permissions and fail closed. ## Linked Issues or Issue Description Related: #17251, #9155, and #16229. Those changes cover other run projections, recovery polls, and task history. This change covers live-event delivery, workspace-operation run checks, and event-list access. **What happened?** Live-event delivery loaded every run column for each subscribed viewer, although the permission helper used only the company, run scope, and task binding. Workspace-operation checks did the same. The run-event endpoint also loaded complete run records before returning separate event rows. Large contexts and runner configurations therefore increased database transfer for ordinary access checks. **Expected behavior** Read only the stored access binding. Keep company checks, permission freshness, deleted-task tombstones, and response bodies unchanged. **Steps to reproduce** 1. Create an issue-bound run with large context, runner configuration, and output fields. 2. Subscribe an authorized viewer to company live events and publish run updates. 3. Read the run events and a linked workspace operation. 4. Inspect the run SELECT projections. The previous queries load fields that the access checks do not use. **Paperclip version or commit** The defect is present on the current master base, `62c78e56e53`. **Deployment mode** Authenticated server with PostgreSQL. The defect is not adapter-specific. ## What Changed - Select only company, scope, and issue bindings for live-event and workspace-operation run authorization. - Add a four-field run-access read for event-list requests. Reuse those columns in the existing log-access projection. - Preserve per-delivery and per-request permission checks, company predicates, and event responses. - Add real PostgreSQL and WebSocket coverage with large run fixtures. Cover grant revocation, grant restoration, deleted tasks, missing and foreign-company runs, and failed reads. - Document the bounded access reads in the existing polling-load guide. ## Verification - Six focused suites pass: 40 tests. They cover the changed paths, run-state reads, run logs, disconnect retries, and native event replay. - The new WebSocket test uses the production delivery path and a real database. It verifies that the selected SQL omits large run fields while authorization changes take effect on the same open socket. - The event-route test preserves the returned event content and denies the next request after grant revocation. - Local recursive typecheck, full build, and module-boundary checks pass. [Exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39997370119) completed successfully, covering the full grouped unit suites, typecheck, build/runtime boot, and E2E shards. The separate serial local `pnpm test:run` is still running and is not claimed as completed. - A later CI shard failed an unchanged email test expecting exactly one task after permission grants. The unchanged-base test passes at normal speed; adding only a 2.2-second pause before granting access reproduces the extra tasks because previously denied deliveries become due for retry. No email code or assertions are changed in this PR. The original failure is retained. The single shard rerun passed: 92 suites and 1,489 tests passed, with one suite and two tests skipped by the existing gates. - CI initially found a route-test double missing the new access helper. The follow-up fixes that fixture and adds event reads to malformed-ID and missing/cross-company checks. All 122 tests across the four changed-path suites pass; a final rerun of the expanded route suite passes all 92 tests. - The first CI run also timed out in the signoff changes-requested handoff while waiting for a reviewer run after a cancelled execution blocker. The focused test passes locally on this source. The same focused test passes three times on unchanged master. Controller-lease cancellation messages appear on both versions. The original CI timeout is preserved as an intermittent, unreproduced failure; its exact cause remains unknown. The signoff shard passes in the final exact-head CI run. - Initial fresh-checkout tests could not start because workspace SDK output and native PostgreSQL library links were missing. Dependency builds and the package's native-link setup corrected those prerequisites. The subsequent focused run passed. Focused command: ```sh pnpm --filter @paperclipai/server exec vitest run src/__tests__/live-events-run-read.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts src/services/heartbeat/run-state.test.ts src/__tests__/native-observer-replay.test.ts src/__tests__/heartbeat-run-log.test.ts src/__tests__/heartbeat-run-privacy-disconnect-retry.test.ts ``` ## Risks - A missing access field could cause an incorrect allow or denial. Tests exercise current grants, revocation, company scope, and deleted-task bindings through the real consumers. - No successful permission decision is cached. Every delivery and request still checks current database state. - No schema, migration, polling interval, API response, or deployment configuration changes are included. - This reduces bytes for the affected reads. It does not establish a whole-account savings percentage. Hosted transfer measurements remain separate validation after staging deployment. ## Model Used OpenAI Codex, a GPT-6 based coding agent, used repository tools, code execution, and database-backed tests. The session does not expose the exact backend model identifier or context-window size; those values are not guessed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
62c78e56e5 |
fix: describe unexpected git expiration failures in the result (#17341)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Git cache expiration keeps a workspace and reports
`inspection_failed` when an unexpected error interrupts its checks
> - It dropped that error, so the log never said what went wrong
> - This pull request adds a sanitized description of the error to the
expiration result, which the workspace reaper already logs
> - The benefit is that unexpected expiration failures can be diagnosed
without exposing commands or credentials
## Linked Issues or Issue Description
Refs #17248. Bug report described in-PR (bug report template).
**What happened?**
On one instance, a closed workspace was kept with `inspection_failed`.
The log entry carried only that reason, so the cause could not be
determined from the logs.
**Expected behavior**
An unexpected failure is logged with enough detail to diagnose it,
without leaking the command line or credentials.
**Steps to reproduce**
1. Break a closed issue's worktree so a git command fails, for example
by pointing its `.git` file at a missing directory.
2. Run expiration: the log says `inspection_failed` and nothing else.
**Paperclip version or commit:** `master` at
|
||
|
Devin FoleyandPaperclip
|
0001a4514e |
fix: keep stale pending restore points from blocking retries (#17344)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Git cache expiration saves restore points and tracks them in
`execution_workspace_restore_points`
> - A unique index allows one row per workspace, generation and checkout
unless the row is deleted
> - A pending row left by an interrupted attempt therefore blocked every
retry for an hour, and superseded rows blocked retries until retention
deleted their objects
> - This pull request narrows the index to live rows and supersedes
stale pending rows before inserting
> - The benefit is that an interrupted expiration no longer stalls a
workspace's cleanup
## Linked Issues or Issue Description
Refs #17257. Bug report described in-PR (bug report template).
**What happened?**
On one instance, a closed workspace with a nested checkout failed
expiration with `inspection_failed` on every sweep. An earlier attempt
had left a `pending` row for the nested checkout at the workspace's next
generation. Each retry stored the main worktree's restore point, then
failed to insert the nested row because the unique index on
`(execution_workspace_id, generation, checkout_name) WHERE status <>
'deleted'` still held the stale row. Running the expiration by hand
reproduced the failing insert.
**Expected behavior**
A retry after an interrupted attempt replaces the stale row and expires
the workspace.
**Steps to reproduce**
1. Leave a `pending` restore point row for a workspace's next generation
and a nested checkout, as an interrupted attempt does.
2. Run expiration with restore points: it fails with `inspection_failed`
until retention supersedes the row after the one-hour pending grace
period.
**Paperclip version or commit:** `master` at
|
||
|
|
23bd238f05 |
fix(costs): stop OAuth refreshes from duplicating unverified subscriptions (#16512)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Costs page shows one card for each AI subscription account. Each
card comes from an `ai_subscriptions` row.
> - Paperclip cannot verify an xAI/Grok account with the provider. So
each Grok row used a key made from a hash of the raw credential.
> - The Grok CLI refreshes its OAuth tokens during runs. Paperclip
writes the refreshed `auth.json` back to the vault and keeps the
secret's `aiSessionEpoch`.
> - The next run sees a new credential hash.
`subscriptionService.register()` finds no match and adds a new
subscription row. The old row stays on the Costs page. One Grok
connection becomes many duplicate cards.
> - Claude and OpenAI rows usually avoid this problem, because a
provider profile check merges them. Grok rows never get verified, so
nothing merges them.
> - This pull request keys unverified rows on the credential session
(secret id + `aiSessionEpoch`), not on the token bytes.
> - The benefit is one subscription card per Grok connection through
token refreshes. A real reconnect still gets its own card.
## Linked Issues or Issue Description
No public issue exists. Bug description:
**What happened?**
On a hosted staging instance, the Costs → Subscriptions list showed many
duplicate Grok subscriptions for one Grok connection. Each Grok OAuth
token refresh added one more card.
**Expected behavior**
One Grok connection shows one subscription card. A token refresh does
not change it.
**Steps to reproduce**
1. Connect an xAI/Grok subscription account.
2. Run an agent until the Grok CLI refreshes its OAuth token. Paperclip
writes the refreshed credential back to the vault.
3. Run the agent again.
4. Open Costs → Subscriptions. A second Grok card is there. Each refresh
adds another card.
**Agent adapter(s) involved**
Not adapter-specific (core bug). The trigger is the Grok OAuth refresh
write-back in `server/src/services/ai-connection-runtime.ts`.
**Paperclip version or commit**
`master` at
|
||
|
Nicky LeachandPaperclip
|
a708a13336 |
Separate plugin workspace completion from maintenance inspection (#17329)
## Thinking Path > - Paperclip manages AI agents and their work. > - Storage plugins own remote task workspaces. > - Run completion currently requests the same inspection used for storage cleanup. > - That inspection can fetch remote refs and scan files after the agent has finished. > - Completion needs retained workspace identity and durable writes, not permission to delete files. > - This change adds an optional completion operation and keeps maintenance inspection separate. ## Linked Issues or Issue Description **What happened?** A successful plugin-backed run can fail during completion because the storage provider performs a slow remote-recoverability inspection. Dirty or unpublished output does not need a deletion-safety proof to remain valid task output. **Expected behavior** Completion verifies the assigned workspace and confirms durable writes through the provider. Cleanup still requires the existing full inspection. **Steps to reproduce** Run a task with plugin-owned storage whose inspection fetches remote refs. Let the agent finish, then observe completion waiting for the maintenance inspection. Related context: #16428 added finalization observability. The duplicate search found no existing PR for this provider contract. ## What Changed - Add optional `workspaceProvider.finalizationVersion: 1` and strict `finalize` / `finalized` schemas. - Pass original run and agent context through live completion and recovered completion. - Give the new operation a 30-second deadline and keep ownership checks and lifecycle serialization. - Retain the inspection fallback for providers without the capability. Keep maintenance inspection and removal unchanged. ## Verification - SDK build passed. - Targeted SDK and completion unit tests: 7 passed. - Added integration coverage for capability selection, legacy fallback, maintenance separation, and failure propagation. - Local integration setup timed out starting PostgreSQL; the integration suite passed in GitHub CI. - Full build/typecheck and broader tests passed in GitHub CI (55 successful checks, 2 skipped). Local checks remained targeted as requested. - Diff reviewed for secrets and operational identifiers. ## Risks The capability is opt-in. Providers must not advertise it until their completion implementation is available. An unavailable provider or failed durability barrier remains retryable. No schema or stored-binding migration is required. Test-file size exception: the provider integration suite grows from 973 to 1,002 lines. The 29-line case follows the same prepared binding through legacy completion, opted-in completion, and maintenance inspection using the existing single-connection database fixture. Keeping it there avoids a second embedded-Postgres lifecycle for one transition test. The completion helper unit tests are in a separate focused file. ## Model Used OpenAI GPT-6 Codex, with reasoning and code/test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
0fb152ee94 |
fix: save stale remote-tracking commits in nested restore points (#17289)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Git cache expiration removes closed issues' worktrees once their
remote, or a restore point, can reproduce them
> - A nested `.paperclip-repositories` checkout's remote-tracking refs
can point at commits the remote has since dropped, such as a branch
deleted after a squash merge
> - Strict expiration rightly keeps such a checkout, because it may hold
the only copy; but restore point mode also kept it forever, because
there was nothing else to save
> - This pull request saves those commits in the checkout's restore
point, so the workspace can be reclaimed without losing them
> - The benefit is that long-lived multi-repository workspaces are
reclaimed in restore point mode
## Linked Issues or Issue Description
Refs #17257. Bug report described in-PR (bug report template).
**What happened?**
With restore points enabled, four closed workspaces on one instance were
kept with `unpushed_managed_checkout`, after a dry run had counted them
as candidates. Each nested `paperclip` checkout was clean, on `main` at
a commit that is on GitHub `main`. But it held about 1,000
remote-tracking refs, and between 28 and 41 of them pointed at branches
the remote has since deleted or force-pushed (147 to 203 commits it no
longer has).
The relaxed check passed, so the checkout needed no restore point beyond
those commits. Capture saved only `HEAD` history, so it found nothing to
save and stored no restore point. The final check before deletion then
ran the strict check, which keeps stale remote-tracking commits, and
rolled back. The dry run never reaches that final check, so it
over-reported. Together these workspaces held about 2.4 GB.
**Expected behavior**
In restore point mode, commits that exist only in remote-tracking refs
are saved, and the workspace expires.
**Steps to reproduce**
1. In a nested checkout, fetch a remote branch, so
`refs/remotes/origin/<branch>` exists.
2. Delete that branch on the remote and garbage-collect it.
3. With restore points enabled, close the issue and run expiration: the
workspace is kept with `unpushed_managed_checkout`.
**Paperclip version or commit:** `master` at
|
||
|
Devin FoleyandPaperclip
|
6877760904 |
feat: save ignored files in git restore points (#17290)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Git cache expiration removes closed issues' worktrees after saving a
restore point of anything the remote cannot reproduce
> - Restore points left ignored files out, so any worktree with an
ignored file that was not a known regenerable directory was kept
> - On one instance that was the most common reason a closed worktree
was still kept, mostly build caches and agent evidence files
> - This pull request saves those ignored files in the restore point too
> - The benefit is that more closed worktrees are reclaimed, and nothing
an agent left behind is lost
## Linked Issues or Issue Description
Refs #17248, #17257. Bug report described in-PR (bug report template).
**What happened?**
With restore points enabled, one instance kept 15 of 43 inspected closed
worktrees with `unique_ignored_files`. Among 19 such worktrees on disk:
- 9 held `ui/tsconfig.tsbuildinfo`;
- others held build output outside the regenerable directory names
(`server/ui-dist/`, `test-results/`);
- 2 held materialized skill copies under `.claude/skills/`;
- about 5 held small files agents kept out of git: patches, evidence
archives, recovery receipts.
**Expected behavior**
Ignored files that are not regenerable are saved with the restore point
and restored on reopen, so the worktree can be removed.
**Steps to reproduce**
1. In a closed issue's worktree, leave an ignored file outside
`node_modules`, `dist` and the other regenerable directories (for
example `ui/tsconfig.tsbuildinfo`).
2. With restore points enabled, run expiration: the worktree is kept
with `unique_ignored_files`.
**Paperclip version or commit:** `master` at
|
||
|
DottaandPaperclip
|
3f1ad6b25b |
fix(chat): resume task Retry after retained-source containment (#17317)
## Thinking Path > - Paperclip manages agent work through task and execution ownership. > - A task Retry must retain the user's request while the system verifies an old executor. > - Connected conversations also retain their external principal and accepted source batch. > - Chat Retry admission required containment before it could save a wait for that recovery. > - A saved chat receipt could also remain deferred after containment became available. > - This pull request lets the exact recovery-bound request wait and resume through its chat owner. > - Normal admission still checks containment, current access and policy before one successor starts. ## Linked Issues or Issue Description **What happened?** An ordinary task Retry on a connected conversation failed with `failed_chat_retry_not_authorized` while its retained remote executor still needed verification. The new recovery generation existed, but the chat action failed before it could save a dispatch wait. This occurred after the repairs in #17267 were confirmed live. The local regression found a second gap. A processed chat action with a deferred wake was not reconsidered after containment. Re-entering normal admission also tried to insert its existing durable receipt again. A historical hold for the same failed run could also hide the current Retry's workspace strategy, so the request still waited after the old executor was contained. Preparation could then read that sibling instead of the recovery record that owned the fresh successor, losing the fresh-workspace decision. **Expected behavior** Save the exact authorized Retry while its matching recovery generation verifies the executor. Once containment is recorded, reconsider the same chat receipt through normal admission. Start one successor with the original source batch. Keep publication, principal, pause and budget checks. **Steps to reproduce** 1. Admit a GitHub or Telegram request and record a failed native run with a retained remote lease. 2. Leave the executor's termination unconfirmed and exhaust its prior recovery generation. 3. Use ordinary task Retry to create a new recovery generation. 4. Process the connected Retry. The old admission code fails instead of retaining a wait. 5. With a deferred receipt, record exact containment and reconcile again. The old dispatch path leaves it stranded. **Paperclip version or commit** Reproduced from `412914cfa78d5abefcc34d3eb8d4bb86e461caac`. The branch is rebased onto current master. **Deployment mode** Hosted tenant for the reported failure. Isolated local PostgreSQL and the assembled heartbeat/chat services for the regression. Refs #17267 and #16965. This change does not add a Cloud dependency or change the absence protocol. ## What Changed - Permit a saved admission wait only for the exact current task Retry and active source recovery generation. - Keep promotion and execution dependent on independent containment proof. - Bind a Retry to active automatic recovery without replacing its generation, attempt count or deadline. - Share the current exact Retry strategy across historical holds for the same run. Retain each hold's stop and effect checks. - Select the canonical recovery record that proves the committed fresh successor during workspace preparation. - Let task receipt reconciliation return the same deferred action to its chat owner. - Recheck current board and external authority, then update the existing wake atomically with its successor. - Preserve the exact chat source batch. Keep generic board-comment filtering unchanged. - Add GitHub and Telegram cases for waiting, containment, one successor, stale generations, source edits, endpoint pause and supersession. - Update recovery documentation and capability maps. ## Verification - Red: real PostgreSQL tests reproduced the admission rejection for GitHub and Telegram. Six active-recovery cases then failed on the generation-ID assumption. Two same-run historical-hold cases remained deferred after containment on the old strategy lookup. A preparation regression then returned no readiness when the historical hold was stored first. - Green: 73 selected chat integration tests and 394 continuation, task Retry, workspace recovery, recovery-state and workspace-preparation tests passed. The chat tests use the real chat admission and heartbeat queue. Unrelated capacity stays occupied so no real provider starts. They assert no executor before containment, one successor afterward, unchanged source leases while waiting, preserved recovery budgets, and denial after authority changes. A separate source hold still blocks work. Both sibling insertion orders pass through the committed-readiness lookup and real workspace preparation: the new directory has independent Git storage and files, and unpublished old work remains intact. - `pnpm -r typecheck`, `node scripts/check-maps.mjs` and `git diff --check` passed after the review fixes. The source build passed on `f6660efa48e37a80cd9857217a537606de85b5c8`. - The local host reached its PostgreSQL semaphore limit. Successful runs used a test-only loader to reduce disposable cluster connection and worker limits. No repository database helper or application setting changed. - The full local `pnpm test:run` began before the final review edits, failed in its first server phase and was stopped after those failures. Later phases did not run. This is not a full local pass. The transient Retry-notice case passes after fixture isolation; the four Discord close-owned publication cases also pass in isolation. Other broad-run failure causes are not established. - Exact-head CI and security checks pass on `f6660efa48e37a80cd9857217a537606de85b5c8`; Greptile is 5/5 with zero unresolved review threads. Optional Paperclip Review completed neutral with a 5/5 static assessment of all 11 changed files and no findings; it did not run tests or hosted acceptance. All 57 current-head checks are terminal: 54 success, two skipped and the optional neutral review, with no failures or pending checks. - The real hosted workflow confirmed that the deployed absence repair recorded containment. A later ordinary UI Retry was accepted and saved a durable request, but it remains behind the historical source hold. The useful existing review remains unchanged. This follow-up is not deployed; a live successful successor and final task disposition remain unverified. No missing files are claimed recovered. ## Risks - The existing task reconciliation sweep can restage a deferred chat action. This reuses its receipt and generation; it does not grant new execution or publication authority. - Dispatch must keep the board Retry identity separate from the connected request's principal. The regression checks these bindings and preserves current chat authorization at each phase. - Historical holds reuse a strategy only from the locked same-run recovery that names the current Retry intent. A different run, malformed recovery record or independent source hold remains fenced. - Unknown executors without a matching active recovery still fail admission. Promotion and execution require containment even when admission saved a wait. - No schema or API shape changes. No credential changes, manual deployment or service restart. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution and browser verification. The exact served model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |