mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 19:35:13 +01:00
bf142db65aa90eb4c23bf9084e95b33515234637
2619
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin FoleyandPaperclip
|
bf142db65a |
perf(server): narrow external review wait context reads (#17562)
Project current chat authorization fields for external review-wait discovery instead of saved execution input. Preserve typed source bindings, nested question receipts, candidate eligibility, live revocation checks, and stored context. Validation: 181 native chat integration tests, full build and typecheck, exact-head Greptile 5/5 (medium), full green CI, and no unresolved review threads. The full-context counterfactual fails both direct-chat size regressions. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
08dafde1d6 |
perf(accounting): narrow locked usage receipt run reads (#17559)
Select only the five run fields used by usage receipt persistence under the existing company and row locks. Keep receipt history, replay checks, budget updates, and stored execution evidence intact. Validation: 37 focused tests, full build and typecheck, exact-head Greptile 5/5 (medium), and full green CI with no unresolved threads. The first CI attempt failed an unchanged Runner shutdown test; its qualified local case and same-head failed-lane rerun passed. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
1ad8f25c77 |
perf(heartbeat): narrow queued comment polling run reads (#17560)
Read current control facts and typed issue/adapter bindings for queued-comment polling. Keep full mutation and execution reads, native preparation and steering, authorization, and stored evidence. Validation: full build and typecheck, 160 focused tests plus two assembled route regressions; exact-head Greptile 5/5 (medium), full green CI, zero unresolved threads. Old-route counterfactual transfers 2.5 MB versus less than 2 KB. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8415f4a679 |
fix(github): drain queued pings during active App setup (#17551)
## Thinking Path > - Paperclip connects external work to an agent and its ordinary tasks. > - A dedicated GitHub App must prove signed webhook delivery during setup. > - Cloud queues the original bytes until the tenant can authenticate them. > - An older personal connection can leave a long empty-queue backoff in that tenant. > - Active App setup can then time out before the queued ping is read. > - This change lets active setup request the existing event drain with a short rate limit. > - The same webhook handler still owns signature verification and setup state. ## Linked Issues or Issue Description Refs: #17457, #17513. **What happened?** Cloud accepted a new App's ping, but the tenant read it only after the setup wait timed out and cleanup archived the draft. An existing personal GitHub connection had left an empty-queue backoff. The poller checked that deadline before it read the new App state. **Expected behavior** An active manager setup request must be able to read the queued ping promptly. It must not bypass signature checks or create another App. **Steps to reproduce** 1. Let a personal GitHub connection build its empty-queue backoff. 2. Start a dedicated App and complete its installation. 3. Return to the waiting setup screen while the signed ping is queued. **Paperclip version or commit** This behavior is present on the base commit of this PR. ## What Changed - Let the authenticated wizard request a queue drain only while its Cloud App still lacks signed-delivery verification. Existing manager, credential, and installation checks run first. - Register a narrow callback to the server-owned GitHub poller. Keep provider handling in the GitHub services. Remove the callback on server close. - Share one in-flight attempt between setup and background requests. Limit setup attempts to one per five seconds, including after errors. - Retain the existing passive backoff, sealed queue, acknowledgement rules, and canonical webhook signature handler. - Add regressions for the old backoff, concurrent requests, failed requests, and wizard signature verification. Update the setup documentation and map. ## Verification - Observed the backoff regression fail against the old service: active setup made no additional Cloud lease request. It passes with this fix. - Three focused event-poller tests pass. They use the real admission/backoff code with stubbed SQL and Cloud boundaries. - All 39 startup tests pass. Server TypeScript, map checks, and diff checks pass. - The Postgres integration suites could not start locally because this host exhausted its semaphore limit. They remain required CI checks. This is not a passing local integration result. - Full current-head CI passes, including the Postgres and chat integration suites. Greptile scored this exact head 5/5 with no unresolved findings. - The additional Paperclip review check ended neutral because its execution did not complete. It did not publish a validated assessment. - A fresh deployed setup attempt remains required before live qualification. ## Risks A waiting setup screen now makes a bounded readiness request to Cloud. The existing connector request timeout still applies. Concurrent calls share the same drain, and failures release it for a later retry. The request stops after persisted verification or when the browser stops advancing setup. No timer, durable queue, fleet setting, or sleep blocker is added or removed. The existing background reconciliation path still has its existing sleep restrictions. Live setup-to-review behavior is not established by these local tests. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime model ID and context-window size are not exposed in this session. Used for code execution and regression analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
DottaandPaperclip
|
2a9d527c68 |
fix: preserve native startup during controller restart (#17511)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runtime keeps a durable runner alive across controller restarts. > - A restart can begin while a provider session is still opening. > - The old controller fenced read-only run admission before it could publish that prepared session. > - Admission failure then closed the runner that shutdown needed to detach. > - This pull request lets only the existing startup finish admission within the bounded shutdown wait. > - The controller detaches that exact session before it can submit a turn. New work and state changes stay fenced. ## Linked Issues or Issue Description **What happened?** A controller restart during native provider startup could close the prepared runner with `native control-plane run admission failed`. Recovery then found no running process and placed the task in a reconciliation hold. The same admission path follows both fresh startup and replacement after a failed provider resume. **Expected behavior** Shutdown must detach the existing prepared session so the next controller can recover it through the normal identity checks. Shutdown must not admit a new executor, submit a turn, or override an explicit Stop. **Steps to reproduce** 1. Delay a native backend while it opens its provider session. 2. Request controller restart detachment for that run. 3. Let backend startup finish before the 30-second deadline. 4. Observe that read-only admission fails and shutdown reports the run as inactive instead of detached. **Additional context** Related work: #17073 handles remote process-monitor classification; #17459 handles macOS process fingerprints; #17097 covers broader hot upgrade recovery. This change fixes the startup admission fence. It does not relax process identity or provider launch-profile checks. ## What Changed - Separate the read-only admission guard from the guard on control-plane state changes. - Let shutdown await only the startup already registered for the run. Close its admission window when the wait ends. - Reject a new execution after its run is fenced for shutdown. - Detach the prepared session and exit before runtime-question attachment or provider turn submission. - If Stop is pending before or after publication, retain its exact cancellation acknowledgment and cleanup owner across authority reads and runtime-question attachment. - Share one 30-second admission/Stop-settlement deadline across the restart sweep. Report an unsettled run as pending while healthy runs continue hot-restart adoption. - Cover the real runtime and admission path, deadline expiry, new-start rejection, persisted binding checks, and write fences. - Document the bounded startup handoff in the restart ownership contract. ## Verification - Red: the new actual-runtime/admission regression failed on the previous code. Shutdown reported the run as inactive and did not detach it. - Red: review regressions reproduced detach before a post-publication Stop acknowledgment and an actual hot-restart preparation error when startup Stop cleanup timed out. - Green on `e10d0cb238`: 18 startup/restart tests passed, including Stop during authority reads and runtime-question attachment, the real recovery preparation and shutdown coordinator with a stalled Stop plus healthy peer, and the shared sweep deadline. The earlier 36-case Stop/restart selection also passed before this follow-up. - Both complete changed test files previously passed on head `8e9c2610fe`: 857 tests. They used real PostgreSQL 18 with a fresh database per fixture. An earlier local harness attempt reused one database and hit duplicate fixture keys; that harness was corrected before this complete pass. - Real PostgreSQL: 7 admission and revocation tests passed. These cover wrong runner identity, persisted binding mismatch, checkpoint/result rejection, and in-flight write revocation. The local test harness used a fresh Docker PostgreSQL 18 instance because macOS embedded Postgres exhausted the host semaphore limit. Only database provisioning was replaced; assertions and production port code were unchanged. - Transport safety acceptance: the current source transport controlled a real Linux ARM runner and a deterministic Codex test provider in a disposable Debian container. After startup, detach and generic cleanup preserved the process. A new controller authenticated the same PID and provider session. The provider log had exactly one session start and no turn start. Duplicate launch was forbidden, the retained workspace was unchanged, and no stop or suspend command was issued. - The Linux acceptance check tests the shared native transport safety contract. It is not a live Claude response, a hosted Daytona run, or recovery of the original task. Those remain unverified. No original task, sandbox, credentials, or permissions were changed. - `pnpm -r typecheck` and `pnpm build` passed before the review follow-up. Server typecheck passed on current head `e10d0cb238`; current-head CI also passed the full typecheck and build gates. - The full local `pnpm test:run` command was attempted, then stopped after repeated embedded PostgreSQL bootstrap setup failures. That broad local run is incomplete. The current-head four-file Docker PostgreSQL run passed 879 tests, with one existing warm-copy `claim-uncertain` case timing out and recovery failing to import because the temporary provisioning mock omitted a support export. The harness is corrected and affected cases are being rechecked. The timeout is unexplained; it is not being labeled an infrastructure failure. No assertions are skipped by the database provisioning replacement. - All 55 clean-host CI checks passed on current head `e10d0cb238` (2 conditional skips), including all general test shards and native runner verification. Greptile reviewed that exact head at 5/5 with no findings. The previous Paperclip review findings are addressed in [the review response](https://github.com/paperclipai/paperclip/pull/17511#issuecomment-6391520242); its new advisory review remains queued. ## Risks - Admission must stay read-only. All writes retain the original synchronous controller fence, and both admission checks still validate the exact persisted binding. - Startup that exceeds the 30-second shutdown wait still fails closed. It cannot claim a successful detach or start a turn. - No schema migration or provider configuration change is included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and test execution. The exact runtime model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
8a85d9d647 |
perf(heartbeat): coalesce repeated native stranded releases (#17547)
Skip repeated successful native release checks for the same unchanged source within one recovery pass. Preserve every saved request, failure retry, cursor update, source change, legacy path, and current admission gate. Verified with 201 focused tests, 32 full-control queue cases, build and typecheck. Exact-head CI is green, Greptile is 5/5 with Low impact, and no review threads remain unresolved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
c8a6b4561b |
perf(heartbeat): narrow native Stop evidence rechecks (#17544)
Read current native Stop identity, acknowledgement, and process evidence without saved execution inputs, profiles, excerpts, or unrelated result data. Preserve the complete nested acknowledgement and every existing lease, remote-termination, process, ownership, and admission check. The physical-stop helper body is unchanged. Verified with 101 tests across the affected suites, the full build, and typecheck. The review fix passed 62 focused tests. Large valid saved inputs stay below 2 KB at the raw driver boundary; large malformed string and array results stay below 1 KB and remain stored. CI is green and exact-head Greptile is 5/5 at Medium impact with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
2f637fe8c4 |
perf(tools): scope connection readiness assignment reads (#17541)
Connection readiness reads current permitted connection identities without unrelated catalog rows or complete profile-rule payloads. Reuse profile precedence and selector logic, preserve full public/runtime summaries, and keep installed connections, identity checks, and gateway authorization current. Verified with 106 focused tests, the full build, and typecheck. The direct catalog inventory regression fails at 102 rows when the scope filter is removed and passes at two rows with the filter. One unrelated CI timing assertion differed by 1 ms; the exact local test and the single failed shard rerun passed on unchanged source. CI is green and exact-head Greptile is 5/5 at Medium impact with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
80c1261417 |
perf(recovery): decline native legacy checks before bulk reads (#17540)
Decline native legacy-continuation checks before loading saved execution input. Preserve exact nonblank task bindings, skip reasons, complete legacy recovery inputs, company scope, and retry accounting. Verified with 105 recovery tests, the full build, and typecheck; the binding review fix passed 73 affected tests and server typecheck. CI is green and exact-head Greptile is 5/5 at Medium impact with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
e0855fe92a |
perf(wake-queue): narrow native release reads (#17539)
Native wake-queue release reads omit saved profiles and excerpts. Legacy release reads keep their complete inputs. Ownership, cleanup, and file retention checks remain unchanged. Verified with 275 wake-queue tests, the full build, and typecheck. All CI checks pass. Greptile scored the exact head 5/5 with Medium impact and no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
ac15d883f7 |
fix: preserve remote-tracking history when expiring nested worktrees (#17535)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Nested project repositories are now worktrees of one shared clone
per project repository, and existing standalone clones are converted in
place
> - Conversion keeps the clone's remote-tracking refs as task-owned
branches (`paperclip/<id>/remotes/...`) so commits only they hold
survive
> - Expiration then required every task-owned branch, including those,
to be on the remote
> - Remote-tracking refs for branches deleted or force-pushed upstream
never are, so converted checkouts were kept forever
> - This pull request treats those refs the way the standalone-clone
path already does: commits missing upstream go into the restore point
instead of blocking
> - The benefit is that converted workspaces can expire and free disk
without losing any history
## Linked Issues or Issue Description
Refs #17407, #17382, #17289. Bug report described in-PR (bug report
template).
**What happened?**
On one instance, 17 closed workspaces in a two-repository project were
kept on every expiration sweep with
`unpushed_managed_checkout_branches`. Each nested checkout is a
converted worktree whose task-owned refs include
`paperclip/<id>/remotes/origin/<branch>` for upstream branches since
deleted or force-pushed (dependabot, squashed feature branches). Their
commits are on no remote, so `provenNestedWorktreeHistory` refused. The
task's own branch was fully pushed.
**Expected behavior**
As in the standalone-clone path (#17289), remote-tracking commits
missing upstream are preserved in the restore point, and only genuinely
local branches block expiration.
**Steps to reproduce**
1. Fetch an upstream branch into a standalone nested clone, then delete
that branch upstream.
2. Convert the clone into a worktree (nested clone migration).
3. Close the issue and let expiration inspect it: kept with
`unpushed_managed_checkout_branches`.
**Paperclip version or commit:** `master` at
|
||
|
Devin FoleyandPaperclip
|
ef0a8db502 |
perf(native): narrow reconciliation candidate payloads (#17531)
Select the complete workspace-sync record and native status wake identifier for reconciliation candidates. Preserve candidate policy, JSON types, full later evidence, and stored data. The committed replay regression includes the real reconciliation scan. Full build, typecheck, 114 focused tests, complete CI, and exact-head Greptile5/5 pass. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
39e5c8c1c3 |
perf(server): narrow committed chat authorization reads (#17530)
Select current chat authorization fields for committed responses under the existing scoped share lock. Preserve complete evidence for actual retries, source revocation, JSON types, and saved run data. Full build, typecheck, 113 focused tests, complete CI, and exact-head Greptile5/5 pass. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
e82d86bfeb |
fix(server): gate periodic heartbeat recovery to one chain in flight (#11699)
Bind periodic recovery admission to lifecycle tracking. Keep one physical pass active through cleanup, report skipped ticks and long-held ownership, and preserve direct scheduled delivery. Full build, typecheck, 32 focused tests, complete CI, and exact-head Greptile5/5 pass. Co-Authored-By: Senior Engineer Fable <dotta@paperclip.ing> Co-Authored-By: Dotta <bippadotta@protonmail.com> Co-Authored-By: Devin Foley <devin@paperclip.ing> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
64a8b927c1 |
fix(native): narrow committed finalization and cleanup reads (#17525)
Select fresh metadata for committed finalization and cleanup. Preserve full saved evidence for new finalization, cancelled workspace retention, and legacy Stop. PostgreSQL byte-budget and unchanged-row regressions pass, full CI is green, and Greptile scored the exact head 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
71f06d84dc |
fix(db): keep bounded response captures attributable (#17522)
Keep more caller groups in one bounded database response capture and verify public caller paths in the production bundle. All local checks and CI passed; Greptile scored the exact head 5/5 with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
cb47dbb2da |
fix: resolve git credentials with the issue's identity in background jobs (#17515)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Git cache expiration, isolated source re-home, nested clone
migration and workspace reopen fetch from the workspace's remotes to
prove history is on the remote or to rebuild workspaces
> - They built git credentials from the issue id alone, but the managed
GitHub identity is selected from the run's responsible user and agent
> - So the credential never resolved, they fell back to anonymous
access, and private repositories were unreadable
> - This pull request resolves credentials with the issue's responsible
user and assigned agent, the same identity agent runs use
> - The benefit is that workspaces with private repositories can expire,
re-home, migrate and reopen like public ones
## Linked Issues or Issue Description
Refs #17464, #17407, #17369. Bug report described in-PR (bug report
template).
**What happened?**
On one instance, 18 long-closed workspaces in one project were kept on
every expiration sweep with `managed_checkout_upstream_unavailable`.
Their nested checkout is a private GitHub repository. Expiration called
`createGitRemoteAuthProvider(db, companyId, { issueId })`. With that
context, `resolveManagedGitHubCredential` returned "No managed GitHub
identity is available for this run". `withAnonymousGitFallback` then
tried anonymous access, which cannot read the private repository. Called
with the issue's `responsibleUserId` and `agentId`, the same lookup
returned a credential.
**Expected behavior**
Background workspace jobs authenticate as the issue's identity, as agent
runs do, and can verify private repositories.
**Steps to reproduce**
1. Configure a project with a private GitHub repository and a managed
GitHub identity tied to the issue's responsible user.
2. Close an issue whose workspace includes that repository and let git
cache expiration inspect it.
3. It is kept with `managed_checkout_upstream_unavailable`.
**Paperclip version or commit:** `master` at
|
||
|
DottaandPaperclip
|
5c3dce09b5 |
fix(runner): preserve verified remote artifacts across controller releases (#17512)
## Thinking Path > - Paperclip manages AI agents and owns their task authority. > - Remote native agents can keep working while their controller restarts. > - The controller must authenticate the same runner and verify its provider files before recovery. > - The current path compares a retained provider pack with the new controller release. A release change can reject intact files. > - The new controller binary digest can also reject the original daemon during PRP authentication. > - This PR records the admitted release authority in private controller state and uses it for exact process reattachment. ## Linked Issues or Issue Description **What happened?** A remote Claude session started and used tools. After controller shutdown, recovery failed with `runner_remote_provider_artifact_incompatible`. The transport combined stderr into stdout, so the error omitted the verifier reason. The historical rejected manifest is unavailable. This PR reproduces controller release drift as a cause; it does not claim that the historical mismatch is proven. Reaching pack verification implies the earlier exact-process/controller-identity checks passed. That supports a preserved-state recovery path but is not a direct inspection of the deployed volume. **Expected behavior** A compatible, unchanged remote runner can reattach after a controller release changes. Full package hashes, exact process scope, protocol capabilities, and authenticated runner identity must still pass. Fresh launches use the current release. **Steps to reproduce** 1. Verify and admit a remote native session with release A. 2. Preserve its runner process and replace the controller with release B. 3. Reattach the original process. The old path compares its pack with B and fails at `manifest mismatch`. 4. With this fix, the controller reads its saved A authority and verifies all A hashes. Changed files still fail. **Paperclip version or commit** Reproduced from master `470cb34c0e42cb9d0b232d0bcf5d45cc7b0a81ca`; rebased onto `a3675ddcdf`. **Deployment mode** Native remote runner on a Daytona-backed Paperclip Computer. Local acceptance uses isolated Linux containers and actual packaged artifacts. Related work: #17361 and #17448 handle runner upgrade and qualification permissions. #17097 and #17439 add durable controller checkpoints and server adoption. That stack can skip remote artifact preparation with a saved checkpoint. This PR fixes the current file-backed path and retains full provider-pack verification; it has no checkpoint schema or empty-disk controller recovery. ## What Changed - Save the complete verified pack manifest and authenticated runner version/digest after successful session admission. - Bind the receipt to company, agent, workspace, session, runner, environment lease, remote directory, and provider descriptor. Keep it outside guest backup directories. - Reuse the saved authority for exact reattachment. Keep current release selection for fresh launches and explicit session replacement. - Fail closed for mismatched or corrupt receipts. Historical sessions without a receipt retain the current release integrity gate. - Report bounded, redacted verifier output from both transport streams. Warn on receipt write failure while returning the already admitted session to its owner. - Add scope, persistence, replacement, storage failure, and backend wiring regressions. Update recovery documentation and the feature map. ## Verification - Test-first red: the original selection chose release B instead of A after controller recreation. The new receipt regression passes. A separate red test showed the missing stdout diagnostic; it passes after the diagnostic fix. - Authority and diagnostic suites: 49 tests pass. Backend pack, process adoption, and harness persistence selection: 47 tests pass. - Actual Linux provider image: complete A manifest/artifact verification passed; controller source-revision drift failed on the old path; saved A authority passed; a modified ACPX sidecar failed its SHA-256 check. - Actual native Linux runner: authenticated a session, detached the controller, changed the controller manifest and binary, then authenticated the original PID and provider session with saved authority. There were zero duplicate spawns, zero new provider turns, no stop/suspend, and the workspace file remained. This probe used the real runner with a deterministic Codex fixture; it is separate from the real ACPX package integrity proof. - Full workspace typecheck and build passed. The full native executor suite first hit host PostgreSQL semaphore exhaustion. The complete changed files then passed 864/864 tests with separate real PostgreSQL databases in a disposable Linux container. The temporary local fixture changed only database provisioning and used normal repository migrations. Current-head CI also passed the full required test, Runner, browser, typecheck, and build lanes. - `node scripts/check-maps.mjs` and `git diff --check` passed. These structural checks do not prove live recovery. - Current head `48fca2ae33d99f7f6d2fe12f431cf76d25185c70`: 55 successful checks, two conditional skips, no pending or failed checks; fresh Greptile 5/5 with zero findings and zero unresolved review threads. Commitperclip quality review passed. - No original task was retried or changed. Existing Daytona access could not resolve the original sandbox or its image; a fresh staging Claude recovery is not claimed. ## Risks This prevents recurrence for sessions admitted with the new receipt. It does not retroactively recover the historical failed run. Saved authority exists only after this change has admitted a session. It cannot repair an ambiguous historical launch or restore a receipt on an empty replacement controller. A stale protocol or modified pack remains a hard failure. Receipt write errors warn without losing the live session handle, but later recovery still needs matching authority. The broader checkpoint stack changes the same server preparation and runner-artifact option and will require a deliberate integration. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d0c8f0b090 |
fix: explain project access denials and add an authorized repair path (#17509)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Low-trust agents may read only the projects in their allowed scope. > - Workspace setup correctly stops a task outside that scope. > - The task previously showed a generic failure, and agent settings did not offer an additive repair. > - This pull request explains the denial and lets authorized operators add projects while preserving other restrictions. > - The saved task then offers an explicit retry. Saving access does not start work. ## Linked Issues or Issue Description **What happened?** A project access denial appeared as an ordinary failed attempt. Operators could not add a second allowed project from agent settings. **Expected behavior** The task should explain the denied access and link to a working permission editor. The editor should preserve existing scope and restrictions, enforce operator authority, and retain drafts after a failed save. **Steps to reproduce** 1. Assign a low-trust agent to a task whose project is outside its allowed projects. 2. Let workspace setup deny project access. 3. Open the task, follow Manage project access, add the project, save, reload, and return to the task. 4. Confirm that the saved message remains and Retry task is a separate action. ## What Changed - Record a typed, non-retryable project denial before provider dispatch. Project denials do not enter automatic continuation or provider comment policy. - Project current and historical setup denials into a safe task notice. Keep unrelated failures, prior assignees, and approval/pause/budget gates separate. Recheck current project authority, including standard trust and root-issue grants. Accepted retries keep their working state. - Add board-only project-access reads and additive saves under the agent row lock. Enforce company scope, agent-management authority, pending-approval restrictions, and an audit record for new additions. - Add the production notice and searchable multi-project editor, including read-only, saving, cancellation, and failed-save states. Preserve the return-to-task query through canonical agent redirects. - Register board-only project-access routes in OpenAPI with a strict request schema. - Add production Storybook examples and API/persistence/concurrency regressions. Retain the approved design snapshot and update the behavior maps and implementation contract. ## Verification - Passed: `pnpm -r typecheck`, `pnpm build`, `pnpm build-storybook`, `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `node scripts/storybook-catalog.mjs`. - Passed: 11 permission/projection tests and 30 run-completion tests against an isolated PostgreSQL database. These cover historical failures, concurrent additions, idempotent audit, preserved restrictions, foreign projects, read-only users, agent self-grants, and no automatic wakeup. - Passed: all 27 OpenAPI route tests, including route coverage and the new board-only request/response contract. - Browser: the real app displayed a historical denial; Manage project access opened the editor; saving retained the original project and survived reload; Return to task showed the ready notice. Saving created one audit event and zero runs. Explicit Retry returned 202 and reached the separate sandbox requirement, which remained enforced. No external provider was used. - All five production Storybook interaction checks pass: blocked task, save, cancellation, failed-save recovery, and read-only settings. These use fixture callbacks; durable authorization is covered separately above. - `pnpm test:run` was attempted locally but stopped after repeated embedded PostgreSQL startup failures caused by macOS semaphore exhaustion. Full-suite CI is required before merge. The separate UI suite passed all 9,095 tests. The 41 focused server regressions also passed after rebasing onto the latest master. A further 26 recovery-classification tests pass, including a red-to-green regression for automatic continuation after a project denial. ## Risks - No schema migration. The new endpoint only adds same-company project IDs and preserves the rest of the stored policy. - The historical notice recognizes one exact setup denial message. It does not display arbitrary run errors or disclose a different source project's identity. - The retry still passes all current admission and isolation checks. The notice does not promise that other execution requirements are satisfied. - Related open work: #13309 covers inherited secret restrictions; #11234 covers the broader repository catalog. This change does not replace either policy surface. ## Model Used OpenAI Codex (GPT-6), with code execution and browser tools. The precise runtime model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused server and full UI suites; full-suite infrastructure limit above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
470cb34c0e |
Read reply repair and ownership evidence without execution payloads (#17501)
Narrow native reply repair and conversation ownership reads to their consumed evidence while preserving full stored history, result receipts, locks, current authority checks, and publication authorization. Verify successful repair from equivalent unpublished states, exact final replies and decisions, no duplicate repair, preserved history, and 213 recovery/presentation tests. Build/typecheck and all CI pass. Greptile 5/5 Medium. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
b6591c93a0 |
fix(issues): validate UUID query parameters on list and count endpoints (#17389)
Reject malformed UUID filters (participantAgentId, projectId, workspaceId, executionWorkspaceId, parentId, parentIssueId, descendantOf, createdFromIssueId, labelId) on GET /api/companies/:companyId/issues and /issues/count with a 422 instead of letting Postgres fail the uuid cast with a 500. Fixes PAPERCLIP-BACKEND-4M. Co-Authored-By: Devin Foley <devinfoley@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
a7006a9506 |
Reclaim finished isolated task directories with restore points (#17472)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents do task work in execution workspaces on the instance disk
> - A task without a project repository runs in a plain (non-Git) task
directory under `<instance>/isolated-workspaces/<company>/<issue>/`
> - Nothing removes these directories after the work is done, so the
disk fills up
> - The directories can hold the only copy of agent work, so plain
deletion is not safe
> - This pull request saves each finished task directory in an encrypted
restore point, removes it, and restores it when the work reopens. It
also removes orphaned directories after a grace period
> - The benefit is that finished task directories stop using disk, and
no agent work is lost silently
## Linked Issues or Issue Description
**What happened?**
On one instance, `instances/<id>/isolated-workspaces/` held 251 task
directories and 12 GB, and it grew every day. 61 directories belonged to
open workspace rows whose issues closed more than 7 days before (3.9
GB). 12 belonged to issues closed 3 to 7 days before (0.7 GB). 4.0 GB
were directories with no open workspace row (the row was archived or
missing).
The cause has two parts:
1. The terminal workspace reaper archives a workspace only when it
proves delivery through Git (a merged pull request or a merged branch).
A task directory is not a Git checkout, so the Git status inspection
fails and the reaper always skips the row as "undelivered". Git cache
expiration handles only `git_worktree` workspaces.
2. When a person archives such a workspace by hand, artifact cleanup
removes a `local_fs` directory only when `metadata.createdByRuntime` is
true. Task directory rows never set it, so the directory stays and the
row becomes `cleanup_failed`.
**Expected behavior**
Paperclip reclaims the task directories of finished work by itself. It
must not lose files that only exist in those directories.
**Steps to reproduce**
1. Run an agent on an issue without a project repository (for example a
low-trust sandbox task, or an issue with no project), so it works in
`isolated-workspaces/<company>/<issue>/`.
2. Let the agent write files there, then mark the issue `done`.
3. Wait any number of days. The directory and its workspace row stay.
The reaper logs the row as skipped (undelivered).
4. Archive the workspace by hand. The directory still stays.
**Paperclip version or commit**
`master` at
|
||
|
Devin FoleyandPaperclip
|
cb6c878c38 |
Read execution blocker evidence without full execution payloads (#17491)
Read the lifecycle, process, and typed context evidence consumed by execution blockers. Preserve full result evidence, native workspace receipts, current state checks, and stored history. Validated with 458 real PostgreSQL and recovery tests, full build, recursive typecheck, all55 CI checks, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
1cfaf973cd |
Read release evidence and telemetry without unused output (#17485)
Read execution release evidence and terminal telemetry without unused output. Keep full stored history, in-flight guards, and accepted messages. Validated with real PostgreSQL tests, build, typecheck, green CI, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
991c47bc6e |
fix(server): project recovery result metadata (#17481)
Keep full execution output in storage while selecting only recovery metadata for repeated latest-run and legacy eligibility checks. Retain complete results for provider-text classification and preserve all admission and write fences. Verified 456 focused PostgreSQL tests, full build and workspace typecheck, all 55 exact-head CI checks, and Greptile 5/5 with medium impact and no findings. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
fe85fd7ac4 |
Expire closed workspaces that open descendants do not use (#17464)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Git cache expiration removes closed issues' worktrees and saves
restore points for unsaved work
> - It treated a workspace as closed only when the source issue and
every descendant were done or cancelled
> - A routine that closes a parent and leaves children open for days
therefore kept every parent worktree, although the children work in
their own workspaces
> - The idle path did not help, because the children are updated every
day
> - This pull request counts only the issues that use the workspace when
it decides that a closed workspace can expire
> - The benefit is that closed parent worktrees are removed while their
children work elsewhere, and a child that later binds to the workspace
gets it back from the restore point
## Linked Issues or Issue Description
Refs #17271. Bug report described in-PR (bug report template).
**What happened?**
On one instance, an hourly routine creates a parent issue, works it in
its own worktree, marks the parent done, and leaves child issues open
(`blocked`, `in_review` or `in_progress`) for days. The children work in
their own workspaces. They are not bound to the parent's workspace. Git
cache expiration kept every parent worktree (about 300 MB each), because
the closed-tree check and the workspace reaper required all descendants
to be closed. In one project, 18 of 19 long-closed parent worktrees were
kept only by open children that did not use them. The idle path never
ran, because the children were updated daily.
**Expected behavior**
A closed issue's worktree expires while descendants are open, as long as
no open descendant uses that workspace.
**Steps to reproduce**
1. Enable git cache expiration and restore points.
2. Close an issue that has an isolated git worktree, and leave a child
issue open that works in its own workspace (or has no workspace
binding).
3. Wait past the terminal delay and run the workspace reaper: the parent
worktree stays, and the reaper counts it as `skippedNonTerminalTree`.
**Paperclip version or commit:** `master` at
|
||
|
Devin FoleyandPaperclip
|
3a2c56780c |
fix(server): avoid execution payloads in run authority checks (#17467)
## Thinking Path > - Paperclip manages AI agents and their tasks. > - Each operation must use current task and user authority. > - Chat recovery also checks the exact source-run ancestry. > - These checks can repeatedly read full run payloads. > - Their decisions need identity fields and two typed task keys. > - This change selects those fields and keeps the current checks and locks. > - Complete execution context and output remain stored and available. ## Linked Issues or Issue Description **What happened?** Ancestry checks, instruction identity reads, task locks, and identity activation receipts transfer full execution context or full run rows for metadata decisions. **Expected behavior** Select only required identity and binding fields. Keep fresh membership and key revocation checks, task-before-run locking, native task precedence, and strict validation of legacy JSON task identifiers. **Steps to reproduce** 1. Store a run with large context, Runner profile, and output fields. 2. Read chat recovery ancestry or authorize an instruction operation. 3. Initialize or refresh its run identity. 4. Observe full context or full-row responses in these metadata paths. **Paperclip version or commit** Baseline: `14e1f59fbf4d9cc7875ed35ebaa9e997f6e2ea7c`. Complementary work: Refs #17460 and Refs #17463. Searched open and closed PRs for ancestry projections and run read optimizations. No duplicate was found. This PR has no dependency on #17463. **Deployment mode** External PostgreSQL. Tests and measurements use disposable real PostgreSQL. ## What Changed - Share a SQL projection of typed legacy issue/task keys with task-binding readers. - Select only source identity and retry links while checking chat ancestry. - Select current identity fields during instruction authorization. - Narrow identity task-lock reads and activation receipts to task binding fields. - Preserve existing authorization, ancestry proof, revocation checks, and lock order. ## Verification - **72 focused tests pass** across read projections, run identity, instruction authority, and chat admission retry. - **49 assembled heartbeat recovery tests pass** for chat and automatic ancestry; 359 unrelated tests were outside the filter. - Real PostgreSQL tests include large payloads, task/company/agent mismatch, current membership revocation, malformed JSON types, native binding precedence, and invalid legacy identifiers. - Real ancestry service SQL: 526,477 to 306 serialized response bytes (99.942% smaller) with three 174,764-byte incompressible execution fields. - Every selected non-context value matches the baseline. Both complete service calls return the same result. Typed binding cases are checked in PostgreSQL. - Five alternating warm samples after two warmups: median 2.152 ms before, 0.812 ms after. These are local query/client timings, not hosted request latency. - Full build and recursive typecheck pass. The full local stable suite is running; no full local pass is claimed. Current-head CI and review must pass before merge. - The first new-fixture run failed eight tests because it reused a unique company prefix. Each fixture now has its own prefix; the focused rerun passes all 72 tests. - All current-head CI checks pass on `ecc951b25d67fddc7dc2e0e3e2de861a40ee5ed0`. Greptile: **5/5, medium impact**, no findings or unresolved threads. - Staging effect is not measured yet. ## Risks Missing identity fields could change an authorization or recovery decision. The tests exercise current authority and fail-closed behavior. The change keeps full execution reads intact, preserves JSON types, and does not cache authority. There is no migration, message deletion, timing change, or data truncation. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
7bd449dbb6 |
fix(recovery): project context evidence for latest-run checks (#17463)
## Thinking Path > - Paperclip manages AI agents and their tasks. > - Recovery sweeps decide whether tasks have a valid next step. > - Those decisions inspect saved run metadata. > - Latest-run queries also return full prompts and execution setup data. > - This repeats large responses when a sweep leaves a task unchanged. > - This change projects the context evidence recovery actually reads. > - A row-version check keeps stale evidence from changing a task. ## Linked Issues or Issue Description **What happened?** The latest-run recovery queries retrieve large stored contexts for each candidate. The decision usually consumes small recovery, message, or workspace-mode fields. **Expected behavior** Keep all decision evidence while selecting a small response. Preserve complete stored contexts and full execution reads. Detect concurrent source changes. **Steps to reproduce** 1. Store a terminal run with large task Markdown, wake, workspace, and continuation bodies. 2. Call `reconcileStrandedAssignedIssues` for its assigned task. 3. Observe the full context returned by the latest-run query on each sweep. **Paperclip version or commit** Baseline: `410e5c6eedba98588a87e4104153898fb0437532`. Related read optimization: Refs #17405. Complementary fix: Refs #17460. These PRs have no code dependency. Searched existing PRs and issues for run projections, recovery reads, and database transfer. No duplicate was found. **Deployment mode** External PostgreSQL. Verification uses disposable real PostgreSQL and synthetic data. ## What Changed - Project typed context evidence in all three latest-run recovery queries. - Keep only the used workspace mode and external-interaction flag from nested setup objects. - Preserve original JSON types, message references, retry evidence, and existing query scopes and order. - Read PostgreSQL `xmin` with the run. Require the same row version during native blocked-wait repair. - Retain the original full-context comparison for callers that supply a full run without a row version. - Test large payloads, malformed JSON shapes, review-message recovery, and concurrent source edits. ## Verification - Final-source integration checks: **450 passed** across context projections, native finalization recovery, and the assembled heartbeat recovery service. - Focused issue-lock race check: **9 passed** (401 outside the filter), including omitted-context and late-output edits and successful repair on a fresh sweep. - Final-source recursive typecheck and full build pass. The full local stable suite is still running; current-head CI must also pass before merge. - Real-service SQL capture against the baseline: one returned row falls from 699,724 to 573 serialized JSON bytes (99.918% smaller). - The fixture uses four 174,764-byte incompressible execution fields. All unaffected returned columns compare equal. Regression tests check the retained decision evidence and unchanged stored records. - Five alternating warm samples after two warmups: baseline median 1.479 ms, treatment 1.084 ms. Ranges overlap; no request-latency speedup is claimed. Plans were captured outside the timing interval. - An initial full-context hash projection cut bytes but raised the local median to 7.457 ms. The final row-version check removes that cost. - One integration run overlapped the hash-export rename and failed seven tests on the removed export. A fresh final-source run passed all 450 tests. The earlier mixed-source failure remains recorded and is not counted as a pass. - All current-head CI checks pass on `17add20e391c30ed0496f4b39eb9c59779efa6b4`. Greptile: **5/5, medium impact**, no findings or unresolved threads on that head. - Hosted transfer and product behavior remain to be verified on staging after merge. ## Risks The context selection must track its recovery consumers. Tests preserve typed flags, retry budgets, external-chat references, and malformed-input behavior. The row-version guard is conservative: any intervening run update defers native blocker repair to a fresh sweep. Concurrent omitted-context and output edits are covered through the real service and issue-lock wait; a fresh sweep must still repair. There is no migration, prompt truncation, history deletion, or authorization cache. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
60ca3a903a | Keep warm sessions across project revision transitions (#17461) | ||
|
Devin FoleyandPaperclip
|
14e1f59fbf |
fix(heartbeat): avoid large run reads during deferred-message recovery (#17460)
## Thinking Path > - Paperclip manages AI agents and their tasks. > - Deferred messages must survive restarts and execution cleanup. > - Periodic recovery checks the latest run before it attempts release. > - That check and two release checks load large execution records. > - These checks use only admission evidence and completion identity. > - This change selects those fields while keeping the release policy and fresh reads. > - Full execution records remain available for execution and detail reads. ## Linked Issues or Issue Description **What happened?** Deferred-message recovery repeatedly transfers stored context, Runner profiles, and output excerpts to make small admission and release decisions. **Expected behavior** Select the fields each decision consumes. Preserve queued messages, current admission checks, cleanup ownership, and fresh completion reads. **Steps to reproduce** 1. Save a terminal run with large context, profile, and output fields. 2. Leave an accepted message deferred after the issue lock is released. 3. Call `resumeQueuedRuns` and follow execution release. 4. Observe full-row reads before the admission decision. **Paperclip version or commit** Baseline: `410e5c6eedb`. Related read optimization: Refs #17405. Searched open and closed PRs for run projections and stranded-queue reads. Searched open issues for database transfer. No matching fix was found. **Deployment mode** External PostgreSQL. The regression tests use disposable real PostgreSQL. ## What Changed - Select seven admission fields for the stranded queue's latest-run lookup. - Select authentication classification evidence before execution release. - Read completion status and typed issue identifiers after post-commit effects. - Keep company scoping on both release reads. - Add large-payload and fresh-state regressions. Document the read contract. ## Verification - Focused PostgreSQL tests: 68 passed across queue, run control, and the new release-read suite. - Release reads return less than 2 KiB with three large synthetic execution fields. - Queue recovery passes less than 512 bytes to release, retaining the full stored message. - Coverage includes legacy/native owners, paused recipients, live successors, authentication repair, and fresh completion identity after post-commit effects. - Root recursive typecheck and full build pass. The full local stable suite remains in progress; no local full-suite pass is claimed. - All current-head CI checks pass on `c819fbb6b46bd5480a197407f06e02f245667792`. Greptile: **5/5, medium impact**, zero comments and zero unresolved threads on that head. - The first focused attempt found a syntax error in the new test; it was corrected before the 68-test passing run. A redundant standalone compiler was stopped while the required recursive check continued. - Hosted impact is not measured yet. Staging verification follows merge. ## Risks The projection must retain all admission evidence and preserve JSON types. The release policy still owns transactions, locks, cleanup, and promotion. There is no schema migration, polling change, message deletion, or API change. Configuration repair evidence is read directly, including when unrelated result output is large. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
410e5c6eed | fix(native): use current authority for warm plugin completion (#17444) | ||
|
Nicky Leach
|
988fe0180b | fix(heartbeat): keep source-free project policy stable across turns (#17447) | ||
|
DottaandPaperclip
|
0eb2c964a7 |
fix(agents): allow runner qualification with agent configuration permission (#17448)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent owners configure the harness and computer used for that work. > - A legacy runner upgrade must first qualify an unverified sandbox. > - The upgrade called an instance-admin probe, so a company owner with agent configuration permission received HTTP 403. > - This pull request qualifies the saved computer through the agent's existing configuration authorization. > - Company owners can complete the upgrade while general instance environment management stays restricted. ## Linked Issues or Issue Description Refs #17361. Related: #17122. **What happened?** Upgrade runner displayed for an existing Claude agent on a Daytona Paperclip Computer. Clicking it returned “Instance admin access required” for a cloud company owner. The agent remained on the legacy runner. **Expected behavior** A caller permitted to update the agent can qualify that agent's saved computer and upgrade. Instance-admin access is not required for this agent-scoped operation. **Steps to reproduce** 1. Sign in as a cloud company owner with `agents:configure` and no instance-admin role. 2. Open an existing legacy Claude agent on an unqualified Daytona snapshot. 3. Select Upgrade runner. ## What Changed - Add `POST /api/agents/{id}/qualify-runner` with the existing `agent_config:update` guard. - Reuse the server's effective environment resolver. The submitted environment ID only detects stale state; it cannot select a different computer. - Preserve company binding checks, check binding again at lease acquisition, and record agent-scoped audit activity. - Return qualification status and the lease receipt without instance-level provider metadata or raw provider errors. - Use the new endpoint for the existing-agent upgrade. Keep the generic environment probe restricted to instance admins. - Disable Upgrade runner when an unsaved computer change resolves to a different target, with a save-first explanation. Other draft settings still accompany an upgrade. - Add real-database authorization regression tests and update the API contract and access maps. ## Verification - Red: the existing UI called the instance-wide probe and failed the new agent-scoped call assertions in eight upgrade cases. - Green: 349 related route, provider qualification, runner selection, and UI tests passed. - Green: 12 authorization cases passed with real PostgreSQL, real HTTP routes, real saved role grants, and a simulated provider boot. They cover the cloud owner, removed grants, foreign agent/computer, inherited targets, stale target, invalid body, archived/local target, redacted failure, audit scope, and unchanged agent settings. - Review regression: two cases first reproduced the enabled upgrade button after an unsaved computer change; both now pass, along with all 168 related UI tests. Qualified/unqualified computers and restoring the saved selection are covered. - All 26 OpenAPI cases passed; the 12 real-database authorization cases also passed after rebase. - The host exhausted PostgreSQL semaphores. For the 12 cases, a temporary local-only setup used disposable PostgreSQL 18 in Docker instead of the embedded database launcher. The committed tests use the normal embedded database helper. No authorization decisions were mocked. - `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, map validation, module-boundary validation, and `git diff --check` passed. - `pnpm test:run` was attempted but did not establish a full local pass: the host could not start embedded PostgreSQL clusters. All current-head CI gates passed, including the full suite in CI. The commitperclip review workflow timed out before reporting any executed steps on its first attempt and passed on one retry. - Fresh staging acceptance remains pending deployment. Verify persisted native configuration and a successful runtime test on the existing Daytona computer after merge. Earlier live Daytona proof from #17361 is separate evidence. ## Risks - Qualification can start a fresh, temporary sandbox. It uses the existing probe lifecycle and cleanup contract. - The new path must remain scoped to the saved agent and its company. Denial and redaction tests cover that boundary. General instance probe authorization is unchanged. - No migration, role change, credential change, or automatic agent upgrade is included. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model variant and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7ae81c9fa7 |
fix(slack): support rich questions and complete review delivery (#17255)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Questions and review decisions must work in the conversation where a task started. > - Slack omitted question context and did not publish several review kinds. > - Some answers saved in Paperclip also lost their authorized continuation or their final Slack reply. > - A settled card and a successful run did not prove that the user received the result. > - This pull request preserves complete questions and follows the existing answer and delivery receipts through continuation. > - Users can read the request, submit an authorized decision, and receive the resumed task's result in the same conversation. ## Linked Issues or Issue Description Refs #17045 and #17218. **What happened?** Slack question cards could omit instructions and editable starting text. Checkbox reviews, item verdicts, task suggestions, and connection requests had no publication path. Complex question answers submitted in Paperclip could fail continuation authorization. Several review continuations completed without delivering their result to Slack. **Expected behavior** Show the actual question and context. Use native Slack controls when they preserve the form contract. Otherwise provide an explicit Paperclip review link. Persist one authorized response, resume through the existing delivery path, settle the card, and deliver the useful result to its originating conversation. **Steps to reproduce** 1. Start a task in Slack and ask a text or choice question with instructions and help text. 2. Submit native text and choice responses, then submit a complex form through Paperclip. 3. Accept a checklist, submit item decisions, select suggested tasks, or decline a connection request. 4. Inspect the saved answer, continuation run, original Slack card, and final Slack reply independently. ## What Changed - Preserve full prompts, context, descriptions, recommendations, help, and editable starting text. Use direct buttons for short choices, inline searchable selects for longer lists, and native checkboxes with explicit submission for multiple choices. - Add native custom-answer and validated-text modals, ordinary confirmation buttons, checklist decisions, per-item verdicts with reasons, suggested-task creation/dismissal, and connection decline. Keep governed approvals, permission grants, and forms that cannot be represented faithfully in the full Paperclip review surface. - Keep Slack-specific canonical-validation feedback in its owning platform module. Shared callbacks delegate without interpreting provider presentation. - Reuse the existing canonical interaction transactions, callback receipts, publication queue, and continuation delivery. Share ordinary confirmation handling across Telegram and Slack. Recheck current authorization and commit response receipts atomically. - Settle cards with the saved answer and remove stale controls. Ignore passive Slack link callbacks so opening a task does not display a false answer error. - Validate exact multi-select receipt mappings during native continuation. Recover automation-origin connection replies from their published prompt after validating the durable connection receipt. - Keep oversized multi-select handoffs free of unused native tokens, and retain the complete Paperclip form for 101–128 choices plus Other. Make the original-conversation audience explicit before submitting a Slack modal. - Add rollback, provider serialization, callback, continuation, and negative authorization regressions. Update the connector playbook and feature/access maps. No React or application UI changes are included. - Add a durable Q01–Q11 qualification index and provider coverage matrix. Require distinct functional, good-experience, and best-available-control assessments; real unanswered/answering/settled evidence; saved results, continuation and navigation proof; and retained failures. Mark other providers unaudited rather than assuming Slack parity. ## Verification Candidate: `e0017d446b1016a38d65382366cd1408cce33039`. The eleven-core-format live Slack gallery is pinned to `c3d458b2a06b660448940fd8cf33c5906f0058d5`; subsequent boundary fixes and the sharing notice have automated verification, not new live screenshots. - Real Slack UI testing covered all 11 formats: text, closed choice, long single select, multiple choices, choices with a custom answer, pattern-validated text, confirmation, checklist, item verdicts, suggested tasks, and connection requests. Each has unanswered/answering/answered screenshots as applicable, an inspected saved result, one continuation, and an individual experience review in the local HTML gallery. - The dedicated private QA app/channel used signed callbacks to the local candidate. A separately approved password-protected, read-only preview proved connection-review, suggested-task review, and created-task navigation. Anonymous access, writes, and credential routes were denied. No OAuth grant was attempted. - Real testing exposed and repaired a multi-select continuation failure and a misleading error on link-button clicks. Failed attempts remain in the evidence. The final navigation replay opened the exact pending connection request and declined it in Slack without granting access. Suggested-task creation produced the selected child with the correct parent and description. - Final review-fix verification: **32 form tests**, **28 provider/form-rendering tests**, **30 publication tests**, and **171 native continuation/authorization tests** passed. Earlier **3 atomic-resolution rollback**, **9 targeted callback/publication/Telegram**, and **28 review-continuation** cases passed. The final Slack/Teams modal subset also passed. The automation-origin connection cases, 101/128 custom-choice cases, and oversized multi-select continuation reproduced before their fixes and passed afterward; failing logs remain retained. These counts describe relevant runs, not a reliability percentage. - CI on `b5885b85fd` exposed a test fixture that assumed every Slack modal block was an input; the new sharing notice correctly renders a non-input block. The fixture now selects input blocks safely. The exact signed-modal correction/one-continuation test passed locally, together with 14 Slack form/modal tests. The failing CI log and its specific cause remain recorded. - `pnpm -r typecheck` and `pnpm build` passed. Server typechecking was repeated after the final source changes. Map and module-boundary checks and `git diff --check` passed. - The full `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL semaphore exhaustion on this host. Its 1,395-test chat suite found two stale native-control assertions; both were corrected and re-tested in a focused run. Focused database suites used separate disposable databases on existing local PostgreSQL. The full suite is not claimed green; new-head CI must complete before merge. - Final-head CI passed on `e0017d446b1016a38d65382366cd1408cce33039`: all test/build jobs and required `ci / verify` and `ci / e2e` gates are green ([run](https://github.com/paperclipai/paperclip/actions/runs/40115430194)). Greptile and Ponytail both reviewed this exact head at 5/5, the security scan passed, and all review threads are resolved. - All three QA app callbacks were restored to staging. Both temporary tunnels and local proxies were closed. The local endpoint was archived and its local credential copies removed. The candidate app was stopped. Limits: provider execution used a deterministic Claude protocol fixture, not an LLM. This is local real-provider UX proof, not deployed staging or broader Codie/Claude qualification. The public preview intentionally blocked ancillary artifact routes, so its destination pages could show a partial-history warning. The complete permission-grant flow and every oversized/mixed-form branch were not driven manually. Earlier fixture, test, and provider failures remain recorded. ## Risks - New reply routes require durable origin proof. Incorrect receipt matching could lose a reply or route it incorrectly. Tests cover missing, forged, mismatched, and revoked authority. - Governed forms keep their Paperclip resolution checks. This change does not grant new approval authority to Slack. - Existing completed review results can become deliverable through the repaired publication path without another provider run. - No schema, lockfile, workflow, public task-status model, or runtime-instruction change is included. This branch is not deployed. ## Model Used OpenAI Codex, GPT-6 family, with code execution and tool use. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the new head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the new head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b7506671b9 |
fix: preserve qualified snapshots during native setup validation (#17445)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agent setup can validate a selected sandbox before the agent runs. > - A qualified snapshot binds the saved environment and resolved runtime to an immutable provider snapshot. > - Setup validation changes lease retention so each test starts fresh and leaves an archived sandbox for inspection. > - The test passed that changed config as the saved environment, which made valid snapshot proof appear stale. > - This pull request keeps saved qualification separate from the two test retention settings. > - Native setup can use the qualified snapshot and still clean up its test lease. ## Linked Issues or Issue Description Related: #15997, #17262. **What happened?** Native setup validation can return `environment_lease_acquire_failed` for a sandbox whose saved snapshot qualification is valid. The test changes `reuseLease` and `archiveOnRelease` before the runtime checks the saved configuration fingerprint. **Expected behavior** The test must qualify the original saved environment, acquire a fresh lease from its immutable snapshot, and preserve archive-on-release in the lease metadata. A changed image, provider, runtime config, or stale proof must still fail. **Steps to reproduce** 1. Save a Daytona snapshot environment with `reuseLease: true` and `archiveOnRelease: false`. 2. Store valid company-scoped snapshot qualification for the saved and resolved runtime configurations. 3. Run native agent setup validation against that environment. 4. Observe that the validation retention overrides invalidate the saved fingerprint before a provider lease is acquired. **Paperclip version or commit** Reproduced from master at `785d54636e1c8513ee5503d602daf6a5ee1970d6`. The fix is based on `62f60bf158746c5807763778326b1eba4f2ed18a`. **Deployment mode** Any server with a qualified native Daytona sandbox environment. ## What Changed - Pass the original saved environment to native setup lease acquisition. - Allow only `reuseLease` and `archiveOnRelease` to differ in the run configuration. Apply those fields after saved snapshot qualification. - Keep both retention fields in the acquired lease metadata so provider release still archives the test sandbox. - Add service-to-runtime cases for valid and stale proof, immutable snapshot selection, fresh lease retention, release behavior, and rejected non-retention overrides. - Extend the route regression to require the original saved configuration. ## Verification - The focused route regression failed before the fix because acquisition had no `configuredEnvironment`. - All 101 focused route, snapshot, and orchestration tests passed on the exact rebased head. - Full repository typecheck and build passed on exact head `15e13e119941090e85638005de54b2d14ea52c00`. - The local database regression could not start PostgreSQL because macOS could not allocate system semaphores. The full `pnpm test:run` attempt also reported database startup failures and was stopped as incomplete. Neither attempt is counted as passing. - Exact-head Linux CI passed all 179 environment runtime integration tests, including the seven new regression cases. Its server shard passed 3,635 tests with 16 existing skips. All exact-head CI gates passed. The final repository review must also complete before merge. - No live provider or model call is part of this verification. This PR does not deploy the fix. ## Risks The acquisition boundary now accepts two explicit retention-only differences. All other config fields, environment identity, company-scoped proof, resolved runtime fingerprint, and template binding remain subject to the existing checks. Regression cases reject changed snapshot, image, provider, and environment variables before provider dispatch. No schema or API contract changes. No user-facing command or documentation contract changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, code execution, and test tools. The session does not expose an exact backend model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
62f60bf158 | Measure first deliverable text in native run traces (#17434) | ||
|
DottaandPaperclip
|
5db5734774 |
fix(plugins): scope operational reads to the caller's companies (#17435)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One instance hosts many companies, and company boundaries are the main isolation contract > - Plugin worker logs, job runs and webhook deliveries store a `company_id` attribution (NULL for instance scope) > - The plugin diagnostics routes checked board org access, but they did not filter those rows by the caller's companies > - A board member of one company could therefore read another company's log messages, run errors and delivery errors > - This pull request scopes those reads to the caller's companies and keeps the instance-wide view for instance administrators > - The benefit is that plugin diagnostics obey the same tenant boundary as the rest of the plugin routes ## Linked Issues or Issue Description **What happened?** A board user who is a member of only company A could read plugin operational rows attributed to company B. `GET /api/plugins/:pluginId/logs` returned log rows with `company_id` values outside the caller's companies. `GET /api/plugins/:pluginId/jobs/:jobId/runs` returned other companies' run rows, including their error text and run logs. `GET /api/plugins/:pluginId/dashboard` returned other companies' recent job runs and webhook deliveries. The routes only required board org access and filtered on plugin id or job id. **Expected behavior** A board member sees only operational rows for companies the member can access. An instance administrator (and the local implicit board) keeps the instance-wide view, which includes instance-scope rows where `company_id` is NULL. **Steps to reproduce** 1. Create two companies, A and B, on an authenticated instance. 2. Install a plugin and produce a log row, a job run and a webhook delivery for company B. 3. Sign in as a board user who is a member of company A only. 4. Call `GET /api/plugins/:pluginId/logs`, `GET /api/plugins/:pluginId/jobs/:jobId/runs` and `GET /api/plugins/:pluginId/dashboard`. Before this change, the responses included company B rows. After this change, they do not. **Paperclip version or commit** Reproduced on `2e6d3f7` (`master`, 2026-10-24). The report came from a scoped security audit of `52a66af29ddccf1fcac25f1f6efa9e9958e11fbd`. **Deployment mode** Self-hosted server, authenticated mode. The defect does not depend on the deployment mode, but `local_trusted` always yields an instance-admin actor and hides it. ## What Changed - Added `pluginOperationalReadCompanyIds()` in `server/src/routes/plugins.ts`. It returns `null` for instance admins and the local implicit board, and the actor's company ids for every other board actor. - Scoped `GET /api/plugins/:pluginId/logs` with `inArray(pluginLogs.companyId, companyIds)`. - Scoped `GET /api/plugins/:pluginId/jobs/:jobId/runs` through a new optional `companyIds` argument on `pluginJobStore.listRunsByJob()`. - Scoped the dashboard's recent job runs through the same optional argument on `pluginJobStore.listRunsByPlugin()`, and scoped its webhook delivery query. - Added `server/src/__tests__/plugin-operational-reads-tenant-scope.test.ts`. It starts embedded Postgres, seeds two companies plus instance-scope rows, and asserts the single-company view and the instance-admin view for all three routes. ## Verification - Red first: with the test added and the route unchanged, 3 of 4 cases failed and the response contained the company B rows. - Green after the fix: `pnpm exec vitest run server/src/__tests__/plugin-operational-reads-tenant-scope.test.ts` passed (4 tests). - Related suites: `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-tenant-isolation.test.ts server/src/__tests__/plugin-install-route-security.test.ts` passed (59 tests). - Typecheck: `pnpm --filter @paperclipai/server typecheck` passed. ## Risks - Behavior change for non-admin board members: they no longer see instance-scope rows (`company_id` NULL) or rows for companies they cannot access. Instance administrators keep the full view. For a single-company instance the visible rows are the same, except instance-scope rows. - No schema or migration change. No API shape change. Row-level filtering only. - A plugin that writes diagnostics without a `company_id` will show fewer rows to non-admin users. That is the intended tenant boundary. ## Model Used - Provider: OpenRouter - Model: `deepseek/deepseek-v4.1-flash-1017` (DeepSeek V4.1 Flash) - Harness: OpenCode, with tool use, code execution, and multi-agent delegation - Reasoning: enabled (thinking mode) - Context window: not exposed by the harness ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
785d54636e |
Delete re-homed isolated clones when their contents are reproducible (#17410)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Isolated task workspaces are Git worktrees on the server disk, and the terminal workspace reaper keeps that disk usage bounded > - #17369 moves old workspaces off their private per-issue clones onto the project's shared clone and then deletes the private clone > - On one instance, the first live sweep moved 12 workspaces but deleted none of their clones (about 600 MB each), and 3 workspaces could not move because their restore point was "too large" > - The kept clones are not referenced by any workspace, so no other cleanup removes them > - This pull request replaces the clone check with a real reproducibility check, sweeps the orphaned clones, lifts the restore point limits for local transfers, and fixes a false `source_identity_changed` > - The benefit is that the disk space #17369 was meant to reclaim is reclaimed, without deleting unique work ## Linked Issues or Issue Description Refs #17369. Bug report described in-PR (bug report template). **What happened?** On one instance, with `PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED=true` and dry run off, the first sweep re-homed 12 workspaces, skipped 5 with `live_or_unresolved_consumer` and 3 with `restore_point_too_large`. No clone was removed: 11 logged `cloneKeptReason: "clone_checkout_has_files"` and 1 logged `"source_identity_changed"`. Each clone's own checkout was clean (`git status --porcelain --untracked-files=all` empty). Its only history that the shared clone lacked was remote branches and release tags that the shared clone never fetched, which exist on the remote. An operator removed 11 clones by hand after verifying them. **Expected behavior** A re-homed clone whose checkout is clean and whose local history is in the shared clone or on the remote is deleted. A clone with real unique work is kept. Workspaces whose changes exceed the upload limits still move, because the transfer is local. **Steps to reproduce** 1. Use a project repository whose `.gitignore` (or the clone's `info/exclude`) contains `.paperclip/`, with a release tag on a commit that is not on any branch. 2. Re-home an isolated workspace of that repository with #17369. 3. The old worktree is removed, the empty `.paperclip/worktrees` directory stays, and `git ls-files --others --ignored --directory` lists `.paperclip/`. The clone is kept with `clone_checkout_has_files`. Without the ignore line, the local release tag keeps it with `unique_clone_history`. 4. Add a file larger than 64 MiB to a workspace. The move skips with `restore_point_too_large`. **Paperclip version or commit:** `master` at |
||
|
DottaandPaperclip
|
71c04c0bfb |
fix: stop idle environment cleanup polling (#17145)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent environments can leave pending teardown or unverified provider resources. > - Recovery must preserve those resources until ownership and workspace safety allow cleanup. > - Empty maintenance scans keep an idle instance and its database active. > - The existing rows, transaction signals, and scheduler can represent the actual recovery obligation. > - This pull request schedules pending cleanup and historical verification only while work remains. > - An empty or fully verified instance stops these scans without losing cleanup recovery. ## Linked Issues or Issue Description **What happened?** The server scanned pending environment cleanup on its maintenance tick. The run reaper also scanned it even when it recovered no runs or active leases. Historical resource verification scanned every five minutes, including when all history already had valid cleanup proof. **What did you expect?** Writes should wake cleanup. Pending rows, deferred orphan buffers, and unverified history should retain recovery deadlines. When none remain, these workers should make no periodic database or provider calls. **Steps to reproduce** Start a configured instance with no pending leases and fully verified history. Leave it idle. Observe repeated pending-cleanup and historical-verification reads from the maintenance paths. Related: #17106 supplies the preceding login-cleanup migration. #13523 improves lease-reaper query indexes. #16809 addresses reusable-sandbox recovery authority. This PR keeps those authority checks in their existing services. ## What Changed - Register two workers with the existing delivery coordinator and shared scheduler. Remove their unconditional maintenance-tick calls. Run the reaper's inline pending cleanup only after it recovers actual work. - Add commit-aware signals to the audited lease and custom-image writers. Existing transactions notify directly; other writers use one small transaction helper. No tables or migrations are added. - Derive pending cleanup deadlines from the existing row backoff and retry or claim metadata. Retain a historical retry only while an unverified candidate exists. - Register unwritten orphan buffers across runtimes that share the database handle. Buffer registration wakes recovery without a database connection. Load the existing spool at startup and retry failed or capacity-limited loads while needed. - Isolate spool failures per runtime. Attempt all buffers and persisted pending rows, then retain a retry for any failed spool. - Document producer coverage, idle behavior, and remaining blockers. Update the sleep inventory. - Repair the inherited CSS-recovery E2E test: the current build no longer emits task-page CSS. Exercise the actual lazy environment stylesheet, preserving real network failure, recovery, and bounded reload assertions. ## Verification October 24 integration update: merged current master and resolved the runtime/spool conflicts. Preserve serialized ingestion of records appended after startup. Plugin replacement failures now register deferred recovery with an idle hold; concurrent callers share one persistence attempt. The producer audit includes this new writer. - Focused cleanup selection: 270 tests now pass across five suites. The initial run lacked a built plugin SDK and executed no tests; building the SDK fixed setup. The new concurrent-flush assertion then reproduced duplicate rows; a single-flight guard fixed it and all 18 plugin integration tests passed on rerun. - Server typecheck passed. After review fixes, all 211 tests across the cleanup worker, real orphan spool, environment runtime, and plugin integration suites passed. These overlap the earlier focused selection. The rate-limit test initially expected a write after twenty idle minutes to wait; corrected it to assert that subsequent writes cannot accelerate the five-minute cadence. - Review fixes preserve retries for unreadable spool directories/records, keep historical provider checks off the startup wait while joining them at shutdown, and restore their independent five-minute minimum spacing. The complete [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/40108905033) passed on final commit `79ae1589c3c7b924db4bedb337d0c7ed3d95020b`, including all E2E shards, server/workspace tests, runner checks, typecheck, build/public-install verification, and security checks. - CI found a stale E2E asset matcher: `issue-detail-*.css` is no longer emitted. Both corrected browser tests passed against a fresh production UI build and disposable local instance. Fresh Greptile review is 5/5 on that same final commit, with all review threads resolved. Previous implementation verification: - `pnpm -r typecheck` and `pnpm build` passed. - New PostgreSQL worker suite: 12 tests passed. Covers quiet configured state, verified history, writes, outer commit and rollback, retry deadlines, restart, drain, deferred orphan buffers, spool failure, pagination, provider uncertainty, shutdown, and the idle run-reaper path. - Worker and environment runtime suites after review fix: 183 tests passed. A new assertion reproduced buffer starvation before the fix. Updated transaction fault injection and isolated simulated-instance database handles. Server typecheck also passed after the fix. - Run-recovery suites: 58 tests passed. Pending-cleanup suite: 38 tests passed after updating the logging fixture to seed real orphan recovery; an idle reaper no longer invokes cleanup. - Stop and workspace-recovery integration selection: 268 tests passed; two live-provider tests skipped by their environment gate. This overlaps the worker suite and is not an additional unique-test count. - Started the full local `pnpm test:run`, then stopped that older duplicate run after CI found the fixture mismatch. The corrected 38-test suite passed locally. The full sharded [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39678174479) passed on final commit `fc0ff3c4554ca4190a8b0b81ec06ef7065b8ad2c`, including build, typecheck, runner, server, workspace, and E2E checks. This is not a claim that the local full run completed. ## Risks - Missing a future producer signal could delay discovery until restart. The producer audit is documented in `doc/idle-environment-cleanup.md`. Independent SQL writers and another server process are outside the existing process-local notification contract. - Historical candidates blocked by ownership or provider uncertainty keep a five-minute retry until they can be verified. Provider destruction, claim fencing, Stop retention, and workspace safety remain controlled by their existing services. - This does not remove orphan-active discovery or legacy shutdown settlement from run recovery. Active custom-image expiry is also still periodic. External wake coverage does not change. - Local tests prove these workers become quiet and recover. They do not establish deployed instance suspension or fleet cost savings. ## Model Used OpenAI Codex (GPT-6), with code execution and reasoning. The exact serving model ID and context size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
e5c9e60c5a | Avoid session resets when reusing default task directories (#17419) | ||
|
Nicky Leach
|
f7716ec65b | Allow exact warm-state reads during retained task retirement (#17418) | ||
|
Nicky Leach
|
6b4bdc515b | Fix GitHub ownership checks for warm plugin tasks (#17417) | ||
|
DottaandPaperclip
|
87bf89756e |
fix: support Daytona runner upgrades and ignore obsolete Claude options (#17361)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Existing Claude and Codex agents can upgrade to Paperclip Runner. > - A Daytona computer needs platform qualification before Runner becomes its default. > - The upgrade notice required that default, so it hid the action that could start qualification. > - Saved Claude turn limits also caused conversion to fail even though Runner can run without them. > - This pull request verifies the computer on click and ignores obsolete Claude options with a warning. > - Agents retain their model, account, environment, instructions, and permission policy. ## Linked Issues or Issue Description Related: #17122. **What happened?** A legacy Claude agent on a Daytona snapshot had no upgrade button until its environment had platform qualification. A saved `maxTurnsPerRun: 1000` then caused conversion to return HTTP 422. **Expected behavior** The upgrade action verifies a supported Daytona computer and converts the agent. Unsupported tuning options produce a visible warning and do not block execution. **Steps to reproduce** 1. Create a legacy Claude agent with `maxTurnsPerRun: 1000` on an active Daytona snapshot. 2. Open Runtime before the environment has platform qualification. 3. Try to upgrade to Paperclip Runner. **Paperclip version or commit** After #17122, reproduced against `9c442cf48e`. ## What Changed - Offer upgrade on probe-capable Daytona snapshots and immutable sandbox images. Perform the public environment probe only after a click. Save after fresh adapter discovery confirms support. - Resolve the actual computer for explicit, instance-default, and hidden managed environments. Preserve pending edits and reject stale probe results. - Ignore Claude `maxTurnsPerRun`, `chrome`, `warmHandleIdleMs`, and `acpWarmHandleIdleMs`. Return structured warnings from create, hire, and update. Show warnings in Runtime and after a save. - Keep validation for commands, credentials, filesystem, network, and permission settings. - Retain the legacy turn limit in the existing live conversion fixture. Update regression tests, upgrade stories, and capability maps. ## Verification - Red: the old route returns HTTP 422 for the saved turn limit; the old form hides the Daytona upgrade notice. - Green: 417 focused server and form tests pass. A further 188 tests pass after the review fixes. Qualification coverage includes failed probes, unsupported targets, changed settings, managed defaults, unsaved environment variables, and forced Kubernetes targets. - `pnpm -r typecheck` and `pnpm build` pass. Token gates, map checks, map tests, and Storybook catalog checks pass. - Real local Claude Opus 5.5 conversion and task pass, with the downloaded artifact checked against seeded source bytes, SHA-256, and Git commit. - Real browser upgrade and full Claude Opus 5.5 task on a disposable named Daytona snapshot pass at `b81cec7d8c`. The saved agent retains its model, high effort, credentials, timeout, and grace settings while dropping the turn limit with a visible warning. Exactly one successful run completed the task. The downloaded artifact matched independently seeded source bytes, SHA-256, and Git commit; the source was unchanged, no interactions remained pending, and the delivery link survived a browser reload. The snapshot used current runtime assets built from public source. The supplied Daytona account could not access the original staging snapshot, so this verifies an equivalent local fixture rather than that exact snapshot. Disposable test resources were cleaned up. - Earlier live attempts remain recorded: an image-based sandbox creation failed, and the first named-snapshot probe returned failure with unresolved provisional cleanup identity. Later named-snapshot probing and the task succeeded; the initial creation failures are not attributed to this PR or declared resolved. A separate attempt using the Oct 18 cached runtime failed the current provider-pack compatibility check; rebuilding the runtime from matching public source removed that mismatch. Owned resources were identified and cleaned up, with diagnostic databases retained. - 54 checks passed at `b81cec7d8c`, with no failed checks; Greptile and Paperclip Review both scored 5/5, with both review threads resolved. Master advanced during the live proof, so the branch was rebased with both sides of additive import/test conflicts retained. On the rebased head `2b74222b94`, 472 focused tests, UI typecheck, and UI build pass. All required CI gates pass; Greptile and Paperclip Review both score 5/5 on this head, with no unresolved review threads. The PR is mergeable. The serial local `pnpm test:run` was stopped after the CI test lanes passed; that local run is not claimed as a pass. ## Risks - An explicit upgrade can allocate a temporary probe sandbox and take several minutes. Probe failures leave the agent on its existing runner. - Ignored options no longer enforce a turn cap, Chrome integration, or legacy idle-time tuning. The UI warns about each active ignored option. - Unsafe or ambiguous execution settings still fail validation. This change does not weaken environment qualification or runtime artifact checks. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
515ff0f284 |
fix(server): report which S3 call failed in storage error events (#17424)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server keeps attachments and other files in a storage provider. Hosted instances use the S3 provider. > - Sentry received `SignatureDoesNotMatch` errors from the S3 provider on two staging hosts. > - The events have only AWS SDK stack frames. They do not show which S3 call failed, the HTTP status, or the S3 request ID. > - Without these facts, we cannot find the cause or make a safe fix. > - This pull request records a small, fixed-shape failure record on the S3 error and sends it with the Sentry event. > - The benefit is that the next event shows the failed operation and the provider request ID, and no private data goes to Sentry. ## Linked Issues or Issue Description **What happened?** The S3 storage provider threw `SignatureDoesNotMatch` four times on staging. The Sentry events do not identify the S3 operation or the caller. **Expected behavior** A storage provider failure in Sentry shows which storage call failed and the provider's correlation fields. **Steps to reproduce** 1. Configure the S3 storage provider against an endpoint that rejects the request signature. 2. Read, write, check, or delete a stored file. 3. Look at the Sentry event. It has no first-party stack frame and no operation name. No public GitHub issue exists. I searched open and closed pull requests for `SignatureDoesNotMatch` and storage failure diagnostics. No pull request covers this. ## What Changed - Add `server/src/storage/failure-diagnostics.ts`. It keeps a frozen, fixed-shape record for each failed storage error in a `WeakMap`. - Add `server/src/storage/s3-failure.ts`. It wraps each S3 call and records the operation, error code, HTTP status, request ID, and SDK attempt count. It reads only data properties. A value that does not match its bounded format becomes `unknown` or `null`. - Wrap `PutObject`, `GetObject`, `HeadObject`, `DeleteObject`, `CreateMultipartUpload`, `UploadPart`, and `CompleteMultipartUpload`. - In `captureException`, add the `storage_operation` and `storage_error_code` tags and a `storage_failure` context to the event of that error only. The original error and its default grouping do not change. - Document the new context in `doc/observability.md`. ## Verification - `pnpm exec vitest run src/__tests__/storage-s3-failure-diagnostics.test.ts src/__tests__/sentry.test.ts` (in `server/`): 37 passed, 5 skipped. The skipped tests are the existing real-SDK tests, which need an environment flag. - `pnpm exec vitest run src/__tests__/storage-s3-multipart.test.ts src/__tests__/storage-local-provider.test.ts`: 8 passed. - The new tests use a local HTTP server that returns a real S3 `SignatureDoesNotMatch` XML body with a `StringToSign` field. The tests make sure that the record has the operation, code, status and request ID, and that it does not contain the private string. - I typechecked the changed source files with a scoped `tsc` project. They have no errors. I could not run the full `pnpm -r typecheck` and `pnpm build` in my sandbox: it has a 4 GiB memory limit and no `cargo` for the runner binary. CI runs the full checks. ## Risks - Low risk. The change adds a `try`/`catch` that records data and throws the same error again. Storage behavior, not-found handling, and HTTP responses do not change. - Secret-sensitive routes replace the error before reporting, so those events do not get the record. This is intended. - The record never contains the bucket, object key, endpoint, credentials, error message, or provider response body. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude Code with tool use (shell, file edit) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5e26b733b9 |
fix(server): finish workspace operations when the log write fails (#17415)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Before a run starts, Paperclip prepares the execution workspace (for example `git worktree add`). Each step is a workspace operation. The workspace operation recorder writes it as a row and as a log file on disk. > - When an operation fails, the recorder writes the error to the log file, then sets the row to `failed`. > - If that log write also fails (for example ENOSPC when the disk is full), the recorder throws before it updates the row. > - The row stays `running` with no `finishedAt`. The log write error also replaces the real error of the operation. > - This pull request makes the failure-path log write best-effort, so the row always reaches `failed` and callers get the correct error. > - The benefit: on a full disk, the operator sees a terminal operation with the real failure reason. Operations do not stay "running" for ever. ## Linked Issues or Issue Description No public issue exists. Bug description: **What happened?** A run failed during workspace setup on a host with a full disk (`ENOSPC: no space left on device, write`). The stack shows the error in the workspace operation log append inside `recordOperation`. In the failure path, the recorder calls the log append before it updates the row. Because the append threw again, the row stayed `running` with no `finishedAt`. When the operation itself failed (for example a git error), callers got the ENOSPC log error and not the git error. **Expected behavior** A failed workspace operation always reaches `failed` with `finishedAt` set. Callers get the operation's own error. A log write failure must not hide that error. **Steps to reproduce** 1. Make the operation log append throw ENOSPC (the new tests stub `fs.appendFile` for the log root). 2. Record an operation whose `run` throws, or whose output append throws. 3. Before this fix: the row stays `running`, and the caller gets the ENOSPC error and not the operation's error. **Paperclip version or commit** `ee29e28` (current `master`). **Agent adapter(s) involved** Not adapter-specific (core bug). Related open PRs with disk-full handling in other places (no overlap with this code path): #5882 (run-log rotation), #16773 (backup disk use). ## What Changed - `server/src/services/workspace-operations.ts`: in the `recordOperation` failure path, the stderr log append is now best-effort (`.catch(() => undefined)`). The stderr excerpt still records the message, because the excerpt updates before the file write. `logStore.finalize` was already best-effort in this path. - `server/src/__tests__/workspace-operations-reconciliation.test.ts`: two regression tests with log appends that fail with ENOSPC: - The output append fails after a successful run. The row reaches `failed` with `finishedAt` and the ENOSPC message in the stderr excerpt. The caller gets the ENOSPC error. - The operation itself throws. The row reaches `failed` and the caller gets the operation's own error, not the log write error. ## Verification - Red before the fix: both new tests failed on `master`. The row stayed `running`, and the caller got the ENOSPC error and not the git error. - Green after the fix: `pnpm exec vitest run src/__tests__/workspace-operations-reconciliation.test.ts src/__tests__/workspace-runtime-control-recovery.test.ts` passes (17 tests). - Sibling suites that use the recorder (26 files, 945 tests): all pass except 4 tests in `workspace-runtime.test.ts`. Those 4 tests also fail on unchanged `master` in the same sandbox, because they need the local pnpm and worktree CLI. This change does not cause them. - Typecheck: `tsc --noEmit` on `workspace-operations.ts` passes. The full server `tsc` was not run locally because the 4 GB sandbox ran out of memory. CI runs the full typecheck. ## Risks - Low risk. The change affects only the failure path, and only the log file write. The success path does not change: if a log write fails after a successful run, the operation still fails with the write error. - The log file for a failed operation can lose its last stderr line on a full disk. The row's stderr excerpt still has the message. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ee29e2850d |
fix: keep GitHub guest provenance from failing tool discovery (#17340)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents reach their tools through the MCP tool gateway. Every run starts with `tools/list`. > - Tool discovery runs a GitHub guest sponsor probe (`githubGuestBotConnectionForSession`) before it lists any connection. > - The probe walked the run's identity chain before it knew whether the task had a guest sponsored GitHub message at all. > - When agent A works a task and the task is reassigned to agent B, B's run chain continues A's run. The chain walk refuses with a plain `HttpError`. The gateway returns 403 for `tools/list` with no audit record. B loses every tool, built-in and connected, even when the company has no GitHub endpoint. > - This pull request makes the probe look up guest sponsored GitHub messages for the task and agent first. When there are none, the probe answers `null` and does not consult provenance. > - The benefit is that a reassigned or continued task keeps its tools, while a guest sponsored task still refuses a broken chain. ## Linked Issues or Issue Description Fixes #17307 **What happened?** `tools/list` on the MCP gateway returned `403 {"error":"Chat tools cannot be inherited by another task"}` for every run whose identity chain contained a run of another agent or another task. No audit event was recorded. The agent received no tools at all, including unrelated MCP connections. The company had no GitHub chat endpoint. **Expected behavior** A run on a task with no GitHub guest sponsor lists its tools. The guest probe only decides whether the run must be limited to the sponsoring bot's connection. When the task has no guest sponsored GitHub message, the answer is "no sponsor" and discovery continues. **Steps to reproduce** 1. Let agent A run on a task. The run gets an accepted identity context. 2. Reassign the task to agent B and let B run. B's identity context has A's context as parent. 3. Call `tools/list` for B's run. Before this change the gateway returns 403 with the message above. The same happens when the parent context belongs to a run on another task (see #17307). **Call path** `handleMcpGatewayProtocol` → `listToolsForNamedGateway` → `listToolsForContext` → `buildToolsForContext` → `connectionToolsForContext` → `githubGuestBotConnectionForSession` → `chatRunSourceMessageId`, which throws `forbidden(...)`. The error is an `HttpError`, not a `ToolGatewayHttpError`, so the route sends it as a bare 403 and the discovery audit never runs. **Paperclip version or commit** `d69631b01b9c658f80bf8bed389863dce82643e1` (image `sha-d69631b`), Docker, authenticated mode with PostgreSQL. ## What Changed - `server/src/services/chat-github-tools.ts`: `githubGuestBotConnectionForSession` first selects the inbound GitHub message links on the task whose delivery carries `githubAuthority.guest = true`, for endpoints assigned to the session agent. When there are none it returns `null` without reading the run or its provenance chain. When there are some, it verifies provenance as before and returns the connection of the link that matches the verified source message. - `server/src/services/chat-channels/run-origin.ts` is unchanged. The cross-agent and cross-task refusals stay in place for Slack authority, GitHub reviews and response comments, and for guest sponsored tasks. - New test `server/src/services/chat-github-tools.test.ts` (embedded PostgreSQL, same pattern as `run-origin.test.ts`): - a run whose chain continues another agent, or another task, with no guest message → `null` (fails before this change with the 403 above, passes after); - a guest sponsored task whose chain continues another agent → still rejects with status 403; - a run sponsored by a GitHub guest → returns the bot connection; the same run with `guest: false` → `null`. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/services/chat-github-tools.test.ts`: 2 failed / 3 passed before the fix, 5 passed after. - `pnpm --filter @paperclipai/server exec vitest run src/services/chat-channels/run-origin.test.ts src/services/chat-github-tools.test.ts`: all passed. - `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && pnpm --filter @paperclipai/server exec tsc --noEmit`: clean. - Manual: with a patched image, a run on a task that was reassigned between two agents receives its full tool list through `tools/list` and a `discovery` audit event is written. Before the patch the same kind of run received the 403 and no audit event. ## Risks - Low risk. The probe's result is unchanged for every session where it returned a value before. The only behavior change is for sessions that previously threw: when the task has no guest sponsored GitHub message, they now get `null`, which is the value the old code could only have returned for them anyway, because the final match also required a guest link on this task for this agent. - One extra query per discovery on guest sponsored tasks. On every other session the run and provenance queries are skipped. ## Model Used - Claude (Anthropic), Claude Code agent session; model id `claude-fable-5-1`, extended thinking, tool use (shell, git, vitest with embedded PostgreSQL). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LoF4rrmx35JJqbhEpaLQ9Q --------- Co-authored-by: Ingvar Petrov <ip@protrener.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
1a79522a4e |
perf: narrow run-to-issue authorization reads (#17405)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Run pages show links to issues associated with a run. > - The issue-link route checks the run's company and issue visibility before it reads those links. > - That check needs four binding fields, but the route reads the full run. > - Full runs can include large prompt, output, and result fields. > - This pull request uses the existing narrow authorization read. > - The route transfers less database data and keeps its permission checks and response contract. ## Linked Issues or Issue Description **Problem or motivation** `GET /api/heartbeat-runs/:runId/issues` calls `heartbeat.getRun` for a permission check. Its consumers only need the run ID, company ID, scope kind, and issue ID. The full read also fetches context, output excerpts, result data, and runtime metadata. Repeated link reads pay for that unused data. **Proposed solution** Use `getRunReadAccess`, the existing four-field read from #17348. Keep authentication, company access, company-scope authorization, run issue visibility, and linked-issue filtering in their current owners. Keep the existing empty response for missing and inaccessible runs. **Alternatives considered** A new query contract is unnecessary because the existing read provides all required fields. A permission cache could become stale. Changing the full run reader could affect consumers that need run details. **Roadmap alignment** This is a focused improvement to an existing route. It adds no feature, route, provider, cache, or background work. Searches for run-issue links, narrow run authorization, and `getRunReadAccess` found no open duplicate. Related work: #17348. ## What Changed - Change the issue-link route to use the existing narrow run read. - Reject full run reads in the route tests. Cover readable company and issue scopes, denied issue access, missing bindings, anonymous requests, and the missing/cross-company empty response. - Add a developer note about the read contract. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/activity-routes.test.ts src/services/heartbeat/run-state.test.ts src/__tests__/agent-live-run-routes.test.ts`: 126 tests passed. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - The route tests use the real run visibility helper with only the four binding fields. The full reader throws if a route calls it. The existing run-state suite checks the exact four-column selection and covers database wiring with disposable Postgres. - `git diff --check`: passed. Reviewed the committed diff for secrets and private identifiers before push. - Apex review: 5/5 on exact head `888a4ce9ea88bb2b284eb5e28fafc01b6f48b386`, completed successfully, with zero review threads. - Full current-head CI passed: 54 successful checks and two expected skips. [Workflow attempt 2](https://github.com/paperclipai/paperclip/actions/runs/40051399577/attempts/2). [Initial serialized shard failure](https://github.com/paperclipai/paperclip/actions/runs/40051399577/job/120744710081): the first company-history case in unchanged `agent-live-run-routes.test.ts` hit its 15-second timeout; the shard had 391 passing tests and one failure. That file, its agent route, the heartbeat state reader, and the service export are unchanged from base. The five company-history cases pass on unchanged base `9fb28d06ffebbb615f413765078aeeb7c4291311` (9.12s suite) and this head (8.29s suite). They also passed in the 126-test focused run. The timeout was not reproduced locally; its cause is not established. Only the failed shard was retried on the same head. It passed in attempt 2. This establishes recovery; the initial timeout remains in this record and its cause is unproven. - The prior local full-suite failures and baseline triage remain documented in #17384. No new local full-suite pass is claimed here. This change uses focused local tests and the full current-head CI gate. - No hosted user journey, staging deployment, production deployment, or billed savings is claimed for this follow-up. ## Risks - Run visibility depends on the canonical `scopeKind` and `issueId`. The existing helper returns those same fields. It does not use prompt or result data for access decisions. - Linked issues still pass the existing SQL visibility filter. Missing and cross-company runs still return `200 []`. Anonymous requests still fail before the run read. - The database query count stays the same. The change reduces data selected for this authorization read. - No schema or API response changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
70de7d49e9 |
Migrate standalone nested clones onto the shared project clone (#17407)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Each task runs in a git worktree. Extra project repositories sit in
the task under `.paperclip-repositories/`.
> - #17382 made new nested checkouts git worktrees of one shared clone
per project repository. Existing standalone nested clones stayed as full
clones.
> - Long-lived tasks keep those clones for a long time. Many tasks hold
the same large repository, so most of the disk is duplicate `.git` data.
> - #17369 added a locked and verified move of isolated workspaces onto
the shared clone, with a reaper sweep behind a setting.
> - This pull request adds the same kind of locked and verified
conversion for standalone nested clones in open tasks, and runs it from
the same reaper path behind a new setting.
> - The benefit is that the duplicate clones go away without losing any
state or local history, and the disk stops filling.
## Linked Issues or Issue Description
Refs #17382 #17369. Bug report described in-PR (bug report template).
**What happened?**
On one instance, 44 open task worktrees in one project each hold a
standalone nested clone of the same large repository. Together they use
about 25 GB. About 16 GB of that is duplicate `.git` data. Another
project has 33 smaller clones. The clones are removed only when their
tasks expire, and many tasks stay open for a long time. The disk keeps
filling.
**Expected behavior**
Existing standalone nested clones in open tasks become worktrees of the
project's shared clone, with the same state and local history, and the
standalone clones are deleted.
**Steps to reproduce**
1. Prepare a task with a nested project repository before #17382, or
without a shared clone, so that `.paperclip-repositories/<name>` is a
standalone clone.
2. Create the project's shared clone, as #17382 does for new tasks.
3. Observe that the old checkout stays a full clone until the task
expires.
**Paperclip version or commit:** `master` at
|
||
|
Nicky Leach
|
aa0037a137 | Support Codex warm plugin task reuse and recovery (#17390) | ||
|
Devin FoleyandPaperclip
|
0c3b5ce273 |
perf: filter sandbox provider lookups in Postgres (#17395)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox execution selects a provider from installed plugin manifests. > - Driver-key lookup reads every plugin row before it finds a matching declaration. > - Repeated lookups transfer manifests for unrelated plugins. > - The registry owns this database read and can filter candidates before returning rows. > - This pull request adds that filter and keeps the existing provider selection checks. > - The returned data now grows with matching providers instead of the full catalog. ## Linked Issues or Issue Description **Problem or motivation** `resolvePluginSandboxProviderDriverByKey` calls the unfiltered registry catalog on each lookup. The caller only needs plugins with a sandbox-provider declaration for one driver key. Unrelated plugin manifests still cross the database connection and are decoded in the server. **Proposed solution** Add an optional sandbox-provider driver-key filter to `pluginRegistryService().list`. Match the key and kind on the same JSON declaration in Postgres. Pass the key from the existing resolver. Keep full matching rows, install order, status checks and worker checks. **Alternatives considered** A catalog cache would need an invalidation contract for plugin updates and independent writers. This change keeps fresh database reads. Returning only one SQL row could skip a later eligible provider when an earlier worker is unavailable. The filter therefore keeps all matching candidates. **Roadmap alignment** This is a small performance fix to existing plugin execution. It adds no product feature or provider. Searches for sandbox/driver lookup and plugin catalog/filter work found no duplicate PR or issue. ## What Changed - Add an optional JSONB containment filter to the existing registry list read. - Use it in sandbox-provider lookup by driver key. The unfiltered catalog call keeps its existing behavior. - Add real-Postgres regression coverage and a short developer note. ## Verification - Four focused provider-selection/recovery/seam suites passed: 27 tests. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - The new Postgres tests seed large unrelated manifests and verify the actual resolver SQL filters rows in the database. They cover key/kind matching within one declaration, unfiltered reads, install order, disabled and unavailable workers, live status/declaration changes, unknown keys and bound unusual input. - The filter keeps the existing SQL call count. It does not add a network call, cache, timer, or background worker. - `git diff --check`: passed. Reviewed the staged diff for secrets and private account data before push. - Full PR CI passed on `6f97c984a931416b6347e9885d840557c54d8aab`: 54 successful checks and two expected skips. [Workflow evidence](https://github.com/paperclipai/paperclip/actions/runs/40041294925). - Apex review: 5/5 on that exact head, completed cleanly, with no comments or unresolved threads. Merge state is clean. - The prior local full run on parent work is documented in #17384. It did not pass because of test setup, host Git-default assumptions and transient failures. Those results were preserved and triaged against unchanged code. This change uses focused local checks and full current-head CI; no new local full-suite pass is claimed. - No hosted user journey, live provider throughput, production deployment or billed savings is claimed. ## Risks - JSONB containment checks the stored manifest before returning rows. It still scans candidate metadata in the database; no index or SQL-call reduction is claimed. - Both key and kind must match the same declaration. The regression fixture includes a plugin with those values on different declarations. - Selection still uses the existing install-order and live readiness/worker checks. Matching disabled records remain visible to callers that do not require a running worker. - No schema, API response, provider protocol, file-copy behavior or scheduling changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |