mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
f8cb4957dcd9f46c67f12e0feac0ba7cd1528a42
2325
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin FoleyandClaude Opus 5.5
|
4033f7f17f |
fix(recovery): let a task Retry run keep its bounded transient retry (#19384)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A user can click Retry on a blocked task. Paperclip then admits one new run that continues in the saved workspace > - That run can fail before the provider starts, for example when the download of the saved workspace snapshot gets a transient `ECONNRESET` > - The scheduler classifies this failure as transient and tries to schedule the normal bounded retry > - The automatic-retry admission accepts only continuations that a user comment started. It refuses a Retry continuation (`commentId: null`, `taskRetryRequestId` set) with `continuation_user_authorization_missing` > - So the task does not retry. Paperclip escalates the restore failure, and the task becomes blocked again after one network reset > - This pull request lets a plain task Retry renew its authority for the bounded transient retry, with its own receipt for each new run > - The benefit is that a transient setup failure after Retry uses the normal retry budget and does not block the task ## Linked Issues or Issue Description **What happened?** A run started by the task Retry button failed during setup: `remote_workspace_restore_failed: step=snapshot_download; storage=transport_closed code=ECONNRESET`. The run event said "The automatic retry could not revalidate the original user continuation." (retry reason `transient_failure`, attempt 1 of 2). No retry was scheduled, and the task became blocked. **Expected behavior** A transient setup failure of a Retry-admitted run gets the same bounded automatic retry as a run that a user comment started. **Root cause** `admitExplicitContinuationRetry` in `server/src/services/explicit-native-continuation.ts` required `explicitUserContinuation.commentId` to be a GUID. A Retry continuation has `commentId: null`. Also, dispatch in `server/src/services/execution-continuation.ts` bound a Retry receipt to exactly the run that it admitted, so a successor run could not prove its authority. Refs #19207 (it named remote restore failures and left a board recovery action) Refs #19340 (it retries quarantined tasks in their saved workspace) Refs #19326 (same bounded retry path for an out-of-date Runner) ## What Changed - `admitExplicitContinuationRetry` now accepts two kinds of authority. The comment path is unchanged and moved into `readCommentRetryAuthority`. The new `readTaskRetryRetryAuthority` accepts a plain task Retry only. The Retry request must be coalesced, from a user, with the same generation and actor, strategy `continue`, and no chat, queued-message, canonical-intent or fresh-workspace readiness. Interrupt and failed-run retries stay one-run. - The successor run gets its own `explicit_user_continuation_retry` receipt with the same `taskRetryRequestId` and actor and `automaticRetry.sourceRunId` set to the parent. The Retry request stays bound to the run that it admitted. - Dispatch (`buildExecutionContinuation`) accepts a Retry successor only from its own receipt, its exact `retryOfRunId` parent, the current `executionRunId` and the unchanged Retry intent. The direct path for the admitted run is unchanged. - Tests in `server/src/services/task-retry.test.ts`: one regression for the full retry chain, and eight refusal cases. ## Verification - Red: before the fix, the new test failed with `{ outcome: "not_scheduled", errorCode: "continuation_user_authorization_missing" }`. This is the same refusal as the incident. - Green: `pnpm --filter @paperclipai/server exec vitest run src/services/task-retry.test.ts` gives 58/58. - The regression test proves these points: the first retry is scheduled with `{ previousRunId: parent, commentId: null, taskRetryRequestId }`; the Retry request stays bound to the admitted run; dispatch accepts the successor; a repeated schedule call adopts the same successor without a second receipt; a second transient failure retries from its immediate parent; the budget then stops with `retry_exhausted`; dispatch refuses the successor when its own receipt is removed. - The refusal cases prove that no successor and no receipt are created after: wrong actor, unadmitted request, wrong generation, request bound to another run, chat Retry, fresh-workspace readiness, reassignment, superseding work. - Nearby suites pass (601/601): `execution-continuation`, `legacy-explicit-continuation-recovery`, `explicit-native-continuation`, `execution-continuation-provenance`, `quarantined-task-retry`, `issue-queued-comments-routes`, `task-retry`. - I did not run the layered server typecheck locally because of a disk limit in the sandbox. CI must confirm it. ## Risks - Behavior change: a plain task Retry run that fails transiently now retries automatically within the existing bounded budget (2 attempts). The scheduler still decides which failures are transient. The same rule already applies to runs that a user comment started. - Chat, queued-message and fresh-workspace Retries, failed-run retries and Interrupt keep their one-run contracts. - No schema or migration change. - Documentation gap: `doc/execution-semantics.md` (line 1210) says "One-run Retry and Interrupt intents retain their separate admission contracts; they are not renewable user-message authority." That sentence must change to describe this rule. The author's publication path cannot carry the full 320 KB file, so that one-sentence edit is still pending. The proposed text: "A plain task Retry that continues in the saved workspace renews the same way: the successor's receipt names the same Retry request and actor, while the Retry request itself stays bound to the run it admitted. Dispatch accepts that successor only from its own receipt, its exact retry parent, and the unchanged admitted Retry request, so a transient failure before or during that run, such as a reset saved-workspace download, uses the normal bounded retry budget instead of blocking the task. Chat, queued-message and fresh-workspace Retries, failed-run retries, and Interrupt intents retain their separate one-run admission contracts; they are not renewable authority." > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code as a Paperclip agent, with tool use and code execution (shell, Vitest on embedded PostgreSQL). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (the `doc/execution-semantics.md` sentence above is pending) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
8f79a32898 |
fix(connections): keep agent access consistent through setup and permission changes (#19352)
Task-hosted tool setup kept requester-only defaults, and audience edits could leave runtime installations restricted after permission bindings were widened. Large provider catalogs could also fail before creating an access card. Use the normal agent audience defaults for new tool setup, preserve explicit selections through verified OAuth and draft resumption, and synchronize permission bindings and runtime installations atomically. Enforce agent-edit authority and preserve existing access during additive account reuse. Validation: 915 focused regression tests, typechecks, and builds passed. All 54 CI checks passed and Greptile Apex scored the reviewed head 5/5 with no unresolved threads. Broad local test failures, recoveries, and incomplete heartbeat coverage are documented in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
36643d9378 |
fix(native): retire finished owners whose continued turn rotated authority (#19373)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runner runs per-turn agent sessions in managed sandboxes.
A controller keeps a durable session root for each session.
> - When a per-turn owner finishes without a saved runner checkpoint,
the next turn retires that owner. Retirement first checks that the
session root holds only known entries.
> - The runner transport writes two entries that the check does not
know: `authority-epochs/` and `runtime-context.json`. A continued turn
writes `authority-epochs/` when it rotates authority from the turn
before it.
> - So the third turn of an ephemeral-lease task cannot retire the
second turn's owner. The run fails with
`native_continuation_requires_reconciliation` and the task waits for a
user message.
> - This pull request classifies every session-root entry in one
contract module, and adds a test that scans the writers so the list
cannot drift again.
> - The benefit is that follow-up turns continue without manual help,
and any future refusal names the unexpected entry.
## Linked Issues or Issue Description
Refs #18846, Refs #19061, Refs #19355
**What happened?**
On staging, the third turn of every task for one agent (gpt-5.6-sol,
native runner, ephemeral sandbox lease) failed. The error was
`runner_state_identity_mismatch:
prior_owner_terminal_state_indeterminate;
retirement_refused=unexpected_root_entry`. The first two turns
succeeded. The task then went to `blocked` with "Saved work recovered.
Send a message to continue."
**Expected behavior**
The third turn retires the finished second-turn owner and continues the
conversation.
**Steps to reproduce**
1. Assign a task to a native agent that uses an ephemeral sandbox lease.
2. Let the first turn finish. Reopen the task with a comment so a second
turn runs and finishes.
3. Reopen the task again. The third turn fails with the error above.
**Paperclip version or commit**
Staging near 9a79384958; reproduced locally on master
|
||
|
DottaandClaude Opus 5.5
|
de1143448b |
fix(queue): clear interaction receipts a run already delivered (#19376)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents ask people for input through thread cards: questions, confirmations, plan approvals and connection requests > - When a person resolves a card, Paperclip queues the response for the agent. The task chat shows it above the composer as a read-only receipt, for example "Answered: Questions" > - A second wake can carry the same receipt. One example is a recovery continuation that raced the response's own wake. The agent receives the response from the first run, but the second wake stays deferred > - The composer then shows a receipt that the agent already has. The user cannot edit, discard or clear it. The same receipt also blocks the agent's handback with `interaction_response_queued` > - This pull request treats a receipt as pending only until a run that carries the interaction has started > - The benefit is that an answered card leaves the queue once the agent has it, for every card kind ## Linked Issues or Issue Description No public issue exists. A hands-on onboarding test on a staging cloud workspace (Claude subscription path) found this bug. **What happened?** During onboarding, a queued item stayed above the task-chat composer. It said "Answered: Questions", but the user had already answered the questions. Later items also stayed after the interview answers and after the user connected GitHub. The user could not trash, steer or edit them. Evidence from the staging workspace (interactions, wake diagnostics and run records for the onboarding task): - The first questions card (no title, so its label is "Answered: Questions") was answered at 17:37:48. - Run `ba3bc7d6` (`issue.interaction.respond`) delivered that answer from 17:37:48 to 17:38:27. - A recovery wake (`issue.interaction_continuation_recovery`) for the same card was requested at 17:37:48 too. It stayed deferred behind `ba3bc7d6` and then ran again as `cacb095d` (17:38:33 to 17:39:17). During that time the composer showed the stale "Answered: Questions" receipt. - The plan approval had the same pattern from 17:42:06 to 18:20 (fixed at its source by #19332). The declined Slack connection receipt sat behind that chain. **Expected behavior** A receipt leaves the composer queue as soon as a run has received it. The agent can hand back the task after it has received the response. **Steps to reproduce** 1. An agent asks a question card on a task. 2. A user answers it. The response wake starts a run that carries the answer. 3. A second wake for the same card is deferred behind that run or a later run. 4. `GET /api/issues/:id/queued-comments` returns the second wake's receipt as an immutable entry until it runs. ## What Changed - `server/src/services/queued-interaction-response.ts`: `readQueuedInteractionResponse` returns no receipt when a run for the same issue has started with this interaction in its context and is `running` or `succeeded`. A cancelled or failed run does not count, so that response is still owed. `hasQueuedInteractionResponse` (the handback guard) uses the same rule. - `server/src/routes/issues.ts`: `findQueuedCommentWake` skips a delivered receipt, so a later undelivered receipt can still show. - `server/src/__tests__/issue-queued-comments-routes.test.ts`: red/green tests for every card kind and outcome: answered questions, accepted and rejected confirmation, accepted plan (checkbox) approval, connected and declined connection. Each runs with a `succeeded` and with a `running` delivering run. Controls check that an undelivered receipt still shows, that a later undelivered receipt shows instead of a delivered one, and that the handback guard ignores a delivered receipt. - N-02 (hover shows raw JSON): `master` already shows only the one-line summary in the label and hover (`queuedMessageLabel`). This PR does not change the UI. ## Verification Red, on `master` code with only the new tests added: ``` $ cd server && npx vitest run src/__tests__/issue-queued-comments-routes.test.ts -t "receipt" × clears a ask_user_questions answered receipt once a succeeded run already delivered it 1309ms × clears a ask_user_questions answered receipt once a running run already delivered it 1041ms × clears a request_confirmation accepted receipt once a succeeded run already delivered it 1436ms × clears a request_confirmation accepted receipt once a running run already delivered it 829ms × clears a request_confirmation rejected receipt once a succeeded run already delivered it 666ms × clears a request_confirmation rejected receipt once a running run already delivered it 776ms × clears a request_checkbox_confirmation accepted receipt once a succeeded run already delivered it 1179ms × clears a request_checkbox_confirmation accepted receipt once a running run already delivered it 4227ms × clears a connection_intent accepted receipt once a succeeded run already delivered it 1954ms × clears a connection_intent accepted receipt once a running run already delivered it 1634ms × clears a connection_intent rejected receipt once a succeeded run already delivered it 1505ms × clears a connection_intent rejected receipt once a running run already delivered it 1064ms × shows a later undelivered receipt instead of an earlier delivered one 480ms × does not hold agent handback for a receipt the agent already received 1272ms ⎯⎯⎯⎯⎯⎯ Failed Tests 14 ⎯⎯⎯⎯⎯⎯⎯ AssertionError: expected [ 'Answered: Questions' ] to deeply equal [] AssertionError: expected [ 'Accepted: Approve hiring 2 agents' ] to deeply equal [] AssertionError: expected [ 'Rejected: Approve hiring 2 agents' ] to deeply equal [] AssertionError: expected [ Array(1) ] to deeply equal [] AssertionError: expected [ 'Connected: GitHub connection' ] to deeply equal [] AssertionError: expected [ 'Declined: Slack connection' ] to deeply equal [] AssertionError: expected [ Array(1) ] to deeply equal [ 'Declined: Slack connection' ] AssertionError: expected true to be false // Object.is equality Tests 14 failed | 11 passed | 85 skipped (110) ``` Green, with the fix: ``` $ cd server && npx vitest run src/__tests__/issue-queued-comments-routes.test.ts -t "receipt" Tests 25 passed | 85 skipped (110) ``` Neighbouring suites, with the fix: - `issue-queued-comments-routes.test.ts` (full file, on current `master`): 110 passed. - `question-response-delivery`, `queued-comment-consumption`, `issue-thread-interaction-routes`, `satisfied-connection-intents`, `issue-comment-reopen-routes`, `issue-execution-policy-routes` together with the queued-comments file: 494 tests passed. One run hit an embedded-Postgres `afterAll` cleanup timeout under 7-file parallel load. The same file passed alone twice. - `heartbeat-process-recovery.test.ts -t "plan approval|accepted interaction|answered question|answered interaction|onboarding first task"`: 14 passed. - `pnpm --filter @paperclipai/server typecheck`: passes. ## Risks - Low risk. The change only hides a receipt that a started run already carries. An undelivered receipt keeps its current behavior. - The redundant wake itself is not cancelled here. It still runs later as a normal continuation, but it no longer holds the composer queue or blocks handback. #19332 removes the main source of these duplicate wakes. - Each receipt read adds one indexed-by-company lookup on `heartbeat_runs` for the issue. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, file edits, browser inspection of the staging workspace API). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
9ceaec6f97 |
fix(issues): stop showing resource waits as failed attempts (#19369)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The task page shows one interruption banner. The server builds it
from the task's latest run.
> - An auto-retry that waits for a busy workspace, AI connection or
runner update is not a failure. The dispatcher re-checks the resource,
so the run can stay `scheduled_retry` after its retry time.
> - The projection did not know about these waits. After the retry time
passed, it fell through to "Couldn't finish this attempt" with a Retry
button.
> - Retry cannot free the resource. The banner cleared for the 10-second
grace and then came back. Users had no way to fix it.
> - This pull request projects a scheduled resource wait as `working`,
with no Retry.
> - The benefit is that the failed-attempt banner shows only for real
failures.
## Linked Issues or Issue Description
**What happened?**
During onboarding on Paperclip Cloud staging (Claude subscription and
Codex subscription paths), the onboarding task showed "Couldn't finish
this attempt. Your message is saved." with a Retry button. The banner
did not name an attempt. Retry cleared it briefly, then it came back.
**Steps to reproduce**
1. Start a task run while another run holds the same workspace.
2. The run is cancelled with `workspace_busy`, and a `scheduled_retry`
run is created with `scheduledRetryReason: "workspace_busy"`.
3. Keep the workspace busy after `scheduledRetryAt`. On the staging
company, one retry stayed scheduled for 16 minutes past its retry time.
4. Open the task. The failed-attempt banner shows. Click Retry. The
banner comes back.
**Expected behavior**
No failed-attempt banner while the system waits for a shared resource.
The run status already says "Waiting for workspace"
(`execution-projection.ts`).
## What Changed
- `server/src/services/task-interruption-presentation.ts`: a
`scheduled_retry` run whose `scheduledRetryReason` is `workspace_busy`,
`ai_connection_busy`, `ai_connection_pool_wait` or
`runner_revision_wait` (and that has no system recovery) projects as
`state: "working"`, `canRetry: false`. User gates and failed Retry
preparations stay authoritative because they are checked first.
- The batched query now selects `scheduledRetryReason`.
- Tests: a Postgres integration test (`task-interruption-query.test.ts`)
that seeds the staging run chain for each reason, and unit tests for the
guard cases (failure-lane retries, paused agent, failed Retry
preparation).
## Verification
Red on `master` (fix not applied), `cd server && npx vitest run
src/__tests__/task-interruption-query.test.ts -t "overdue"`:
```
× does not present an overdue workspace_busy wait as a failed attempt 213ms
× does not present an overdue ai_connection_busy wait as a failed attempt 25ms
× does not present an overdue ai_connection_pool_wait wait as a failed attempt 13ms
× does not present an overdue runner_revision_wait wait as a failed attempt 11ms
- Expected
+ Received
- "canRetry": false,
+ "canRetry": true,
- "state": "working",
+ "state": "failed",
Tests 4 failed | 11 skipped (15)
```
Green with the fix, same tests plus the new unit tests:
```
✓ task-interruption-query.test.ts > does not present an overdue workspace_busy wait as a failed attempt
✓ task-interruption-query.test.ts > does not present an overdue ai_connection_busy wait as a failed attempt
✓ task-interruption-query.test.ts > does not present an overdue ai_connection_pool_wait wait as a failed attempt
✓ task-interruption-query.test.ts > does not present an overdue runner_revision_wait wait as a failed attempt
✓ task-interruption-presentation.test.ts > scheduled resource wait presentation > keeps an overdue workspace wait out of the failed-attempt UI
✓ task-interruption-presentation.test.ts > scheduled resource wait presentation > keeps failure retries, user gates and failed Retry preparation authoritative
```
Neighbouring suites: `task-interruption-presentation`,
`task-interruption-query`, `workspace-recovery`, `agent-project-access`,
`heartbeat-dependency-scheduling`: 5 files, 458 tests passed.
Server `tsc --noEmit` reports no errors in changed files. The only
errors locally come from the unbuilt `@paperclipai/paperclip-runner`
vendor package, which this change does not touch. No UI change, so `pnpm
check:token-gates` is not applicable.
Evidence: the staging onboarding task's run history shows
`workspace_busy` cancellations at 17:46, 17:52 and 18:10, each followed
by a `scheduled_retry`. One had `scheduledRetryAt` 17:53:57 and started
at 18:10:20. The next run succeeded at 18:20.
## Risks
- Low risk. Presentation only; dispatch and admission do not change.
- If a resource wait never clears, the task shows no failed-attempt
banner. The run status still shows "Waiting for workspace", and the
existing retry-attempt limits still end the wait.
- Not in scope: genuine failures without a provider or workspace
explanation still show the generic text, and an overdue failure-lane
retry still shows as failed.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, browser for read-only staging evidence).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Devin FoleyandClaude Opus 5.5
|
33269e1fec |
fix(observability): report directory-merge lock holder diagnostics (#17755)
A directory-merge lock timeout (ERR_WORKSPACE_RESTORE_LOCK_TIMEOUT) already carries allowlisted holder evidence, but server Sentry capture dropped it. - Add directoryMergeLockTimeoutDiagnosticsFor in adapter-utils: closed labels, booleans and bounded durations; no owner PID or paths. - server captureException: lock timeouts get an event-local directory_merge_lock context and error_code=workspace_restore_lock_timeout tag; default grouping and message unchanged. - run-failure-diagnostics restoreLock* fields use the shared reader (same allowlist and order). - Document the record in doc/workspace-restore-locks.md. Diagnostics only; does not change lock acquisition, timeouts or retries. Refs PAPERCLIP-BACKEND-4Q Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
e6c74982fe |
fix(workspaces): report why a workspace provider operation failed (#19362)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A project can store its task workspaces with a workspace storage
provider plugin. Core calls the plugin's `workspaceOperation` method to
prepare, inspect, finalize and remove a workspace.
> - `callWorkspaceProvider` catches every rejection of that call and
throws the same conflict: "Workspace storage operation is incomplete;
retry the same workspace".
> - A provider error, an RPC timeout, a crashed worker and a worker that
never got the call all look the same in the run, the log and the API.
The message tells the user to retry, also when the failure is a fixed
configuration problem that a retry cannot fix.
> - In one diagnosis, three different configuration failures in a row
showed this same message. Finding each cause took trace span counts and
database reads.
> - This pull request lets a provider send a reason from a closed set,
and makes Core tell transport failures apart from provider failures.
> - The benefit is that a failed run says what failed and whether a
retry can help. Provider text and values still never cross into Core.
## Linked Issues or Issue Description
**What happened?**
A workspace provider's `workspaceOperation` failed. The run failed with
"Workspace storage operation is incomplete; retry the same workspace",
`errorCode: setup_failed`, and no reason in the run record or the
`heartbeat execution setup failed` log. Retrying did not help, because
the cause was a provider configuration problem.
**Expected behavior**
Core tells these cases apart: the worker did not receive the call, the
call timed out, the worker stopped during the call, or the provider
reported a failure. For a provider failure, Core shows a bounded reason.
Core tells the user to retry only when a retry can help.
**Steps to reproduce**
1. Install a workspace storage provider plugin whose
`onWorkspaceOperation` throws during `prepare` (for example, because of
missing provider settings).
2. Run an agent task in a project that uses that provider.
3. See the failed run: the error is the generic retry message, and the
run record and logs do not show the cause.
**Paperclip version / commit**
`master` at
|
||
|
|
f2593b7905 |
feat: make Coordinator and Agent Chat standard and speed up navigation (#19360)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People enter that work through the workspace, agent conversations, and tasks. > - Coordinator and Agent Chat still require experimental enrollment. > - Task link prefetches do not warm the compact queries used by the default task view. Chat also waits for a request when its navigation list already holds the conversation. > - This pull request makes Coordinator and Agent Chat standard and reuses the reads needed for navigation. > - People can choose Classic workspace in Experimental settings. Cached work can appear while fresh data loads. ## Linked Issues or Issue Description **What existing behavior does this improve?** Workspace selection, Agent Chat availability, and navigation between Chat, Tasks, and task details. **Subsystem affected** Cross-cutting: UI, workspace preferences, instance settings, and the database migration. **Current behavior** People must enable Coordinator and Agent Chat in Experimental settings. Opening a task can repeat requests because prefetch warms the full detail and comment queries, while the default view reads compact task data and a recent comment window. Opening Chat waits for a detail request even when the current person's conversation list contains it. **Proposed behavior** Coordinator is the standard workspace for everyone. Experimental settings offer Classic workspace, off by default. Old Coordinator experiment values do not select Classic. Agent Chat is available to everyone, including instances with an old saved or managed off value. Task link intent warms the queries used by the mounted view. Chat can use the current person's company-scoped conversation list during refresh. **Reason and benefit** People can use the standard workspace without setup. Repeat navigation can show cached content while the server refreshes it. **Breaking changes** Existing Coordinator experiment and Agent Chat off values no longer disable these surfaces. Classic requires an explicit new opt-in. The workspace API keeps its existing response shape and audit checks. The migration retains legacy values and adds a separate Classic preference. Execution permissions, company access, and chat privacy checks still apply. **Additional context** Roadmap check: this graduates existing surfaces and improves their navigation. It does not add a new agent execution capability. Duplicate searches found no open Coordinator workspace PR. Related server cache work, Refs: #17828, addresses compact issue-list invalidation; this PR addresses client navigation queries. ## What Changed - Add a default-off Classic workspace preference. Derive the existing workspace API response from that choice. Keep personal persistence, account isolation, and audited writes in the workspace preference service. - Replace the Coordinator experiment control with Classic workspace. Use Coordinator while preferences load. Update the existing settings examples and navigation recipe. - Retire the Agent Chat experiment control. Ignore old stored and managed off values. Retain the compatibility key and the existing execution and access checks. - Reuse account- and company-scoped chat list data with its original freshness timestamp. - Share compact task and recent-comment query options between prefetch and the task view. Keep full-history prefetch for Classic tasks and expanded history. - Add upgrade, replay, cache, account isolation, and browser switching regressions. Update product, database, development, and roadmap documentation. ## Verification - Targeted preference, settings, migration, cache, and UI regression tests passed during implementation. The cache tests hold refresh requests pending and check that content still renders. They also check account isolation and stale refresh behavior. - A real browser test verified fresh Coordinator defaults, Classic opt-in persistence, and return to Coordinator. It used an isolated source checkout with fresh data on macOS. - Token, migration numbering and safety, map, and Storybook catalog checks pass after the rebase. - Repository typecheck and build pass. Typecheck and token gates also pass after review repairs. The broad local run passed all six chat integration shards (1,931 tests). It then found the host-CLI-dependent recovery fixture described below. Its remaining duplicate work was stopped after the full current-head CI matrix passed; this is not a claim that the broad local command completed green. - Review repairs passed 419 UI/migration fixture tests, 19 Coordinator navigation tests, and all 214 tests in the five affected server suites. The 24 Chat/workspace browser tests pass, including standard Coordinator switching and explicit Classic persistence. - The broader browser repair run passed 85 tests. It retained two failed assertions and skipped five later serial cases. The affected files then passed all 11 tests after the fixes: reopen the desktop panel after reload and open More for Audit and Connectors. This includes the five serial cases that had not run. - The broad local suite exposed a recovery fixture that executed the host Codex CLI during a simulated remote probe. On a host with Codex 0.154.0, model fallback broke its fixed-model assertion. The fake sandbox now reports Codex 0.160.0. All 20 selected recovery/provider compatibility cases pass after the repair. - A CLI company import/export suite timed out during server startup in the second CI run. The port fallback is intentional in that fixture. No root cause was established. That suite passes on the final commit in CI. This records recovery, not a claimed product fix for the earlier timeout. - The first CI run exposed remaining assumptions about the retired flags and Classic UI. The repair commit updates those fixtures and navigation steps. It also restores the missing enabled Cases destination in Coordinator. Earlier local recovery attempts used the wrong fixture notification and advanced fake time during live execution; the corrected tests use the subscribed recovery queue and wall time during execution. - Greptile reviewed commit `98972b030d4505d838427a2fca6b196b8cad6d47` at **5/5** with no new comments. The original finding is resolved. - Current-head CI is green: **56 successful checks**, with three checks skipped by workflow filters. This includes typecheck, build, the complete Vitest matrix, all eight browser shards, Runner verification, release checks, and security checks. [Final CI run](https://github.com/paperclipai/paperclip/actions/runs/42132651414). - Typecheck caught missing `pid` and `startedAt` fields in the simulated command result. The fixture now returns the complete process contract, and the server test-layer typecheck passes. - Storybook catalog alignment was checked. Storybook play tests were not run. Hosted latency and cold bundle loading were not measured. ## Risks - The new default changes the workspace for people who previously left the Coordinator experiment off. They can opt into Classic in Experimental settings. - The additive migration must run before the new server code. It is safe to replay and preserves explicit Classic choices. - Cached content can be stale during background refresh. Existing query invalidation and source timestamps still control freshness. - These changes reduce avoidable loading waits. They do not establish a measured improvement in hosted request latency or initial bundle download time. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser automation. The exact serving revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
1a36195868 |
fix(storage): name the S3 read when a response body is cut off (#19284)
Record read_object_body with HTTP status, S3 request id and attempts when a GetObject body stream errors after send() resolved, and add the storage fields to run failure execution diagnostics. Diagnostics only; no retry behavior changes. |
||
|
Devin FoleyandPaperclip
|
8080620fc9 |
fix(workspaces): size archive deadlines and preserve resume budgets (#19364)
Give large workspace archives a bounded per-transfer deadline without changing provider settings. Preserve configured post-upload command limits and the saved provider budget when resuming an exact retained sandbox. Verification: 341 focused Core/SDK/adapter tests and 588 Daytona tests passed. Workspace and provider typecheck/build passed. Hosted CI: 54 successful checks, two intended Storybook skips. Greptile 5/5 with no unresolved review threads. The bounded full local run passed all six chat shards and stopped at twenty minutes during general server suites; that full local run remains incomplete. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
a20dfbc695 |
fix(recovery): continue a message sent during Stop settlement after recovery (#19355)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A user can stop or interrupt a running native task and then send a message to give new direction > - When the stopped run left a workspace hold, the dispatch gate withholds the follow-up run until recovery saves the work > - The gate must save the user's message on its receipt so that the deferred-receipt worker continues it after recovery > - Two conditions made the gate skip the message instead of saving it, so the task stayed in progress with "Saved work recovered. Send a message to continue." although the user already sent one > - This pull request saves the message in both conditions > - The benefit is that the message the user already sent continues once, automatically, after recovery ## Linked Issues or Issue Description **What happened?** After a user stopped or interrupted a native task with an isolated sandbox workspace and sent a message, the follow-up run was cancelled with `execution_reconciliation_required`. Its error said "Saved work recovered. Send a message to continue." Nothing re-woke the task. In the interrupt case, a periodic sweep started a run about four minutes later. In the case where the follow-up first waited for the stopped run's workspace save (`workspace_busy`), each later retry parked on the same resolved recovery action, and the task did not continue. **Expected behavior** When the recovery that the follow-up waited on resolves, the message that the user already sent continues once, automatically. The product does not ask for a second message. **Steps to reproduce** 1. Run a native task in a managed sandbox with an isolated workspace. 2. Stop the run (or send a comment with interrupt), then send a message while the stop and workspace save are still settling. 3. Wait for the workspace recovery to resolve. 4. See that the follow-up run is cancelled with `execution_reconciliation_required` and that no run continues the message. **Paperclip version or commit** `master` before this change. **Root cause** `deferUnstartedUserComment` (run dispatch) refused to save the withheld message in two cases: 1. It compared the message time with the source run's `finishedAt`. A native run finishes only after its provider stop and workspace save settle, which takes seconds. A message written after the Stop request but before that settlement was treated as input written before the Stop. The receipt was marked `skipped`. 2. It required the run's own receipt to be user-owned. A workspace-busy retry or an admission retry carries the user's message under a system-owned receipt, so the message was dropped with the retry. ## What Changed - Add `userStopBoundaryAt(run)` in `run-cancellation.ts`. For a run cancelled by a user Stop or interrupt, it returns the recorded Stop request time (`cancellation.recordedAt`) when that is earlier than `finishedAt`. For all other runs it returns `finishedAt`. - Use this boundary in the dispatch gate's message deferral, in native explicit-continuation admission, and in the remote-Stop continuation check, so the three places agree on "message written after the Stop". - When the withheld run is a system-owned retry, walk a bounded chain (max 8) of retries that did not reach the provider (`workspace_busy` with `providerWorkStarted: false`, or a verified pre-dispatch `execution_reconciliation_required`) to the originating user receipt. Save the message on that receipt with the user's original wake context. Mark the retry's system receipt `skipped`. - Update the "input written before the Stop" test so that the Stop request, and not only the settlement, comes after the message. ## Verification - New regressions in `server/src/services/workspace-recovery.test.ts` (`continues a message ... without a second message, after recovery already resolved`): a direct follow-up and a workspace-busy retry. Each runs recovery to completion first, then lets the gate withhold the follow-up, then runs only the deferred-receipt worker. Each asserts that exactly one successor run is queued with the user's message and the user's authority, and that no provider work starts. - Red before the fix: both tests fail because the receipt is `skipped` instead of `deferred_issue_execution`. Green after the fix. - `npx vitest run src/services/workspace-recovery.test.ts` (384 passed). - `npx vitest run src/services/explicit-native-continuation.test.ts src/modules/run-dispatch src/services/execution-continuation.test.ts src/services/run-cancellation.test.ts` (581 passed). - 19 more heartbeat, retry, queued-comment, and native recovery suites (1272 passed). One failure, `heartbeat-process-recovery > redacts opaque environment-bound credentials from Sentry diagnostics: result`, also fails on unchanged `master` and is not related. ## Risks - A message written after the Stop request but before the stopped run settled now continues. The stopping run cannot consume new input after its Stop request, and admission still removes comments that a run already delivered (`undeliveredLegacyUserCommentIds`). All admission gates (holds, stop proofs, approvals, budget) still apply. - Only user Stops and interrupts move the boundary. Other cancellations keep `finishedAt`. - The retry walk is bounded, stays in one company, agent, and issue, and accepts only retries without provider work. It revives a user receipt only from `cancelled` or `skipped`, so a second retry cannot save the same receipt twice. - Not in scope: a run that fails during native preparation with no user message (for example `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate; retirement_refused=unexpected_root_entry` on a system wake). After its recovery resolves, the task still waits for a user message. That is a separate root cause in the native session executor's prior-owner retirement proof. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
7c0922a7b6 |
fix(native): preserve provider backups during slow workspace saves (#19359)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote native sessions save provider state so another web server can continue the same conversation. > - A provider can suspend while its accepted result is still saving a large workspace. > - The run has released its live lease at that point. A new backup-owner lookup rejects this uncommitted state. > - This pull request captures the real owner during backend creation and retains it with the current session authority. > - The existing database checks still reject a changed boot, generation, result, or successor before inspection and during publication. > - The benefit is that a slow workspace save no longer prevents the provider backup. ## Linked Issues or Issue Description **What happened?** A remote session can fail to save its provider backup with `native_harness_storage_owner_missing`. The provider has suspended, but its accepted result is still in `workspace_finalizing`. The close callback tries to acquire a new owner after the run released its lease. A later allocation cannot use a backup that was never published. **Expected behavior** Save the suspended provider with the authority acquired during execution. Recheck that authority before provider inspection and before and after upload. Keep result finalization and provider-work permissions unchanged. **Steps to reproduce** 1. Start a remote native session with durable provider storage. 2. Accept its result and release the live lease while its workspace save remains pending. 3. Close the provider transport. The old owner lookup rejects the save before provider inspection. **Paperclip version or commit** Reproduced on master `bf10bc695abaef6b20e85b287f456b5ce2edb266`. **Deployment mode** A web server with object storage and remote managed Runner sessions. Related: #19225 retains failed-turn ownership through close. #18436 restores an external controller checkpoint before a changed warm process is replaced. This change fixes late backup-owner acquisition while an accepted result saves its workspace. ## What Changed - Capture the provider backup owner when the remote backend still owns its execution lease. - Bind that owner to the current session authority. An older close callback selects the current warm attachment's owner. Revalidate the captured fence before inspecting or skipping provider state. - Add a regression for the close callback during workspace finalization. - Restore encrypted provider files with a captured owner while workspace finalization remains pending. Reject boot, generation, result, and successor changes during upload. - Document the ownership contract. ## Verification - Observed the callback regression fail on the old code with `native_harness_storage_owner_missing`. The same test passes with this fix. - `cd server && node ../node_modules/vitest/vitest.mjs run --config vitest.config.ts src/services/native-runtime/native-harness-backup-storage.integration.test.ts`: 59 passed, using real PostgreSQL and encrypted object write/readback. - `node scripts/check-module-boundaries.mjs`: passed. - Full workspace typecheck: passed with the complete production fix. - Native execution: 1,151 passed. Managed warm lifecycle: 9 passed. These tests require a lost-owner error during shutdown, including before an incomplete provider checkpoint is skipped. The final change to the lifecycle test also requires rejection after lease renewal is lost. - Full local `pnpm build`: passed on the final head. - The full local `pnpm test:run` stopped in chat shard 6 on five existing deadline assertions. Earlier chat shards passed; the PR CI chat shards passed. This is not a full local-suite pass. - The first CI server shard 4 attempt failed the unchanged legacy environment-inheritance fixture at its five-second completion wait. The same failure reproduced locally on the repair head and on an earlier checkout without this repair. One unchanged CI retry passed. No fixture assertion or timeout change is shipped. - Latest-head CI has passed. Greptile reviewed the final head at 5/5 with no open threads. The separate Paperclip Review is paused. This PR does not claim staging acceptance. ## Risks The backup owner now lives with its session authority. A wrong authority selection must fail rather than publish provider state. Database publication still checks the original company, task, run, boot, generation, and accepted result before and after upload. Tests change the owner during upload and verify that the selected backup stays intact. No schema, data migration, or new provider-work permission changes. ## Model Used OpenAI Codex, GPT-6 model family, with reasoning and code execution tools. The host does not expose the exact runtime model ID or context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandClaude Opus 5.5
|
db95539d46 |
fix(agents): refuse agent purge when cost history references it (#19022)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A board user can permanently delete (purge) a terminated agent > - The purge deletes the agent's heartbeat runs, but cost and finance events still point to those runs > - Postgres then rejects the delete with a foreign key error, and the API returns a 500 > - Cost and finance events are accounting history. The purge must not remove them > - This pull request makes the purge check for this history first and return a clear 409 > - The benefit is a clear, expected response instead of a 500, and accounting history stays intact ## Linked Issues or Issue Description **What happened?** Permanent agent deletion failed with `Failed query: delete from "heartbeat_runs" where "heartbeat_runs"."agent_id" = $1`. The cause is `update or delete on table "heartbeat_runs" violates foreign key constraint "cost_events_heartbeat_run_id_heartbeat_runs_id_fk" on table "cost_events"`. The server reported this as an unexpected error. Sentry: PAPERCLIP-BACKEND-4Z. **Steps to reproduce** 1. Run an agent so that a heartbeat run records a cost event. 2. Terminate the agent. 3. Permanently delete (purge) the agent. **Expected behavior** The purge stops before it deletes anything. It returns a clear conflict, and the agent stays terminated. This matches the existing `agent_case_history_retained` behavior. **Actual behavior** The transaction fails on the `heartbeat_runs` delete with a raw foreign key error (500). Related, older approaches: #9130 and #1987 tried to delete cost and finance rows during agent removal. This PR keeps the ledger instead. ## What Changed - `deleteAgentDependencies` now checks for `cost_events` and `finance_events` rows that reference the agent or one of its heartbeat runs. It does this before any destructive cleanup. - If such rows exist, it throws `409 agent_cost_history_retained`. Termination stays available. - The activity-log cleanup reuses the same agent-runs subquery. - New embedded-Postgres regression test `server/src/services/agent-cost-history-deletion.test.ts`. It covers a cost event on a run, a cost event attributed to the agent, a finance event on a run, and a normal purge with no cost history. ## Verification - Without the fix, 3 of the 4 new tests fail with the exact Sentry error. Examples: `cost_events_heartbeat_run_id_heartbeat_runs_id_fk`, `cost_events_agent_id_agents_id_fk`, `finance_events_heartbeat_run_id_heartbeat_runs_id_fk`. - With the fix, all 4 pass. - Related suites pass (8 files, 249 tests): `lifecycle.test.ts`, `execution-admission/adapters/postgres.test.ts`, `decision-models.test.ts`, `prompt-injection-prevention.test.ts`, `fast-responses.test.ts`, `agent-timer-deadlines.test.ts`, `case-deletion.test.ts`, and the new test. - `tsc --noEmit` in `server/` reports 0 errors. ## Risks - Behavior change: agents with recorded cost or finance history can no longer be purged. Before this change, these purges already failed with a 500, so no purge that used to succeed is now blocked. - No schema change and no migration. - Docs follow-up: `doc/SPEC-implementation.md`, `doc/DEVELOPING.md` and `maps/agent-access-map/agents.md` describe `agent_case_history_retained`. They do not yet mention the new `agent_cost_history_retained` code. I can add that in this PR if reviewers want it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
002455ea06 |
fix(issues): finish a reassignment whose native stop settles after the request (#19354)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A user can stop or interrupt a running native task and then send a message to give new direction > - When the stopped run left a workspace hold, the dispatch gate withholds the follow-up run until recovery saves the work > - The gate must save the user's message on its receipt so that the deferred-receipt worker continues it after recovery > - Two conditions made the gate skip the message instead of saving it, so the task stayed in progress with "Saved work recovered. Send a message to continue." although the user already sent one > - This pull request saves the message in both conditions > - The benefit is that the message the user already sent continues once, automatically, after recovery ## Linked Issues or Issue Description **What happened?** After a user stopped or interrupted a native task with an isolated sandbox workspace and sent a message, the follow-up run was cancelled with `execution_reconciliation_required`. Its error said "Saved work recovered. Send a message to continue." Nothing re-woke the task. In the interrupt case, a periodic sweep started a run about four minutes later. In the case where the follow-up first waited for the stopped run's workspace save (`workspace_busy`), each later retry parked on the same resolved recovery action, and the task did not continue. **Expected behavior** When the recovery that the follow-up waited on resolves, the message that the user already sent continues once, automatically. The product does not ask for a second message. **Steps to reproduce** 1. Run a native task in a managed sandbox with an isolated workspace. 2. Stop the run (or send a comment with interrupt), then send a message while the stop and workspace save are still settling. 3. Wait for the workspace recovery to resolve. 4. See that the follow-up run is cancelled with `execution_reconciliation_required` and that no run continues the message. **Paperclip version or commit** `master` before this change. **Root cause** `deferUnstartedUserComment` (run dispatch) refused to save the withheld message in two cases: 1. It compared the message time with the source run's `finishedAt`. A native run finishes only after its provider stop and workspace save settle, which takes seconds. A message written after the Stop request but before that settlement was treated as input written before the Stop. The receipt was marked `skipped`. 2. It required the run's own receipt to be user-owned. A workspace-busy retry or an admission retry carries the user's message under a system-owned receipt, so the message was dropped with the retry. ## What Changed - Add `userStopBoundaryAt(run)` in `run-cancellation.ts`. For a run cancelled by a user Stop or interrupt, it returns the recorded Stop request time (`cancellation.recordedAt`) when that is earlier than `finishedAt`. For all other runs it returns `finishedAt`. - Use this boundary in the dispatch gate's message deferral, in native explicit-continuation admission, and in the remote-Stop continuation check, so the three places agree on "message written after the Stop". - When the withheld run is a system-owned retry, walk a bounded chain (max 8) of retries that did not reach the provider (`workspace_busy` with `providerWorkStarted: false`, or a verified pre-dispatch `execution_reconciliation_required`) to the originating user receipt. Save the message on that receipt with the user's original wake context. Mark the retry's system receipt `skipped`. - Update the "input written before the Stop" test so that the Stop request, and not only the settlement, comes after the message. ## Verification - New regressions in `server/src/services/workspace-recovery.test.ts` (`continues a message ... without a second message, after recovery already resolved`): a direct follow-up and a workspace-busy retry. Each runs recovery to completion first, then lets the gate withhold the follow-up, then runs only the deferred-receipt worker. Each asserts that exactly one successor run is queued with the user's message and the user's authority, and that no provider work starts. - Red before the fix: both tests fail because the receipt is `skipped` instead of `deferred_issue_execution`. Green after the fix. - `npx vitest run src/services/workspace-recovery.test.ts` (384 passed). - `npx vitest run src/services/explicit-native-continuation.test.ts src/modules/run-dispatch src/services/execution-continuation.test.ts src/services/run-cancellation.test.ts` (581 passed). - 19 more heartbeat, retry, queued-comment, and native recovery suites (1272 passed). One failure, `heartbeat-process-recovery > redacts opaque environment-bound credentials from Sentry diagnostics: result`, also fails on unchanged `master` and is not related. ## Risks - A message written after the Stop request but before the stopped run settled now continues. The stopping run cannot consume new input after its Stop request, and admission still removes comments that a run already delivered (`undeliveredLegacyUserCommentIds`). All admission gates (holds, stop proofs, approvals, budget) still apply. - Only user Stops and interrupts move the boundary. Other cancellations keep `finishedAt`. - The retry walk is bounded, stays in one company, agent, and issue, and accepts only retries without provider work. It revives a user receipt only from `cancelled` or `skipped`, so a second retry cannot save the same receipt twice. - Not in scope: a run that fails during native preparation with no user message (for example `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate; retirement_refused=unexpected_root_entry` on a system wake). After its recovery resolves, the task still waits for a user message. That is a separate root cause in the native session executor's prior-owner retirement proof. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
f9d366acdf |
fix(runner): require a managed AI connection for plugin Runner-task environments (#19341)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A plugin can provide Runner execution through an environment driver that declares `supportsTasks: true`. Agent computers use this path. > - Core let an agent without a managed AI connection run on that path. The run then used the account that was installed on the plugin's execution host. > - That is not a supported credential source. When the host Runner did not match, the run failed three times, 30 seconds apart, and then blocked the task with a generic error. The error did not name the fix. > - The plugin cannot refuse these runs itself. `describe` has no connection data, and Core replaces plugin error text before the user sees it. > - This pull request makes a managed AI connection a requirement for every plugin Runner-task environment. Core refuses a run without one before it acquires a lease or calls the plugin. > - The benefit is one clear notice that tells the operator to choose an AI connection, and no wasted retries or plugin work. ## Linked Issues or Issue Description Refs #19326. That pull request makes a Runner revision mismatch retry. It noted that runs without a managed AI connection reach the Runner check on a different path, which still blocks after 3 attempts. This pull request removes that path for plugin Runner-task environments. **Problem or motivation** A `paperclip_runner` agent with no managed AI connection can run in a plugin environment that provides Runner tasks. The run uses the account installed on the execution host. If the host Runner does not match, the native session retries 3 times and then blocks the task with "Plugin-provided Runner execution operation unavailable". The operator cannot tell what to fix. **Proposed solution** Refuse these runs in Core before any lease, plugin call, secret resolution, or provider work. Use one stable reason, `ai_connection_required`, for run admission, the setup probe, and agent verification. Show a task notice that names the fix. **Alternatives considered** A refusal in the plugin. It cannot see the connection at `describe`, which runs first. Its error text is also replaced twice before the user sees it. A new structured plugin error code would still not cover `describe`. ## What Changed - Add `server/src/services/plugin-runner-ai-connection.ts`. It holds the `ai_connection_required` code and message, the probe result, and the manifest lookup for a Runner-task driver. The lookup moves here from `agent-runner-selection.ts`, so run admission and runner selection use the same rule. - `heartbeat/execution-configuration.ts`: refuse a new `paperclip_runner` run in a Runner-task environment when it has no managed AI connection. This also applies when `homeMode: "preconfigured"` drops the connection. The run fails as `configuration_incomplete` with reason `ai_connection_required` and an action URL to the agent's runtime settings. The check runs before observability, secrets, the managed runtime, the workspace, and the lease. A resumed native run keeps its admitted contract. - `recovery/stranded-notice.ts`: add an "AI connection required" task notice. - `run-failure-report.ts`: treat this pre-dispatch refusal as an owner action, not an unexpected failure. - `adapters/registry.ts`: the unmanaged probe for a Runner-task target now returns a failed `ai_connection_required` check. It calls no task operation. Managed probes still go through `testManagedEnvironment`. - `agent-harness-verification.ts` and `agent-lifecycle/application/failure.ts`: classify the check as `ai_connection_required` with a fixed message. Verification waits for the configuration to change instead of retrying. - Docs: `PLUGIN_RUNNER_EXECUTION.md` states the requirement. `PLUGIN_ENVIRONMENT_TASKS.md` no longer mentions installed accounts. `agent-config-ui.md` notes the requirement. - Tests: the "plugin-owned account" heartbeat test described the behavior that this change removes. It now checks the assembled heartbeat refusal: no plugin call, no native boundary, no adapter, no workspace files. ## Verification - `npx vitest run` in `server/` for these files. All tests passed: - `src/__tests__/heartbeat-execution-configuration.test.ts`. New cases: refused without a connection, refused for a preconfigured home, managed passes, resumed native input passes, non-task plugin driver passes. The refused cases also check that observability, secrets, and the managed runtime did not start. - `src/__tests__/heartbeat-plugin-environment.test.ts`. One test, "inherits the instance default environment across companies…", also fails on unmodified `master` in my environment. This change does not touch it, because its driver does not declare `supportsTasks`. - `src/adapters/registry.test.ts`, `src/__tests__/agent-harness-verification.test.ts`, `src/modules/agent-lifecycle/__tests__/diagnostics.test.ts`, `src/services/recovery/stranded-notice.test.ts`, and `src/services/__tests__/run-failure-report.test.ts`. - 17 more suites that use plugin Runner-task environments, with 705 tests: `plugin-runner-task.integration`, `plugin-warm-tasks.integration`, `task-creation-recovery.integration`, `native-startup-cleanup-recovery`, `environment-task-authority.integration`, `plugin-task-credential-writeback`, agent-lifecycle `lifecycle`, `setup`, and `environment-plugins`, `workspace-preparation`, `agent-runner-selection`, `agent-harness-verification-provider-api`, `agent-harness-verification-remote`, `plugin-environment-driver-seam`, `environment-task-runtime`, `agent-test-environment-routes`, and `adapter-registry`. - `pnpm typecheck` in `server/` reported no errors in the changed files. Its only errors came from sandbox-provider packages that my local install did not link. - Not run: `plugin-runner-probe.test.ts`, because it needs a Cargo build. This change does not modify `probePluginRunnerEnvironment`. Also not run: the full `pnpm test:run` and `pnpm build`. CI runs them. ## Risks - Behavior change: an agent in a plugin Runner-task environment with no managed AI connection now stops before dispatch. Before, it ran with the host's installed account. The agent's owner must choose an AI connection. This includes agents with `homeMode: "preconfigured"` in these environments. - Runs that were already admitted and resume from saved native input are not refused. They keep the earlier behavior until they finish. - The check reads the plugin manifest on each new run admission for plugin environments. This is the same lookup that runner selection already does. - This pull request and #19326 both edit `stranded-notice.ts` and `failure.ts`. The changes are additive, but whichever merges second needs a small rebase. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (file edits, shell, test runs). Default context window, extended thinking on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
bf10bc695a |
feat(task): show the agent's context window in task properties (#19273)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run through a harness (Codex app-server, or Claude through ACPX), and the runner turns harness events into Paperclip events > - Both harnesses report how full the model's context window is. Codex sends `modelContextWindow` with its token usage. Claude sends `used` and `size` in ACP `usage_update` > - Paperclip dropped both values. The task view could not show how close an agent is to compaction, and a compaction left no visible mark in the task chat > - This pull request carries the context-window occupancy through the runner, stores it with the existing usage events, and shows it as a "Context" row in the task Properties panel > - It also shows a "Context compacted · 230k → 38k tokens" divider in the task chat when the agent compacts its context > - The benefit is that operators can see when a long task is near its context limit and when the agent lost detail to a compaction ## Linked Issues or Issue Description Refs #19252 (same pattern: a canonical harness event turned into task-chat UI). Refs #19267 (status notices; it turns Claude's compaction banner into a notice, and this PR reads that notice too). **What existing behavior does this improve?** The task view shows no context-window information, and a compaction is invisible in the task chat. **Subsystem affected** Runner usage normalization (Rust and TypeScript transport), the PRP usage schema, a read-only issue API, the task Properties panel and the task-chat transcript. **Current behavior** Codex `modelContextWindow` and ACP `used`/`size` are dropped in the runner. `context.compacted` always has null sizes, Claude compactions emit no `context.compacted` event at all, and the live runner turn hides every marker except "interrupted". **Proposed behavior** `usage.reported` carries an optional `context {usedTokens, windowTokens}`. The task Properties panel shows a Context row ("10% of 258k") with a hover breakdown (tokens used, input and cached input, output, cost, compactions). The transcript shows a compaction divider, with sizes from the usage before and after the compaction. **Reason and benefit** Operators can see when a long task is close to its context limit, and they can see where the agent's earlier context was summarized. **Breaking changes** None. The new schema field is optional, and the new endpoint is read-only. ## What Changed - PRP `usage.schema.json`: add an optional `context {usedTokens, windowTokens}` object. Regenerate the schema bundle and the protocol manifest. - Rust `provider_events.rs`: add `codex_context_usage` (last request `totalTokens` and `modelContextWindow`) and `acpx_context_usage` (ACP `used`/`size`). ACPX `usage_update` now sets `context`. - Rust `provider_backend.rs`: Codex `thread/tokenUsage/updated` sets `context` on live (non-historical) usage. - Rust `provider_events.rs`: emit `context.compacted` for the completed `context_compaction` provider notice that #19267 produces from Claude's compaction banner. Only this path marks a Claude compaction, so the server counts each compaction once. - `runnerd-codex-transport.ts`: `rehydrateRunnerdUsageNotification` keeps a valid `context` on `tokenUsage`, so it reaches the stored `item.completed` usage item. - Server: new `GET /api/issues/:id/context-usage` (issue read access is required) and `issueContextUsage` service. It reads only the assignee's current session (the session of its newest usage report), and returns that session's latest context reading, non-zero totals, cost, and the count of compactions across all of the session's runs. Added to OpenAPI. Shared type `IssueContextUsage`. - DB: migration 0432 adds two partial indexes on `heartbeat_run_events`, built concurrently. One indexes usage items by run and time. The other indexes `context.compacted` events by company and session. `EXPLAIN` shows that both lookups use them. - UI: new `ContextUsageMeter` (bar, percentage, hover card) and `IssueContextUsageRow` in the Execution section of Properties. The row refreshes every 8 s while a run is active, and once more when the run ends. - UI transcript: `native-run-events.ts` maps `context.compacted` to a `context` activity entry with pre/post sizes from neighbouring usage items. `transcript-adapter.ts` turns it into a new `context_compacted` marker variant. The live runner turn keeps that marker as its own timeline row. - Storybook: `task-context-window.stories.tsx` (Properties panel in a task, all states, hover breakdowns, divider). ## Verification - `cargo test -p paperclip-runner-core --lib provider_events` (new tests: ACPX context, Codex context, banner compaction, notice compaction) - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/live/runnerd-codex-transport.test.ts` - `cd ui && npx vitest run src/components/ContextUsageMeter.test.tsx src/components/transcript src/components/task-chat src/components/issue-properties` - `cd server && npx vitest run src/__tests__/issue-context-usage.test.ts` (embedded Postgres) - `node scripts/typecheck-low-memory.mjs`, `pnpm check:token-gates`, `pnpm --filter @paperclipai/paperclip-runner run check:protocol-types` - Live check on a local worktree instance through the runnerd path: - A Codex `gpt-5.5` task showed "10% of 258k" (27k of 258k, input 77k with 49k cached, output 1.1k). - A Claude Sonnet 4.6 task showed "17% of 200k" (34k of 200k). - Storybook: Tasks and conversations / … / Task chat / Context window. - Not verified live: a real compaction. The divider is covered by unit tests through the same projector path the task chat uses. ## Risks - The schema field is optional, and older runners omit it. The UI renders nothing until a harness reports a window. - Migration 0432 builds two partial indexes concurrently on `heartbeat_run_events`, which is a large table. The indexes cover only usage items and compaction events, so they stay small. They add a little write overhead to those events. - Claude in-progress usage rows still record zero tokens with `requests: 1`. The final turn usage is correct. This PR does not change accounting. - A "Compact now" action is out of scope. It needs a compaction-only run that does not produce a run result. It will come in a follow-up. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, in-app browser for Storybook and the live instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
a6ab6a301c |
fix(server): retry a run when its remote Runner is out of date (#19326)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - An agent can run on a remote Runner that an environment plugin
provides
> - A release can update the app before the remote Runner, so for a
short time the app sees an older Runner
> - In that window the plugin can refuse the task, but Core cannot tell
this refusal from other errors
> - Today the run fails as `setup_failed` with no retry, and the task
goes to board recovery. Before #19246, it was also reported as a project
authentication conflict
> - Agent verification also counts the refusal as a failed harness test,
and it parks the agent after three tries
> - This pull request gives the plugin a stable error code for a Runner
mismatch, and Core waits and retries when it gets that code
> - The benefit is that a release no longer blocks tasks or parks
agents, and a real authentication conflict keeps its own clear notice
## Linked Issues or Issue Description
Refs #19246 (it stopped reporting every precheck error as a project
authentication conflict)
Refs #19249 (the same bounded bootstrap retry for a stopped plugin
worker)
Refs #19302 (shares `environment-task-error.ts`; its diagnostic envelope
is unchanged)
Refs #19172 (related open PR: it reports an undescribable plugin Runner
as a retryable check in the same precheck)
Refs #18667 (a real `ai_connection_incompatible` conflict, which now
gets its own notice)
**What happened?**
A run started while its remote Runner was older than the app. The
environment plugin refused the task because the installed Runner did not
match the revision it expects. Core recorded a generic setup failure,
scheduled no retry, and blocked the task. Before #19246, Core reported
this as a project authentication conflict. An agent in `verifying`
counted the same refusal as a failed harness test. After three tries it
stopped verification until a person chose Retry.
**Expected behavior**
Core treats a Runner mismatch as temporary. A run retries without
blocking its task. A `verifying` agent checks again until the Runner
matches. Only a real `ai_connection_incompatible` result blocks the
task, with a notice that names the conflict.
**Steps to reproduce**
1. Use an agent on a plugin environment whose installed remote Runner
revision differs from the revision the plugin expects.
2. Wake the agent on a task, or put a new agent through verification.
3. The run fails with `setup_failed` and the task is blocked.
Verification parks after three tries.
**Paperclip version or commit**
master at
|
||
|
DottaandClaude Opus 5.5
|
4ece2107b8 |
fix(connections): offer aggregator Slack sign-in on Cloud instead of a customer OAuth app (#19328)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Connectors let people give agents tools, such as Slack, from the
Connectors catalog or from an agent's inline "Connect" card
> - On Paperclip Cloud, the only Slack path is "Slack needs its own
OAuth app". The user must create a Slack app, paste a callback URL, and
enter a Client ID and secret
> - Managed Slack chat setup exists, but it is behind the experimental
Chat connectors setting, which is off on Cloud. Composio's and Arcade's
managed Slack sign-in exists, but the native Slack row hides it
> - In an onboarding test on Cloud, a tester got this card from the
onboarding agent, spent 7 minutes on it, and declined it. No Slack
connection was made
> - This pull request follows the existing Google Workspace pattern: on
Cloud, a native connector whose every setup path needs a
customer-created OAuth app yields new setup to the aggregators' managed
sign-in, in the catalog and in agent connection discovery
> - The benefit is that Cloud users can connect Slack with a sign-in
instead of creating a Slack app
## Linked Issues or Issue Description
**What happened?**
On a Paperclip Cloud instance, the onboarding agent asked the user to
connect Slack. Connectors > Slack > Connect showed "Slack needs its own
OAuth app. Slack does not let Paperclip register itself automatically,
so this connector needs an OAuth app you create." The form asks for a
callback URL, a Client ID and a Client secret. It has no link to Slack
app settings. The method description said "Use the provider-hosted
connection for the quickest setup." The tester gave up. The original
notes said that the cloud connector asked for an HTTPS server. That is
not correct: no HTTPS or deployment-mode check is in this path. The real
blocker is the customer OAuth app requirement.
**Expected behavior**
Slack can be connected from the supported Cloud flow without setup steps
that only an operator can do.
**Steps to reproduce**
1. On a Cloud instance with Chat connectors off (the Cloud default),
open Connectors and search for "Slack".
2. Click Connect on Slack. The "needs its own OAuth app" form opens.
3. Or, let an agent call `connections_search("Slack")`. It returns
native `slack` first, and its card opens the same form.
**Additional context**
Evidence: the Slack gallery entry on the Cloud instance had
`ownershipAvailability` `platform_shared: false`, `platform_provisioned:
false`, `customer: true`. `GET
/api/companies/:id/chat-slack/setup-options` returned 403 "Chat
connectors are disabled".
## What Changed
- `packages/shared`: add `nativeConnectorRequiresCustomerOAuthApp(app,
settings)`. It is true when every available setup path of a native
connector needs a customer-created OAuth app: no Paperclip-managed
client, no dynamic client registration, and no enabled chat setup.
- `ui/src/pages/apps/Browse.tsx`: on Cloud, such a connector yields to
aggregators, the same as the built-in Google connectors. The catalog
shows the "Connect through Composio or Arcade" row, and its Connect
button opens the provider choice. A saved native connection, a chat
endpoint, Chat connectors on, a self-hosted instance, or no aggregator
route keeps the native row.
- `server/src/services/connection-intents.ts` and new
`connection-native-yield.ts`: agent `connections_search` uses the same
rule. On Cloud, it does not prefer native Slack for new setup, so the
agent asks the provider-choice question (Composio, Arcade or Zapier).
The rule applies the advertised Paperclip Cloud connector profiles, so
apps with a managed Cloud client (for example Asana) do not yield. If
the profiles cannot be read, the native connector stays.
- `slack.json`: correct the Slack tool method copy, which said "quickest
setup". Add a "register" console link to `https://api.slack.com/apps`,
so the form now shows "Open Slack app settings".
- Tests in all three layers.
## Verification
Red/green. I committed the tests first (`2386fd83df`) and ran them
against unchanged source code.
**Red (before the fix):**
```
server › aggregator connection fallback › Slack on Paperclip Cloud
× offers aggregator Slack routes while native Slack only supports a customer-created OAuth app
AssertionError: expected [ 'slack', 'google-calendar', …(7) ] to deeply equal [ 'via:composio:slack', …(2) ]
+ "slack", "google-calendar", "google-docs", ...
Tests 1 failed | 3 passed | 45 skipped (49)
ui › Connectors landing page
× offers aggregator Slack sign-in on Cloud while the built-in Slack connector needs a customer OAuth app
AssertionError: expected 'SlackSend and read messages in your t…' to contain 'Connect through Composio or Arcade.'
Received: "SlackSend and read messages in your team's channels.Connect"
Tests 1 failed | 5 passed | 92 skipped (98)
shared › Slack tool connection guidance
× describes the customer-created Slack app honestly and links to Slack app settings
AssertionError: expected 'Use the provider-hosted connection fo…' not to match /quickest/i
Tests 1 failed | 3 passed (4)
```
**Green (after the fix):**
```
✓ server › Slack on Paperclip Cloud › offers aggregator Slack routes while native Slack only supports a customer-created OAuth app
✓ server › Slack on Paperclip Cloud › keeps native Slack when Chat connectors provide its own setup path
✓ server › Slack on Paperclip Cloud › keeps native Slack for a company that already saved a Slack connection
✓ server › keeps native Slack on self-hosted instances, where operators register their own Slack app
✓ ui › offers aggregator Slack sign-in on Cloud while the built-in Slack connector needs a customer OAuth app
✓ ui › keeps the built-in Slack connector when Chat connectors offer its setup path
✓ ui › keeps the built-in Slack connector when the instance is self-hosted
✓ ui › keeps a saved built-in Slack connection on its native row on Cloud
✓ shared › describes the customer-created Slack app honestly and links to Slack app settings
✓ shared › flags Slack only while its chat setup is unavailable
✓ shared › does not flag connectors with a managed, dynamic-registration, or credential path
```
Neighbouring suites:
- `packages/shared` (all): 98 files, 1060 tests passed
- `ui` `src/pages/apps` + `src/features/connections`: 54 files, 980
tests passed
- `server` `connection-aggregator-fallback` 49/49,
`connection-intents-service`, `native-connection-baseline-fixture`,
`runner-connection-eval-fixtures` passed. `satisfied-connection-intents`
52/52 passed when run alone. It had a suite-setup error only when three
embedded-Postgres runs went in parallel.
- `pnpm check:token-gates`: clean. `tsc --noEmit` shows no errors in
changed files. Local runs show unrelated errors because the
`paperclip-runner` workspace package is not built. CI runs the full
typecheck.
Still needs a live Slack workspace to prove:
- Complete Slack sign-in through Composio or Arcade on a Cloud instance,
and an agent tool call to Slack.
- An agent run on Cloud that asks the provider-choice question for Slack
and then connects through the selected aggregator.
## Risks
- Behaviour change on Cloud only, for native connectors whose every path
needs a customer-created OAuth app and that an aggregator serves. Today
that is Slack (when Chat connectors are off), and Box and Xero if the
aggregator catalog has them. On Cloud, these move to the aggregator
route for new setup, the same as the built-in Google connectors. Saved
connections keep their native row and discovery.
- Cloud users who want their own Slack app can still open
`/apps/connect?source=slack`, but the catalog row no longer links to it
while the rule applies.
- Self-hosted instances keep the native connector unchanged.
- Follow-up, not in this PR: enable managed Slack chat on Cloud (Chat
connectors Cloud default and `PAPERCLIP_SLACK_MANAGED_SETUP_ENABLED`).
This is a flag and operations change. It needs the Cloud Slack broker to
be proven first. When it is enabled, this rule stops yielding Slack
automatically.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, browser inspection of a staging instance, file
editing).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DottaandPaperclip
|
789a71bc17 |
fix(runner): retry quarantined tasks in their saved workspace (#19340)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native sessions retain execution ownership and action history across turns. > - Task Retry must prove that old execution has stopped before it starts another turn. > - An attempt can inherit its predecessor's identity and fail before it launches a provider. > - The cleanup proof searched under that failed attempt and compared unrelated event sequences. > - This pull request verifies the retained owner and starts one fresh authorized turn. > - Saved results and uncertain action outcomes remain intact. ## Linked Issues or Issue Description **What happened?** Task Retry accepted repeated clicks but kept the canonical request deferred with `executionWait.reason: local_cleanup`. It created no successor, although the retained local worker and provider had stopped. **Expected behavior:** After exact cleanup proof, admit one fresh turn. Preserve the old journal and result. Keep unresolved action outcomes unknown. **Steps to reproduce:** 1. Finish a local native turn with retained cleanup state and a saved result. 2. Let the next attempt inherit its identity and fail with `native_session_cleanup_quarantined` before any launch. 3. Upgrade across a tool-contract change, then click task Retry. 4. The old code requires the failed attempt's stop receipt and journal identity. It also requires the current tool fingerprint and equal sequence numbers across raw and normalized events. Those requirements cannot match the retained owner. **Paperclip version or commit:** Reproduced on `31bf6e4482`. **Deployment mode / installation method:** Authenticated, self-hosted source checkout. **Agent adapter(s) involved:** Native `paperclip_runner`, local Codex app-server. The shared physical-stop proof also covers ACPX. **Database mode:** Embedded PostgreSQL 18.1. **Access context:** Board task Retry. **Node.js version / operating system:** Incident: Node 24.20.0, Linux ARM64. Local verification: macOS ARM64. **Relevant logs or output:** Accepted Retry stays in `deferred_issue_execution`; no new run or provider launch occurs. **Relevant config:** Per-turn local native execution. The old tool contract differs from the current contract. **Additional context:** Related: #19311 prevents the oversized tool result that can lead to retained cleanup. #18217 covers broader cleanup isolation and retained-result maintenance. #19134 admits stopped Runner sources under the current run lease. This patch fixes explicit continuation admission on current master. ## What Changed - Resolve the retained journal's owner under the same company, task, agent, workspace, native session, and runner instance. - Require terminal ownership, settled controllers, released local leases, and exact inherited process birth. Refuse any launch evidence on the failed attempt. - Accept a current host's verified process absence from its exact launch receipt when an older server lacks a stop receipt. A later launch invalidates that observation. - Validate normalized receipts against their own IDs, sequences, and digests. Match ordered provider identities to the independent raw journal. - Keep tool compatibility at the session reuse boundary. Explicit continuation always uses a fresh session. - Bind the stop proof to its exact successor and canonical Retry receipt. Reuse the proven local workspace through normal preparation. Refuse changed bindings, closed or remote workspaces, pending operations, and other live leases. Projectless run-ID placeholders keep the existing fresh-workspace fallback. - Retain journals, results, files, and uncertain action outcomes. Add regression and opt-in live acceptance coverage. - Make the split-handoff browser test wait for both child-completion notices and all run settlement before teardown. Assert the final task set stays unchanged. ## Verification - Red: canonical Retry on the original code accepts one saved intent but creates zero successors and records `local_cleanup`. - Red: distinct raw and normalized event sequences reproduce that wait after owner resolution alone. After admission was repaired, live acceptance then exposed the empty-attempt workspace problem. A PostgreSQL regression also reproduced that wrong workspace selection. - Green at `6684df5e82`: **479 targeted cases**, including 26 new PostgreSQL cases for duplicate clicks, exact process and workspace ownership, controller/lease fences, changed state, and independent receipt integrity. The complete 1,150-test native executor suite passed earlier. - `pnpm -r typecheck` and `pnpm build` pass again at final head `432a52bea8`, including its build stamp. Token gates pass. Master now contains the unrelated directory-fixture type correction found during the first typecheck. - Real authenticated HTTP Retry with Codex `gpt-6.1-sol`, high effort, and per-turn execution passes again at final head `432a52bea8` (93.3 seconds). It preserves the first file line, appends the second once, starts a fresh session in the existing workspace, keeps the retained journal unchanged, commits finalization, and clears task locks. - The strengthened split-handoff browser scenario passes **three consecutive local runs**. Its original CI teardown and one same-head rerun failed when cleanup cancelled an active legacy completion-notification run and hit a PostgreSQL row-lock timeout. The scenario now waits for the complete handoff before cleanup. [Initial failure](https://github.com/paperclipai/paperclip/actions/runs/42106466518/job/127523693634), [same-head rerun](https://github.com/paperclipai/paperclip/actions/runs/42106466518/job/127531698321). These failures remain part of the verification record. - The local full `pnpm test:run` passed all six chat integration shards. Its obsolete pre-rebase attempt was stopped during the general server phase; no complete local-suite pass is claimed. Full CI, including both required gates, passed at `6684df5e82`. At final head `432a52bea8`, all initial CI groups passed except the unchanged status-card deadline test, whose post-wakeup clock assertion missed its bound by 10 ms. That exact case passed three unchanged local repetitions (16.8/27.0/18.3 seconds); the single isolated shard rerun passed all **2,218 tests** (123 files), with no source changes. [Initial timing failure](https://github.com/paperclipai/paperclip/actions/runs/42114473652/job/127550722187). - Greptile reviewed `6684df5e82` at **5/5**, with no actionable findings. A second reviewer identified the supported projectless placeholder case. A new real PostgreSQL regression reproduced its workspace-binding failure before the one-line fallback correction; all **45 focused cases** now pass, including default environment selection and changed binding/cwd refusal. Greptile reviewed final head `432a52bea8` at **5/5**, with no blocking findings. The second reviewer’s thread is resolved. - The patched Linux build verified the actual retained worker/provider ownership and process absence in a transaction that was rolled back. No retained state was retired or provider started during that pre-deployment check. ## Risks - This changes an execution admission proof. Incorrect ownership could permit a second executor. The proof retains process, controller, lease, scope, journal, and receipt fences, and duplicate admission is tested through the real queue. - Missing or conflicting evidence remains deferred. Windows, remote execution, automatic replacement, and session reuse retain their existing policies. - No database migration or UI change. The original affected task still needs deployment of the fix before recovery can be claimed. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, SSH diagnostics, and live provider testing. This session does not expose the exact backend model build or context-window size. The live test agent uses OpenAI `gpt-6.1-sol` at high effort. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
15249a689d |
fix(server): preserve sessions across cosmetic sandbox updates (#19349)
## Thinking Path > - Paperclip manages agent work and saves provider sessions between turns. > - Session and workspace compatibility include the selected environment revision. > - The managed sandbox reconciler also updates names, descriptions, and stock metadata. > - Those cosmetic updates advanced the revision even when the execution configuration stayed the same. > - A later turn therefore discarded an otherwise compatible saved session. > - This pull request keeps that revision stable for cosmetic reconciliation and retains resets for real configuration changes. ## Linked Issues or Issue Description Related: #18943 and #18552. **What happened?** A managed sandbox rename or description update made a later task turn start a fresh provider session. The driver, provider, snapshot, and agent configuration could all remain unchanged. Both session and workspace freshness use the environment's `updatedAt` as its configuration revision. **Expected behavior** Cosmetic managed reconciliation preserves compatible task sessions. Actual sandbox configuration or availability changes retain the normal replacement checks. **Steps to reproduce** 1. Create a managed sandbox and save a task-session fingerprint for it. 2. Reconcile the same sandbox configuration with a new name, description, or managed metadata. 3. Evaluate the saved session against that environment. Before this fix, the decision resets the session and reports the environment category. **Paperclip version or commit** Reproduced on master `d3713d827f70ddba63a323e57c562384dbd648ba`. **Deployment mode** Managed sandbox server. The regression uses an isolated PostgreSQL database and the actual reconciler and session-freshness resolver. ## What Changed - Keep the existing revision for an active managed sandbox when its execution configuration stays equal. - Continue advancing the revision when configuration or availability changes. - Retain activity records and managed-stock binding updates for cosmetic reconciliation. - Add a regression that checks session reuse, audit history, and rejection after a real snapshot change. - Document the freshness contract. ## Verification - The new PostgreSQL regression failed before the production change: it reported `reset: true` and changed category `environment`. - All 53 environment-service cases pass after the fix. - All 190 session-freshness and assembled heartbeat session-preparation cases pass. - Full local typecheck passes. The full local Vitest run remains active, with its local build queued afterward. Initial typecheck setup failures were retained and resolved by repairing generated SDK links; no source workaround was added. - [Full current-head CI](https://github.com/paperclipai/paperclip/actions/runs/42114984839) passes, including the sharded test suites and build: 55 successful checks, 3 skipped, and 1 neutral. - Greptile and Ponytail both reviewed this commit at 5/5 with no findings. No review threads remain unresolved. - Deployed verification remains pending normal delivery. No pinned release or manual rollout is required. ## Risks - Cosmetic managed reconciliation keeps `environments.updatedAt` unchanged, so it does not reorder that environment by this timestamp. Activity records and managed-resource bindings still record the update. - The comparison uses canonical configuration hashes, so JSON property order cannot hide a real change or create false drift. - Existing fingerprint formats and saved session records remain compatible. Real configuration and availability changes still advance the revision. ## Model Used OpenAI GPT-6 through Codex desktop, with reasoning and terminal/browser tools. The host does not expose a more specific deployment suffix or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
da3084a05d |
fix(recovery): do not race a just-accepted interaction's own continuation wake (#19332)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents ask humans for decisions through thread interactions, such as
a plan approval
> - When a human accepts an interaction, the accept route enqueues a
continuation wake for the agent
> - A recovery sweep also re-wakes the agent when an accepted
interaction appears to have no continuation
> - The sweep can read the issue in the short gap between the accept and
the accept-time wake, so it queues a second continuation for the same
approval
> - That redundant wake carries the interaction receipt, so the composer
queue shows an immutable "Accepted: <plan>" item that the user cannot
edit or discard
> - This pull request makes the sweep wait a short grace window after
resolution before it treats an accepted interaction as stranded
> - The benefit is that one approval produces one continuation, and the
queue state agrees with the approval
## Linked Issues or Issue Description
No public issue exists. This bug was found in a hands-on onboarding test
on a staging cloud workspace.
**What happened?**
After the user approved the plan, a queued item stayed in the composer
queue. The user could not edit or discard it. The same item stayed.
Approval did not create a new item each time.
Evidence from the staging workspace (wake diagnostics and run records):
- The plan approval resolved at 17:42:05.692.
- The accept-time wake and its run were created at 17:42:05.992. That
run delivered the approval and succeeded at 17:42:51.
- The recovery sweep (`source: issue.interaction_continuation_recovery`)
requested a second wake for the same approval at 17:42:06.201. Its
`retryOfRunId` fell back to the interaction's source run. This shows
that the sweep saw no run after the resolution, so it read the issue
before the accept-time wake existed.
- The redundant wake was deferred behind later runs and a busy shared
workspace. It ran only at 18:19. During that time, `GET
/issues/:id/queued-comments` returned it as an immutable interaction
entry ("Accepted: Approve storefront prototype plan").
**Expected behavior**
One accepted interaction produces one continuation. A delivered approval
does not stay in the composer queue.
**Steps to reproduce**
1. An agent asks for plan approval on a task.
2. A user accepts the plan.
3. The stranded-issue recovery sweep runs within the moment between the
accept and the accept-time wake enqueue.
4. The sweep queues a second continuation that carries the same
approval. The composer queue shows it as an uneditable item until that
duplicate run finishes.
## What Changed
- `server/src/services/recovery/service.ts`: the accepted-interaction
recovery branch now skips the requeue while the interaction resolved
less than `INTERACTION_CONTINUATION_RECOVERY_GRACE_MS` (2 minutes) ago.
The check applies only when no successful run has delivered the
interaction yet. All other recovery branches for the issue keep their
behavior.
- The skip reports the grace deadline through `onDeferredDeadline`. The
sleep-aware recovery scheduler then rechecks the issue at that deadline,
so a lost accept-time wake still recovers without another write.
- `server/src/__tests__/heartbeat-process-recovery.test.ts`: two
regression tests.
- `does not race a just-accepted plan approval before its accept-time
wake is visible` reproduces the staging race and is the red/green test.
It also asserts that the skip registers the deadline, and that a sweep
after the deadline requeues the approval.
- `does not requeue an accepted plan approval whose accept-time wake is
deferred behind another run` guards the deferred accept-time wake case.
This test already passed on `master`.
- `dispatches the onboarding first task once the user answered the
opening card` now seeds an answer 3 minutes old instead of "now". The
test checks that a lost continuation is requeued. An answer inside the
grace window is not yet lost, so the older seed keeps the test's intent.
Related: the Slack connector cancel bug from the same onboarding test (a
stuck steering item after the user declined the connector) is likely the
same redundant plan-approval wake. At 17:52:10, right after the decline,
the item still queued was a retry of this duplicate wake (it carried the
plan approval, not the Slack interaction). This PR removes the
duplicate, so it most likely covers that symptom too. That bug has its
own investigation.
## Verification
Red, on `master` code with only the new tests added:
```
$ cd server && npx vitest run src/__tests__/heartbeat-process-recovery.test.ts -t "plan approval"
× does not race a just-accepted plan approval before its accept-time wake is visible 1029ms
FAIL src/__tests__/heartbeat-process-recovery.test.ts > heartbeat orphaned process recovery > does not race a just-accepted plan approval before its accept-time wake is visible
AssertionError: expected 1 to be +0 // Object.is equality
Test Files 1 failed (1)
Tests 1 failed | 3 passed | 411 skipped (415)
```
Green, with the fix (the final commit also checks the registered
deadline and the requeue after it):
```
$ cd server && npx vitest run src/__tests__/heartbeat-process-recovery.test.ts -t "plan approval"
Test Files 1 passed (1)
Tests 4 passed | 411 skipped (415)
```
Neighbouring interaction-recovery tests on the final commit (`-t "plan
approval|accepted interaction|answered question|answered interaction"`):
9 passed and 1 failed. The failing test, `schedules bounded retries for
failed accepted interaction continuation wakes`, is a timing flake. It
uses a resolution time in March 2026, so the grace window does not apply
to it. It failed 3 of 3 runs with the original `service.ts` on the same
machine under load, and it passed 1 of 2 runs with the fix.
- `pnpm --filter @paperclipai/server... typecheck`: passes.
- `-t "onboarding first task|plan approval"` on the final commit: 8
passed.
- CI runs the full `heartbeat-process-recovery` suite in the serialized
server shards.
## Risks
- Low risk. If an accept-time wake is really lost, recovery now re-wakes
the agent up to 2 minutes later than before. The sweep already runs on
an interval, so this adds at most one sweep of delay.
- The grace window does not change behavior when a successful run
already delivered the interaction.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, file edits, browser inspection of the staging
workspace API).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DottaandClaude Opus 5.5
|
2bd2e650fe |
fix(connections): let agents outside a grant request scoped access to large catalogs (#19327)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use company app connections, such as GitHub, through
`connections_search` and `connection_request`.
> - A per-user OAuth connection is shared only with the agent that
requested it. Other agents must request their own scoped access.
> - In a hands-on onboarding test, the CEO agent connected GitHub. A
Frontend Engineer hired earlier did not get access. It asked for access
and got a refusal that it could not act on.
> - The engineer then reported "no GitHub access" while the CEO reported
"GitHub is connected". The user saw this as inconsistent and unreliable.
> - This pull request makes the refusal actionable. It also makes an
agent with existing access get `ready`.
> - The benefit is that each agent can request, and then accurately
report, its own access to a connection.
## Linked Issues or Issue Description
**What happened?**
An onboarding test on staging connected GitHub through the CEO agent's
connection request. The grant was delegated and installed only for the
CEO. The Frontend Engineer (hired two minutes earlier) called
`connection_request({ service: "github" })` and then
`connection_request({ service: "github", connectionId })`. Both calls
returned 422 `Specify the tools needed for this task (up to 20)`. The
GitHub catalog has 46 active tools. The refusal gave no tool names.
`connections_search` gave `connectionId: null` for that agent. The agent
told the user that it had no GitHub access. The CEO said GitHub was
connected.
**Expected behavior**
Each agent accurately reports, and can request and use, the connector
access granted to it.
**Steps to reproduce**
1. Connect a per-user GitHub connection (more than 20 indexed tools)
through agent A's connection request.
2. Assign a task that needs GitHub to agent B, which existed before the
grant.
3. Agent B calls `connection_request` with `service: "github"`.
4. The call fails with the bare "up to 20" message, and agent B cannot
find the tool names to request.
**Deployment mode**
Paperclip cloud (staging).
**Agent adapter(s) involved**
Not adapter-specific (core bug).
**Additional context**
Root cause from the staging evidence:
- Connection `GitHub` (oauth, `credentialPolicy: per_user`, 46 active
tools). One user grant. Delegations and installs: CEO at 17:45, Frontend
Engineer at 18:18.
- Frontend Engineer run, 17:53–18:10: git push logged `GitHub
capability_missing`. Two `connection_request` calls returned the 422
above.
- The engineer found the names through `call_api` later and filed a
scoped card at 18:10. The user approved it at 18:18. After that, the
engineer had access.
- The grant scope is intentional: a per-user grant is not shared
automatically with other agents. The defect is the dead-end refusal, not
the scope.
- Repository creation after the grant (a separate onboarding report) is
not in this PR.
## What Changed
- `server/src/services/connection-intents.ts`: when a scoped access
request needs more than 20 tools, the 422 now names the eligible
connection id and lists its indexed tool names (first 100). It tells the
agent to retry with only the needed names. No access is granted.
- `server/src/services/connection-intents.ts`: a request without
`toolNames` from an agent that already has allowed tools on the
connection now reports `ready`. Before, it demanded a grant for every
remaining catalog tool.
- The guessed-tool-name path keeps its existing 20-name bound.
- `server/src/__tests__/connection-intents-service.test.ts`: a
regression test that reproduces the staging setup. The setup is a
per-user GitHub grant shared with the CEO only, 46 tools, and an
engineer outside the grant.
## Verification
Red (test added first, service unchanged):
```
pnpm --dir server exec vitest run src/__tests__/connection-intents-service.test.ts -t "hired before a GitHub grant"
FAIL connectionIntentService > lets an agent hired before a GitHub grant request the access the grant left out
AssertionError: expected 'Specify the tools needed for this tas…' to contain '6c2d6faf-e059-4578-9ac4-8a5fbd81ecfa'
Expected: "6c2d6faf-e059-4578-9ac4-8a5fbd81ecfa"
Received: "Specify the tools needed for this task (up to 20)"
Tests 1 failed | 24 skipped (25)
```
Green (after the fix; the same test):
```
Test Files 1 passed (1)
Tests 1 passed | 24 skipped (25)
```
The test checks these steps:
1. The engineer's search gives `needs_user_action`.
2. The refusal names the connection and the needed tools
(`create_repository`, `push_files`).
3. The scoped retry creates a card.
4. The user approves the card.
5. A plain `connection_request("github")` gives `ready` for that
connection.
During development, an intermediate version of the fix failed step 5 of
the same test. That failure caused the second change.
Neighbouring suites:
- `connection-intents-service.test.ts`: 25 passed.
- `connection-aggregator-fallback.test.ts`: 45 passed. This includes the
bounded guessed-name recovery tests.
- `satisfied-connection-intents.test.ts`: 52 passed. Its `afterAll` hook
times out locally after 30 s. This also occurs on unchanged master code,
so it is not caused by this change.
`tsc --noEmit` for `server` gives no errors in the changed files. The
local workspace had other errors because some workspace packages were
not built. CI runs the full typecheck.
## Risks
- Low risk. The refusal message is longer, and it lists up to 100 tool
names. It lists names only for a connection that the requesting identity
is already eligible to use (same check as before).
- Behaviour shift: an agent with partial access that asks without
`toolNames` gets `ready` and no new access card. To get more tools, the
agent must name them. This matches the tool description ("pass
connectionId and the exact required toolNames").
- No schema, API shape, or permission change. No access is granted
without human approval.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, browser inspection of the staging run history, test
execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Nicky LeachandPaperclip
|
9a79384958 |
Relay bounded Runner source refusal diagnostics (#19302)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An environment plugin can submit a Runner continuation from a retained source. > - The host can refuse that continuation when its source is unverified or has an unresolved operation. > - The worker and Core currently discard that bounded reason. > - Raw provider errors can contain credentials, text or foreign identities. > - This pull request relays only a closed diagnostic and keeps the existing admission and redaction rules. ## Linked Issues or Issue Description Refs #19134 and #19157. This independent fix is based on master. A search found no duplicate relay. **What happened?** A source submit can receive HTTP 409 with `SESSION_STATE_UNAVAILABLE`. The worker drops its structured data. Core then replaces the error with a generic message. The caller cannot distinguish the two known diagnostic reasons safely. **Expected behavior** Carry only the fixed status, code and reason through SDK worker RPC and Core. Keep all provider text and identities out of the relayed error. A diagnostic must not authorize a retry or change state. **Steps to reproduce** Submit an authorized retained source through a task provider. Have the provider throw the recognized state-unavailable error. Read the error after worker RPC and Core dispatch. ## What Changed - Export `PluginEnvironmentTaskErrorDiagnostic`, `environmentTaskErrorData` and `readEnvironmentTaskErrorDiagnostic`. - Accept only `httpStatus: 409`, `errorCode: "SESSION_STATE_UNAVAILABLE"` and reason `"unresolved_operation"` or `"source_state_unverified"`. - Attach the versioned envelope only to an environment task `submit` with `resumeFrom`. Replace its provider message with a fixed message. - Revalidate and reconstruct the closed error in Core. Keep other failures under the existing generic redaction. - Document older SDK behavior and the diagnostic-only boundary. No grant, retry, discard, lifecycle or admission rule changes. ## Verification - Rebased head `7d02bb3e4c` includes master’s UI fixture fix from #19342. The sole conflict was in the worker integration test imports. It now imports both the physical-task result schema from #19300 and the diagnostic reader and RPC error type from this PR. The 39 SDK/worker cases and 57 Core/runtime cases passed, including PostgreSQL worker integration. The SDK build and Runner/server TypeScript checks passed. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/42111889388) passed without retries. All 56 check-rollup entries were green, including Snyk and Socket; [standard Greptile review 2](https://github.com/paperclipai/paperclip/pull/19302#issuecomment-6665823815) scored this exact head **5/5**, with zero unresolved threads. The PR was marked ready on November 5 at 21:37 UTC. - Both new worker relay cases fail against the baseline worker and pass with the fix. - 46 SDK/worker cases passed, including both reasons, malformed/foreign envelopes, extra identities, unsafe getters and non-source operations. - 51 Core/runtime cases passed. The integration uses real SDK serialization and dispatch with PostgreSQL through the existing authorized retained-source submit. It verifies both reasons, malformed RPC rejection, extra-field removal, preserved source outcome and existing dispatch bookkeeping. A valid envelope on an ordinary submit or status stays generic. - Runner and SDK builds and server TypeScript check passed. An initial Runner build lacked its eval-kernel build prerequisite; it passed after that prerequisite was built. - The first expanded DB fixture used a shared environment name and failed its unique constraint. It now uses its unique fixture ID; both complete focused files pass. - Diff and added-line secret scans passed. [GitHub workflow 42083410387](https://github.com/paperclipai/paperclip/actions/runs/42083410387) passed on its second attempt. All CI checks completed, and Snyk passed. Standard Greptile scored exact head `38559ec4df8c` 5/5 with no unresolved threads. That result applies to the previous head. - The first GitHub attempt failed in the unchanged concurrent-budget case in `decision-models.test.ts` (one provider call observed, two expected). That exact case passed locally without code or assertion changes. The workspace shard also passed 4,960 tests before the unchanged CLI company import/export fixture exceeded its 30-second child-server startup wait. No port-collision cause is claimed; that fixture deliberately reserves the requested port. Both initial logs are retained. One failed-job-only retry passed without source changes. ## Risks Older SDKs keep the generic failure. The envelope is diagnostic evidence only. Provider operation and call identities are never accepted. Specific user-visible operation reporting must use Core's retained event or receipt under its existing authority. Master does not contain the new claimed predecessor orchestration. This PR tests the existing retained-source submission boundary. It does not claim the #19157 source recovery or old-journal end-to-end qualification. Runtime pins and deployments remain separate. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning. Used repository tools, TypeScript, SDK RPC and real PostgreSQL tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the bug template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run the focused tests locally and they pass - [x] I have added tests for the new behavior and rejection paths - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved findings - [x] I will address reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
00c7738513 |
fix(task-chat): keep messages sendable after a declined connection request (#19331)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The task chat composer shows a queue of saved input that waits for
the agent's next turn
> - When a person declines a connection request (for example Slack), the
server queues a read-only receipt so the agent learns the outcome
> - The composer showed that receipt with its raw JSON receipt in hover
text, and the receipt hid the person's next message
> - The next message could not be edited, discarded, or sent, so the
chat looked stuck after a simple "Decline"
> - This pull request shows a readable status for receipts and keeps the
person's own later messages visible and actionable
> - The benefit is that declining a connection resolves cleanly and the
conversation continues
## Linked Issues or Issue Description
**What happened?**
During an onboarding test on staging, a person declined the agent's
"Connect Slack" request. A queued steering item then stayed above the
composer. It could not be edited. Its hover text showed raw JSON.
Sending the next message produced an error.
**Steps to reproduce**
1. Give a task to a Paperclip runner agent whose run is active or waits
on a busy workspace.
2. Let the agent request a connection, then decline it.
3. See the queued item "Rejected: Connect Slack". Hover it: the title is
the full JSON receipt.
4. Send another message. The message never appears in the queue. Its row
controls stay disabled. Send or Interrupt on the receipt queue returns
`409 This queue does not support interruption`.
**Expected behavior**
A declined connection request resolves cleanly. It shows a readable
status. The next message can be sent.
**Evidence (staging wake diagnostics for the test task)**
- 17:52:09: the decline wake for the Slack request. It stayed deferred,
was never claimed, and was skipped at 18:21.
- 17:52:10: a later retry wake for the plan-approval continuation.
- `GET /issues/:id/queued-comments` projects the oldest matching wake.
From 17:52 to 18:21 the composer showed the Slack receipt.
Refs: related to the separate plan-approval queued-message fix, which
changes the recovery sweep. This PR only touches the queue projection
and the receipt label.
## What Changed
- `server/src/services/queued-interaction-response.ts`: add
`queuedInteractionSummary()`. A receipt now has one readable line, for
example `Declined: Slack connection`, `Connected: GitHub connection`, or
`Accepted: Build the app`. The entry `source.summary` carries it. The
steering body for the agent still includes the typed JSON receipt.
- `server/src/routes/issues.ts` (`findQueuedCommentWake`): a queued
user-message wake now takes precedence over a receipt wake. A receipt
shows only when it is the only queued input. The receipt wake itself is
unchanged and is still delivered to the agent.
- `server/src/modules/wake-queue/adapters/postgres.ts`
(`findExistingDeferredWake`): skip interaction-receipt wakes when a new
message picks the deferred wake it merges into. Before, the receipt was
the oldest deferred wake, the merge was skipped, and every later message
started its own wake. Only the first message then appeared in the queue.
Receipts still never absorb user input.
- `packages/shared/src/types/issue.ts`: add the optional `summary` field
to `IssueQueuedCommentEntry.source`.
- `ui/src/components/task-chat/TaskChatQueuedMessages.tsx`: add
`queuedMessageLabel()`. The row text, the hover title, and all
aria-labels use the summary. They never use the raw steering body.
## Verification
Red/green tests were written first and run against the unchanged code.
New tests:
- `server/src/__tests__/issue-queued-comments-routes.test.ts`
- `shows a declined connection receipt as a readable status, not its
JSON`
- `keeps the next message visible and actionable after a declined
connection (running run | stopped run)`: declines Slack, posts the next
message, and asserts that the queue lists that message with
`canEdit`/`canDiscard`, that the discard works, and that the receipt
wake is still pending for the agent.
- `queues every later message together after a declined connection
(legacy run | stopped run)`: two messages after a decline both appear in
one editable queue, and the receipt wake keeps no message ids.
- `ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx`
- `shows a declined connection request as readable text, never its raw
JSON receipt (BUG-08)`
**Red (unchanged code):**
```
server: × shows a declined connection receipt as a readable status, not its JSON
AssertionError: expected { kind: 'interaction', …(3) } to match object { …, summary: 'Declined: Slack connection' }
server: × keeps the next message visible and actionable after a declined connection (running run)
server: × keeps the next message visible and actionable after a declined connection (stopped run)
AssertionError: ["Declined: Slack connection"]: expected undefined to match object { canEdit: true, canDiscard: true }
(the queue lists only the receipt; the new message is missing)
Tests 3 failed | 85 skipped (88)
ui: × shows a declined connection request as readable text, never its raw JSON receipt (BUG-08)
AssertionError: expected 'Reorder queued message: Rejected: Con…' not to contain '{'
Tests 1 failed | 23 skipped (24)
```
An exploratory probe on the unchanged code also showed the send error on
this path: `POST /queued-comments/interrupt` on the receipt queue with a
running native run returns `409 {"error":"This queue does not support
interruption"}`. The exact UI text that the tester saw was not
available.
Second commit (review follow-up), unchanged `findExistingDeferredWake`:
```
server: × queues every later message together after a declined connection (legacy run)
AssertionError: expected [ Array(1) ] to deeply equal [ …(2) ]
Tests 1 failed | 1 passed | 88 skipped (90)
```
With the change: `Tests 2 passed | 88 skipped (90)`.
**Green (with this change):**
```
pnpm --filter server exec vitest run src/__tests__/issue-queued-comments-routes.test.ts
Test Files 1 passed (1) Tests 90 passed (90)
server: heartbeat-comment-wake-batching + src/modules/wake-queue
Test Files 11 passed Tests 322 passed (322)
server: retry-scheduling, wake-diagnostics, question-response-delivery, comment-wake-batching, dot-runner, responsible-user-invariant, stale-queue-invalidation, queued-comments routes
Test Files 8 passed Tests 357 passed (357)
pnpm --filter ui exec vitest run src/components/task-chat/TaskChatQueuedMessages.test.tsx src/lib/issue-queued-comment-queue.test.ts
Test Files 2 passed (2) Tests 31 passed (31)
server: question-response-delivery, connection-intents-service, satisfied-connection-intents
Test Files 3 passed Tests 199 passed (199)
```
A first full run of the route file hit a 30s `beforeAll` timeout while
embedded Postgres started on a loaded machine. One new stopped-run case
was also flaky, because the agent could dispatch the new message before
the assertion. I fixed the flake: the test now occupies the agent on
another task (`maxConcurrentRuns: 1`), which is the existing pattern in
this file. I confirmed again that this test fails without the route fix.
Typecheck: `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/server typecheck`, and `pnpm --filter @paperclipai/ui
typecheck` pass. `pnpm check:token-gates`: all gates clean. I did not
run the full `pnpm test:run` or `pnpm build` locally. CI runs them.
## Risks
- Behavior change: when a receipt and a user message wait at the same
time, the composer now shows the message queue. The receipt is no longer
shown there until it is the only queued input. The receipt wake and its
delivery do not change. The resolved card in the thread still shows the
outcome.
- The first line of the steering body for connection receipts changes
from `Rejected: Connect Slack` to `Declined: Slack connection`. The JSON
receipt for the agent does not change.
- Out of scope (unchanged on master, reproduced without any receipt):
while a native run is running, each new message gets its own queued run,
and the queue shows only the first one. Messages that join a
`scheduled_retry` run's context also do not appear in the queue
projection.
- Overlap: the plan-approval queued-message fix (BUG-03) changes the
recovery sweep, not this code. Both fixes touch the same symptom area.
They do not edit the same functions.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, browser inspection of staging, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
b929ad43de |
fix: keep dependency waits neutral and explain GitHub access failures (#19315)
## Thinking Path > - Paperclip manages agent work and shows users what can move it forward. > - Task status and tool approval cards describe different parts of that work. > - A queued dependency wait has no provider attempt to retry. > - GitHub can reject an approved action while other connection tools still work. > - These cases showed a false Retry banner and an unhelpful provider error. > - This pull request keeps dependency waits neutral and gives GitHub access failures a direct repair path. ## Linked Issues or Issue Description Related: #19268 preserves pending Retry preparation and current failure causes. **What happened?** A queued wake cancelled by the dependency gate appeared as a failed attempt with Retry. A GitHub MCP permission denial appeared as raw HTTP error text after an approved repository action. **Expected behavior** Keep the task's dependency state. Show GitHub access failures with a clear explanation and a link to the selected connection's Permissions page. Preserve actual failures, provider details, and approval gates. **Steps to reproduce** 1. Queue a wake, then leave its task dependent on unfinished work before dispatch. 2. Open the task after the dependency gate cancels that wake. 3. Separately approve a GitHub action whose MCP result contains `403 Resource not accessible by integration`. 4. Reopen the saved tool approval. ## What Changed - Recognize both queues' unstarted dependency-wait receipts in cancellation provenance and task projections. No dispatch or recovery gate changes. - Classify GitHub's specific permission error after MCP result normalization. Keep result validation, the raw diagnostic, and working tools available. - Apply the same display code to both historical GitHub error codes. Identify the provider from company-scoped connection metadata, including renamed apps, without rewriting stored decisions. - Show a token-based inline access card with **Manage GitHub access** and collapsed **Error details**. - Give agents trusted recovery steps after an approved GitHub action fails. Keep provider text in the untrusted result and prevent automatic replay. - Add regressions, a production-component Storybook case, and execution/connector documentation. ## Verification - Red before the fix: cancellation provenance, dependency-wait projection, the real queue cancellation, approved MCP failure classification, saved interaction projection, and the error card assertions failed at the expected behavior. - Green focused tests: queue cancellation, one ready-task executor, approval-before-dispatch, one failed GitHub request, connection reuse, retained provider diagnostics, saved interaction reads, task list/detail projection and redaction. - UI: 106 component tests passed. Token gates and Storybook catalog checks passed. - `node scripts/typecheck-low-memory.mjs`: passed. - Local source instance: verified a saved dependency wait stays neutral after reload; a saved GitHub error shows repair guidance, expands its original error, and opens the correct connection Permissions page. PostgreSQL and HTTP APIs were real. The GitHub denial was a controlled MCP fixture; no account or provider permission changed. - `pnpm build`: passed for all workspace packages, the Runner binary, server, CLI and UI. The server build and all server typecheck layers also passed after the review repair. - Additional safety checks passed for realistic repository arguments, active recovery, failed Retry preparation, and bounded provider diagnostics. - Review repair: 45 pure tests and 7 PostgreSQL cases passed. These cover both historical error codes, renamed apps, and company/provider/application identity boundaries. A second local browser check verified the renamed app, retained error detail, and correct Permissions destination. - Final review regressions: the dispatcher and metadata cases failed before the repair. Afterward, 25 focused cases and 59 projection/identity cases passed, including connection-key precedence. The native wait and app-only GitHub source also passed a local browser check. - The historical read adds at most one connection query per interaction read batch. It selects IDs and the provider key only and runs only for candidate permission failures. It adds no provider requests. - Full local `pnpm test:run` stopped at the chat suite's `afterAll` cleanup timeout. All 268 assertions in that group and the three prior chat groups passed. The failure is retained here; it is not counted as a full-suite pass. CI passed on `e8a692e1e2` after one workspace-startup rerun. Fresh CI is pending for the combined head. Its first workspace-a attempt timed out in the CLI company import/export test's child-server startup wait; 4,968 UI tests and 287 CLI tests passed in that job. The same shard passed on the preceding commit. The workspace shard passed on its one rerun. All other test, build and browser jobs passed. The [first failure](https://github.com/paperclipai/paperclip/actions/runs/42093823928/job/127478869941) is retained in the Actions history. Greptile reviewed the preceding head at 5/5. Fresh review is pending for the combined head. ## Risks The GitHub classifier matches the documented permission-denial phrase, not every 403. Rate limits, missing resources and other providers retain their existing error behavior. Historical normalization affects display only. A dependency cancellation needs an unstarted run and a supported queue receipt; current user gates, Retry preparation and recovery remain authoritative. No schema migration or credential changes. ## Model Used OpenAI GPT-6, through the Codex agent. The exact runtime model ID and context-window size are not exposed in this session. Used reasoning, repository tools, TypeScript, PostgreSQL, MCP fixtures and browser verification. The parallel agent-recovery commit records Anthropic Claude Opus 5.5 as co-author. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the bug template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run the focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open findings - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
2506bcdaf4 |
Admit exact stopped Runner sources under the current run lease (#19134)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native execution keeps provider state and a durable controller journal. > - A completed process can stop before the next run attaches. > - The current run must own recovery of that exact source. > - An old controller must lose permission to change the consumed source journal. > - This pull request adds explicit source admission and protected task dispatch. > - The benefit is a narrow recovery boundary that preserves the original successful result. ## Linked Issues or Issue Description Native prerequisites #19132 and #19153 are merged into master. This PR targets master and updates the TypeScript Runner driver contract alongside Core and SDK admission. This change adds the SDK and Core admission boundary. Executor selection and fresh source preparation are a separate change. Related #17916 supplies the workspace export holds preserved here. ## What Changed - Plugins advertise `codexSourceRecoveryVersion: 1` for Codex App Server and `acpxSourceRecoveryVersion: 1` for ACPX. Hosts can qualify either, both, or neither. Core selects the matching field, because a stopped-source check for one provider cannot establish the other provider’s recovery guarantees. ACPX orchestration is tracked separately in #19196. Use the existing exact source identity and `resumeFrom` under the authenticated current run. No public recovery mode is added. Require the claimed source, current account and prepared launch at protected dispatch. - Keep the first source binding pinned on the consumer lease, including after uncertain dispatch. A newly acquired coordinator can refresh only the same source. - Claim one stopped successful source under the current run's acquired native lease. Store the token-free consumer reference in existing lease metadata. Preserve the source status and result. - Check the exact six source identity fields, encrypted journal header, company, task, agent, environment, driver, workspace, account selection, and current coordinator generation. Keep workspace export and legacy settlement holds. - Serialize claims and checkpoint writers in the same coordinator lock order. Reject stale source writes after consumption. Check the current consumer before recovered source writes. - Check source admission before and after protected worker dispatch. Keep the request and lease owned by the current run while the native identity remains the source. - Allow the same owner to prepare B after a fresh source submission, using the exact consumed-source proof instead of a generic warm-reuse flag. Recheck that proof after dispatch. - Preserve ordinary cached adoption and repeated preparation after launch. The current authenticated owner can differ from the adopted PRP identity without requesting a new source launch. Keep ordinary suspended recovery and live task attachment checks. Extract the shared source resolver to avoid a module cycle, and make ordinary selection honor the same durable source consumer. ## Verification - Current head `cac9cf188a` includes merged #19300 and the symmetric capabilities introduced in `b7e25a64b4`. The task-owner and warm-rebind fixtures now identify their physical task. All 29 SDK schema cases, 95 Core ownership and source-admission cases, 25 native Codex and ACPX recovery cases, and the binary metadata check passed on this head. The SDK and Runner builds and server typecheck passed. Both native providers reject the other provider’s capability. GitHub CI is green and [standard Greptile scored this commit 5/5](https://github.com/paperclipai/paperclip/pull/19134#issuecomment-6645574114) with no open threads. - Previous head `bda49271a4ae362a15baf3fe6d6d297e25fd2000` is directly based on master after #19132 and #19153 landed. [GitHub workflow 42072192023](https://github.com/paperclipai/paperclip/actions/runs/42072192023) passed on its first attempt. All check runs completed successfully or as intentional workflow skips; Snyk passed. Standard Greptile scored this exact head 5/5. That review had no unresolved findings. - Runner and SDK TypeScript builds and server typecheck passed. The focused SDK, task-owner, task-runtime, PostgreSQL source and authority suites passed all 161 cases. The production transport-factory getter regression and 14 native-lane selection checks also passed. - The Linux native lane executed the new ordinary warm A→B process case. It kept one Runner/provider process and completed two turns on the same thread. The eager-getter mutation fails with `native_runner_source_attachment_pending`; preserving the live getter passes. The production factory test changes the current owner after transport creation and checks that same options object. - PostgreSQL cases cover both concurrent source-writer/claim orders, exact source submission under the current caller, source-switch denial after uncertain dispatch, same-source coordinator refresh, stale same-boot generation, corrupted journal headers, competing consumers, account changes and export holds. - The protected dispatch sequence submits A, prepares B and retries B preparation. It rejects undispatched sources, changed accounts and foreign target bindings. Source status and accepted result remain unchanged. - Ordinary adopted A preparation under current B, repeated ordinary preparation after launch and ordinary suspended continuation retain their existing path. Absent or false claims preserve ordinary identity denial; malformed claims fail closed. - Earlier development attempts exposed ordinary preparation regressions and the live-getter omission; the final regressions cover those fixes. The first embedded PostgreSQL attempt needed missing library aliases repaired in ignored dependencies. Those attempts are not passing evidence. - Historical, superseded CI heads had unchanged UI fake-timer and Playwright health-response-lifetime failures. Their isolated cases and scoped retries passed. The response-lifetime cause remains unproven. No UI/test timeout was changed. The final rebased-head workflow above passed without a retry. - Whitespace and added-line secret/PII scans passed. Full workspace checks ran on GitHub. No live vendor probe or deployment was performed. ## Risks - Any driver that used the earlier Codex TypeScript option must use `codexSourceRecoveryVersion`. The new plugin description rejects the generic spelling. The Launcher’s existing Codex-specific HTTP header can map to this property when the plugin adopts the new SDK. The change requires no journal migration or ROM change. - This is an inactive admission foundation. Executor orchestration is separate in #19157. Native restore integrity and provider stopped-source/account validation must be present before executor activation. The native prerequisite uses ordinary recovery followed by attachment. The composed executor proofs are in #19157. - The source claim is durable. A later controller must recover the same consumer or keep the hold. It cannot select another source after an ambiguous launch. - Admission adds bounded database checks around protected startup calls. Busy issue locks fail closed; other lock waits have a one-second bound. - No migration, snapshot format, proactive receipt, recurring worker, or deployment is included. Full workspace checks and platform coverage run in CI. These tests do not prove vendor history durability or a hosted rollout. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning, through Codex with repository tools and code execution. The runtime does not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the current head - [x] Standard Greptile is 5/5 with no unresolved findings on the current head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
22281d4b3a |
feat(task-chat): approve harness-proposed plans (Codex plan mode, Claude ExitPlanMode) (#19291)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run through the native runner. Codex and Claude Code each have their own plan mode, and each produces a proposed-plan document. > - Paperclip already has a planning work mode, an issue `plan` document, and a `request_confirmation` approval gate for plans. > - These parts were not connected. The Codex proposed-plan item was transcript text only. The server path never put Codex into plan mode. Claude was never put into plan mode, and its ExitPlanMode plan was dropped. A planning run could close its task as done with no approval. > - This pull request turns a harness's own proposed plan into the issue's Plan document and the existing approval gate, for Codex and Claude. > - The benefit is that users approve or send back the agent's real plan from the task chat. Approve resumes the agent. Request changes sends the reason back to the agent. ## Linked Issues or Issue Description Refs #19252 **Subsystem affected** Native runner provider events (Codex and ACPX Claude), the runner Codex provider (Rust), server native plan synchronization, and the task chat UI. **Problem or motivation** Codex and Claude both produce a proposed-plan document in their native plan mode. Paperclip did not use it. In live tests on the server path, a Codex planning run was not in Codex plan mode, saved its own `implementation-plan` document, and marked the task done without approval. Claude planning runs were not in Claude plan mode, and the ExitPlanMode plan text never reached Paperclip. **Proposed solution** Add a canonical `plan.proposed` event for a harness's proposed plan. Put Codex and Claude into their native plan mode for planning runs. Save a completed proposal as the issue `plan` document with the existing `request_confirmation` gate. Agents can also propose a plan on a standard task; the work mode stays standard. **Alternatives considered** Reuse `plan.updated`. Its schema is the within-turn checklist and is not the durable Plan document, so a separate event keeps the two clear. Stronger prompt text without native plan mode. Live tests showed that the model kept its own document workflow. **Roadmap alignment** This reuses the issue plan document, planning work mode and approval gate that already exist. It adds one canonical provider event. ## What Changed - `packages/paperclip-runner/protocol/schemas/*`: add the `plan.proposed` event (`paperclip.plan.proposed.v1`: `proposalId`, `status`, `markdown` up to 60k characters, `source`). Regenerate the schema bundle, validators and manifest. - `packages/paperclip-runner/src/provider-events.ts`: map the Codex `plan` item and a `<proposed_plan>` block in a completed Codex message to `plan.proposed`. Add `claudePlanProposalEvent`. - `packages/paperclip-runner/runner/crates/runner-core/src/codex_provider.rs`: send Codex's native `collaborationMode: plan` on `turn/start` for a read-only (planning) Codex run. Before this change the server path never sent it. - `runner-core/src/acpx_event_payload.rs`: admit `plan.proposed` as ACPX rich display activity. - `drivers/acpx/codex-runtime-adapter.ts`: set Claude Code to native plan mode for a read-only run. Capture Claude's ExitPlanMode plan from its permission request, record it, and decline the native "Ready to code?" prompt. - `drivers/acpx/codex-acpx-driver.ts`, `cli/acpx-runtime-sidecar.ts`: emit the captured proposal as a rich event. Complete that turn instead of reporting it as cancelled. - `backends/codex-native-backend.ts`: per-harness plan-mode instructions, and a short opt-in hint for Claude on standard tasks. - `server/src/services/native-runtime/native-session-executor.ts`: synchronize a completed `plan.proposed` into the `plan` document and a `request_confirmation` in planning and standard mode. Do not change the issue status during the run (that revoked the run's lease). The run finalizer turns the pending confirmation into the review wait. - `server/src/services/heartbeat-run-summary.ts`: keep a `<proposed_plan>` block out of the posted reply. - UI: new `plan_proposal` provider family. The task chat shows a live "Writing plan" card while a proposal streams and a one-line record when it completes. Resolved plan reviews keep a receipt ("Approved Plan v1", "Requested changes to Plan v1: reason"). Plan previews skip section headings. - Storybook: `Tasks and conversations/.../Task chat/Proposed plan approval` shows the journey. - `doc/SPEC-implementation.md`: describe harness-proposed plan approval. ## Verification - `cargo test -p paperclip-runner-core --lib` (new: `read_only_codex_turns_use_native_plan_collaboration_mode`) - `cd packages/paperclip-runner && npx vitest run src/provider-events.test.ts src/drivers/acpx/codex-runtime-adapter.test.ts` - `cd server && npx vitest run src/__tests__/native-plan-proposal-sync.test.ts src/__tests__/heartbeat-run-summary.test.ts` - `cd ui && npx vitest run src/components/transcript/native-run-events.test.ts src/components/task-chat` - Live test on a local worktree instance with real harnesses: - Codex (`gpt-5.6-sol`), planning task: Codex asked one question, then produced a native plan item. The task showed Plan v1 with Approve plan / Request changes and moved to in review. Approve changed the work mode to standard. Codex then implemented the plan and its tests passed. - Codex, Request changes with a reason: Codex woke in planning mode and produced Plan v2 that includes the change. - Claude (`claude-opus-5-5`), planning task: Claude was in plan mode. ExitPlanMode became Plan v1 and the turn completed. Approve resumed Claude in standard mode, and Claude implemented the plan. - Claude, standard task that asks for a plan first: Claude stopped before code changes and asked for plan approval. The work mode stayed standard. - Storybook: open `Proposed plan approval` and check each stage. ## Risks - Codex planning runs now really use Codex plan mode on the server path. Codex can ask a clarifying question before it proposes. This matches Codex plan mode. - Claude planning runs now use Claude plan mode. A declined ExitPlanMode ends the Claude turn; the runner reports it as completed. Claude's tool row for that request records a rejection from Claude's side. - Codex does not stream the proposed plan on the server path yet, so the live card shows a placeholder until the plan completes. - A `<proposed_plan>` block in any completed Codex message on a standard task now asks for approval. This is the documented opt-in path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, browser). Long context. No extended thinking mode was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
d807887534 |
fix(ui): use one status glyph mapping for every task view (#19329)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task status is the main signal on every task surface: sidebar rows, the task header, the status menu, lists, the inbox, pills and chat cards. > - One task state rendered different status icons in different views. A blocked task that waits on an active sub-task showed the blue "in queue" glyph in the task header. The status menu, the chat chip and the sidebar rows showed the red "Blocked" glyph for the same task. > - The cause: each view passed its own subset of the presentation fields to `StatusIcon`. Many views dropped `blockerAttention` and some dropped `externalConversationState`. The status menu rendered its current row from the bare status. The summaries API did not return `blockerAttention`, so the recent-task sidebar could not know the state. > - This pull request puts the state-to-glyph/label mapping in one resolver and makes every task view pass all of its fields. > - The benefit is that the same state uses the same icon and label everywhere. ## Linked Issues or Issue Description No public issue. Found in a hands-on onboarding test on a staging cloud workspace. **What happened?** The sidebar and the task pane/status menu showed different status icons for the same state. On the test task (blocked, waiting on an active sub-task), the task header and the status button showed the blue "in queue" glyph. The selected "Blocked" row in the status menu showed the red blocked glyph. The coordinator sidebar "Recents" and project rows also showed the red blocked glyph, and the chat "Subtask created" chip showed a red "Blocked" chip. **Expected behavior** The same underlying state uses the same icon and label everywhere. **Steps to reproduce** 1. Create a task with a sub-task. Block the parent on the sub-task, and let the sub-task's agent run (the parent is "blocked, covered"). 2. Open the parent task. The header shows the blue "in queue" glyph. 3. Open the status menu. The current "Blocked" row shows the red glyph. 4. With the coordinator sidebar on, look at the task in Recents or in its expanded project. It shows the red glyph. **Paperclip version or commit** `master` at |
||
|
DottaandClaude Opus 5.5
|
b86f55f0ba |
test(server): remove two more server test races (#19330)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server Vitest shards gate every pull request > - Two more server tests failed on pull requests that do not touch them (#19309, #19265) > - #19314 fixed four other flakes. These two have different causes > - Random CI failures slow every contributor and hide real regressions > - This pull request removes both root causes without retries or skips > - The benefit is fewer false CI failures ## Linked Issues or Issue Description Refs #19309, #19265, #19314 **What happened?** - #19309, job 127459915333: `workspace-recovery.test.ts` > "keeps ancestor Retry held when current proof changes: sibling_hold" received `requestId: null` from a second Retry request. - #19265, job 127481105563: `paperclip-control-plane-port.test.ts` > "retains a durably recorded governed terminal while its postcommit observer is pending: checkpoint timeout" failed with "native session checkpoint timed out after 100ms" before the terminal was recorded. **Root causes** 1. Workspace recovery (test race). `POST /issues/:id/retry` returns 202 and then starts `heartbeat.resumeTaskRetry` without waiting. The proof-change tests change state after the receipt. When the background nudge runs first, it admits the Retry and queues a run before the change. The new queued run is then the latest run, so the second Retry in `sibling_hold` is stale. I proved this locally: a 1.5 s pause after the receipt made every gate in the family fail with a queued run in the issue history. 2. Control-plane checkpoint timeout (budget too tight). `checkpointTimeoutMs: 100` applies to every checkpoint in the session, including the normal checkpoints before the terminal. On a loaded runner, one of those took more than 100 ms. The test expects only the checkpoint that the pending observer blocks to time out. ## What Changed - `server/src/services/workspace-recovery.test.ts`: add `receiptOnlyRecoveryApp`. It builds the same route app, but the route nudge does nothing. The two proof-change families ("keeps ancestor Retry held…" and "holds a failed-preparation HTTP Retry…") use it and call `resumeTaskRetry` explicitly, as before. - `server/src/services/native-runtime/paperclip-control-plane-port.test.ts`: the checkpoint budget is 1 s. The timeout case now waits for the run to fail with the timeout while the observer is still blocked, instead of a fixed 700 ms sleep. This also proves that the timeout fires while the observer is pending. ## Verification - With the forced 1.5 s pause and the fix applied: both workspace-recovery families pass (80 passed). - `npx vitest run src/services/workspace-recovery.test.ts -t "keeps ancestor Retry held when current proof changes|holds a failed-preparation HTTP Retry when current proof changes"`: 80 passed. - `npx vitest run src/services/native-runtime/paperclip-control-plane-port.test.ts`: 94 passed, run 3 times. ## Risks - Low risk. The changes are only in tests. The route nudge is a latency hint ("Maintenance redelivers it"). Other tests still cover it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, and GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
ffd9591ab4 |
fix(recovery): preserve cancelled files after a reusable sandbox stops (#19305)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run writes its task files in a remote sandbox. The task
workspace records that run as its writer until the files are saved.
> - When a user stops or reassigns a run, the cancelled-workspace
preservation worker records the unsaved files as file recovery work.
> - A warm run (`keep_running`) that is not successful releases its
reusable sandbox through stop-and-retain. That release can finish before
the worker sees a terminal coordinator.
> - The worker accepted only an active lease or a released ephemeral
lease. It ignored the released reusable lease, so it never recorded the
file recovery.
> - The workspace kept the ended writer's hold. Each successor run
waited, then failed with "The workspace still has an unfinished writer
or save operation" after the one-hour cutoff.
> - This pull request accepts a released reusable lease that has a
confirmed stop-and-retain receipt as a preservation source.
> - The benefit is that the stopped run's files become recovery work,
and the task is not stranded behind a hold that nothing settles.
## Linked Issues or Issue Description
Refs #18820 (preserves files when Stop finds no live controller). Refs
#17677 (schedules cancelled workspace preservation). Refs #19285 (parks
a workspace wait behind an ended writer).
**What happened?**
A board user reassigned a task during a warm native run. Stop cancelled
the run with `issue_reassigned`, and the Stop acknowledgement was
recorded. The executor settled the coordinator to `terminal_failure`
with no owner. About eight seconds later, terminal release stopped and
retained the reusable Daytona sandbox. The lease became `released` with
`leasePolicy: reuse_by_environment`, a `stopped` termination receipt and
a stop-and-retain receipt. The task workspace stayed `dirty` with
`writerRunId` set to the cancelled run. No file recovery was recorded.
The new assignee's runs waited as `workspace_busy` for one hour. Then
each run failed `setup_failed`, and the task became blocked.
The heartbeat run row showed `nativePhase: "observed"`. That value is
stale: the checkpoint persist wrote it during teardown. The coordinator
itself was `terminal_failure`: the run's execution projection was
`recovery_needed` with no matching recovery action.
**Expected behavior**
The preservation worker records the stopped run's prepared files as file
recovery work. The lease keeps its release status and receipts. No
second stop and no new provider turn start.
**Steps to reproduce**
1. Cancel a native run with an acknowledged run-only Stop. Its remote
workspace reference is `prepared`, and the coordinator is
`terminal_failure` with no owner.
2. Release its reusable lease through stop-and-retain (`released`,
`reuse_by_environment`, confirmed stop receipt).
3. Run the cancelled-workspace preservation worker or the startup scan.
4. On master, the run has no `workspaceRestoreRecovery`, and no recovery
action exists.
## What Changed
-
`server/src/services/native-runtime/native-failed-workspace-recovery.ts`:
add one source check, `cancelledWorkspaceSource`, for cancelled
preservation. The worker check and the startup scan both use it. It
keeps the active and ephemeral cases. It adds a released reusable lease
only when the stop-and-retain receipt is confirmed
(`hasConfirmedSandboxStopAndRetain`) and `preparedRemoteWorkspaceSource`
proves the exact remote-workspace binding. This is the same proof the
restore recorder uses. Native-sync runs have no such proof and stay
ineligible. An ineligible source causes no run write or notification.
- The restore recorder (`preserveLegacyWorkspaceRestoreSources`) already
accepts this source. It checks the confirmed receipt and the exact
prepared workspace binding, and it keeps the lease status, policy and
receipts. This pull request does not change it.
- `native-cancelled-workspace-preservation.test.ts`: add the incident
shape as a regression for the worker and the startup scan, and add
safety cases.
- `doc/execution-semantics.md`: document the stopped reusable source and
the writer hold until recovery.
## Verification
From `server`:
```sh
npx vitest run src/services/native-runtime/native-cancelled-workspace-preservation.test.ts
```
- Red before the fix: the two new regression cases failed (`expected {
nativeCancellation: … } to match object { …, workspaceRestoreRecovery: …
}`). 30 other cases passed.
- Green after the fix: 36 passed.
- Review follow-up: the cases `invalid-receipt`, `expired-lease` and
`native-sync` failed on the previous head
|
||
|
DottaandClaude Opus 5.5
|
dd889e6a0b |
feat: show the assignee's provider usage limit in task properties (#19270)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run on the native Runner through the Codex app-server and the Claude ACP harness > - Both harnesses report the provider account's rate-limit windows: Codex sends `account/rateLimits/updated`, and Claude sends its SDK `rate_limit_event` as `usage_update._meta["_claude/rateLimit"]` > - Paperclip dropped both signals, so an operator could not see that an agent was near or past its provider limit > - This pull request normalizes both signals into one account-scoped snapshot, keeps the latest snapshot per agent, and shows it as one "Usage limit" line in the task properties pane > - The benefit is that an operator sees "5h 78% · resets 15:40" (amber above 80%, "Limited until …" when the provider refuses work, "using credits" when a used-up window falls back to credits) next to the assignee, and can click it for every window ## Linked Issues or Issue Description Refs #19246 Refs #19259 **Is your feature request related to a problem?** Codex and Claude report provider rate-limit windows during a run. Paperclip dropped them: the Codex notification fell through the Rust normalizer with no branch, and ACPX dropped Claude's `_meta`. Operators could not see how close an agent was to its provider limit. **Describe the solution you'd like** Store the latest reported windows per agent and show one compact line in the task properties pane, with a popover that lists every window. **Describe alternatives you've considered** A full gauge in the agent list and the run header. A design review chose one property line with a details popover instead. ## What Changed - Runner (Rust, the runnerd path): new `provider_rate_limits.rs`. Codex `account/rateLimits/updated` becomes a `provider.notice.recorded` event (category `provider.rate_limits`, scope `account`). `CodexProviderState` merges sparse updates and emits a notice only when the windows change. ACPX `usage_update` with a sanitized `rateLimit` field also emits the notice. - Runner (sidecar): `acpx-runtime-sidecar.ts` forwards only `status`, `rateLimitType`, `utilization` and `resetsAt` from `_meta["_claude/rateLimit"]`. This needs the ACPX `_meta` passthrough from #19259. Until that merges, Claude emits nothing. - Runner (TypeScript in-process drivers): the same mapping in `codex-session-notifications.ts` and `codex-acpx-driver.ts` through `drivers/provider-rate-limits.ts`. - Shared: `provider-rate-limits.ts` holds the snapshot type, the parser, the merge, and `summarizeProviderRateLimits` (fullest current window, expiry by reset time, limited versus using credits). - Server: `agent-provider-rate-limits.ts` merges each notice into `agent_runtime_state.state_json.providerRateLimits`, scoped by company and agent. A session reset keeps the snapshot. No migration. - UI: `AgentRateLimitPropertyValue` adds a "Usage limit" row under the assignee in `IssueProperties`. Transcripts treat the notice as run-log only. A run status event refreshes the assignee's runtime state. - Storybook: `Tasks and conversations/Task creation and lifecycle/Usage limit property`. - Docs: `doc/run-log-events.md` describes the notice and its storage. ## Verification - `cd packages/paperclip-runner/runner && cargo test -p paperclip-runner-core --lib rate_limit` and `--test acpx_provider_events`: sparse Codex merge, dedupe, reached window, checkpoint round trip, Claude utilization and rejection, notice shape. - `cd packages/paperclip-runner && npx vitest run src/drivers/provider-rate-limits.test.ts src/drivers/codex` - `cd packages/shared && npx vitest run src/provider-rate-limits.test.ts`: summary, expiry, limited, using credits, parse and merge. - `cd server && npx vitest run src/__tests__/agent-provider-rate-limits.test.ts`: create and merge, out-of-order snapshot, company isolation, unrelated notices, session reset keeps the snapshot. - `cd ui && npx vitest run src/components/transcript src/components/issue-properties src/context/LiveUpdatesProvider` - `pnpm -r typecheck` and `pnpm check:token-gates` pass. - Live test on a local worktree instance: a native Codex agent on a ChatGPT subscription login ran a real task. The runner recorded one `provider.rate_limits` notice. The server stored it. The task properties pane showed "7d 100% · using credits" (this account's weekly window was used up and had credits, and the run succeeded). The transcript showed no extra row. - Not verified live: Claude. The local Claude CLI was not signed in, and the Claude path also needs #19259. The Rust and sidecar paths have tests. - Not run locally: the full `pnpm test:run` and `pnpm build`. CI runs them. ## Risks - Low risk. The snapshot is display data. It does not hold or start work. The existing provider-limit wait (#19246) still reads typed run failures. - Codex can send a rate-limit update often. The runner emits a notice only when the windows change, so run-log growth stays small. - With API-key auth, the harnesses report no windows, so no row appears. This is the intended result. - `CodexProviderState` gains two optional serde fields with defaults, so older checkpoints still load. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits, browser) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
31bf6e4482 |
Pin plugin Runner cleanup to its physical task (#19300)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Plugin Runner owners manage a physical task through an environment lease. > - The existing replacement path can assign a new physical task to that same lease. > - A delayed old owner can then stop the replacement or mark it stopped. > - Cleanup must keep the original physical task identity across RPC and database writes. > - This pull request pins that identity and keeps cleanup available to the new owner. ## Linked Issues or Issue Description Refs #19134 and #19157. This fix is based on master and does not enable repeated stopped-source recovery. A search found no duplicate fix. **What happened?** The maintained `createOwnedSandbox` replacement path changes `providerLeaseId` on one lease. An old task owner uses that mutable lease for a later Stop. A stopped-state response already in flight can also update the new task's flags. **Expected behavior** The old owner must address its original task. Its late cleanup writes must not alter the replacement. The replacement's own owner must still stop and complete it. **Steps to reproduce** 1. Create a plugin task owner for a stopped task. 2. Replace that task through the existing describe/startup path on the same lease. 3. Resume an old checkpoint or a held old status response. 4. Observe a Stop for the replacement or an incorrect stopped flag before this fix. ## What Changed - Require each task owner to keep the physical ID returned by its description. Reject a rebind to a different task. - Carry the ID through the existing internal task context after startup ends. Reject a changed physical task before RPC dispatch. - Match the physical task in stopped-state, completion and credential-handoff writes. - Hold the exact lease row through the retained-session database update and local checkpoint save. - Cover the maintained replacement path with real PostgreSQL tests. Preserve positive cleanup for the replacement owner. Document this boundary. ## Verification - Both replacement regressions failed on unmodified master `10a7d039c8`: delayed old-owner Stop targeted the replacement, and a held old status response marked it stopped. - Six focused files passed 160 tests: task authority, task runtime, warm tasks, execution target, credential writeback and task owner. - Four selected executor factory cases passed. They cover task-only providers and the physical task binding after the startup context ends. - Runner TypeScript, plugin SDK and adapter-utils builds passed. Server TypeScript check passed. Diff and added-line secret scans passed. - All 15 setup-probe cases passed with the actual Runner and fake provider binaries. The first local attempt lacked cargo on PATH. A sandboxed native attempt timed out in seven process cases; the unchanged run with loopback permission passed all 15. A separate initial DB integration attempt was denied a loopback listener; its permission-correct run passed. - Review found three missed constructors in the existing SDK/Core lifecycle fixture. They now use the actual describe result. All four cases in that real PostgreSQL and worker RPC file passed. - The first GitHub attempt failed in the unchanged GitHub recovery pagination case in `chat-channels.integration.test.ts`: its zero-request assertion observed a configuration GET. That exact case passed locally without code or assertion changes. The one failed-job-only retry passed; the initial failure log is retained. - [GitHub workflow 42082587576](https://github.com/paperclipai/paperclip/actions/runs/42082587576) passed on its second attempt. All CI checks completed, and Snyk passed. Standard Greptile scored exact head `fb914d98b99e` 5/5 with no unresolved threads. The branch is mergeable and ready for review. ## Risks The task context change is internal. It adds no plugin wire field, schema or Runner mode. A stale owner now gets an error and cannot clean up a replacement. A new owner can still clean up that task. The retained checkpoint write holds one existing lease row during a bounded local file write. Repeated stopped-source incarnation admission, runtime pins and deployments are outside this change. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning. Used repository tools, TypeScript and real PostgreSQL tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the bug template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run the focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved findings - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandClaude Opus 5.5
|
8aa28940bf |
fix(sentry): keep a live-heartbeat native capacity retry out of Sentry (#19320)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server sends unexpected run failures to Sentry, and it keeps known, recovered failures local > - A native Codex "model at capacity" failure gets an automatic retry. That failure must stay out of Sentry while its retry owns recovery (#18295, #19051) > - The check reads the `nativeProviderFailure` diagnostic from the failed run's `resultJson`. On the live heartbeat path, the failed-status write replaces `resultJson` with the adapter's copy, and that copy has no diagnostic > - So every live-path capacity failure with a scheduled retry still goes to Sentry. Only the reconciliation path, which the tests cover, kept the failure local > - This pull request removes the diagnostic check. The scheduled retry run is the proof > - The benefit is that recovered capacity failures no longer create Sentry noise on the normal live path ## Linked Issues or Issue Description Refs #18295 Refs #19051 **What happened?** A native Codex run failed with `native_provider_overloaded`. The committed status decision scheduled the automatic capacity retry, and the retry run existed before the report. The run failure still went to Sentry. In a staging sample, every live-path capacity failure since #19051 was reported (1 of 1). **Expected behavior** When the `native_provider_overloaded` retry run exists for the same task, the failed run does not go to Sentry. The retry reports its own outcome. **Steps to reproduce** 1. Run a native Codex agent through the live heartbeat path (`finalizeNativeRun` without `projectRunStatus`). 2. Let the provider return `serverOverloaded`. The status decision schedules the capacity retry. 3. The heartbeat writes the terminal `failed` status with the adapter's `resultJson`, and `emitTerminalAgentTaskRun` calls `reportRunFailure` with that row. 4. The row has no `nativeProviderFailure`, so `isScheduledNativeCapacityRetry` returns false and the event goes to Sentry. **Relevant logs or output** The failed run row later shows `nativeProviderFailure: codex/server_overloaded`, because a later reconciliation write restores it. The row that the heartbeat write reports does not have the field. The same overwrite also explains the missing `finalizationReasonCode` that #19051 fixed. ## What Changed - `server/src/services/run-failure-report.ts`: `isScheduledNativeCapacityRetry` no longer requires `resultJson.nativeProviderFailure`. The run must still be native and failed, carry `native_provider_overloaded`, have a native task, have no signal, and not be in setup. The same-task successor with `scheduledRetryReason: native_provider_overloaded` is still required. - The successor is enough proof. Only the committed capacity status decision creates that successor, and only `forTerminal` in `native-provider-failure.ts` produces `native_provider_overloaded` together with the typed `codex`/`server_overloaded` diagnostic. - `run-failure-report.test.ts`: a new case keeps the failure local when the row has only the adapter's `resultJson` (no diagnostic, no finalization fields) and the successor exists. The "diagnostic is missing" case now reports only when no retry run exists. The case for a non-capacity diagnostic with a capacity successor is removed, because that state cannot occur. - `doc/observability.md`: the native capacity paragraph now says that the successor is the proof and that the run's own `resultJson` is not required. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/services/__tests__/run-failure-report.test.ts`: 284/284 pass with this change. - With the `master` version of `run-failure-report.ts`, the new case fails: 1 failed, 283 passed. So the test catches the bug. - `native-provider-failure.test.ts`: 57/57 pass. - `native-provider-overload.integration.test.ts` did not load in my sandbox, because `@paperclipai/plugin-sdk` was not built. CI must run it. ## Risks - Low risk. The change only removes one condition from a Sentry filter. - These failures still report: no retry run, exhausted or held retries, other retry lanes, a successor for a different task, legacy runs, signals, timeouts, setup failures, and unreadable retry evidence. - The change does not affect run status, retries, recovery, or the `all` reporting mode. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use (shell, repository reads, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
8464a8c9ae |
fix(native): settle a bootstrap retry that lost its task lock (#19307)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native run that fails before any provider work gets a bounded same-run bootstrap retry. Restart recovery decides whether that retry can run. > - The retry needs the task's issue lock. A user comment or another wake can move that lock to a new run before the retry is due. > - Restart recovery then returned `issue_execution_lock_changed` without a write. The same check ran on every recovery pass and never ended. The coordinator stayed `retryable_failure`. > - The orphaned-lease sweep waits for a terminal coordinator before it preserves the workspace. So the sandbox lease stayed active, and every later run for the task was cancelled as `workspace_busy`. > - This pull request gives a lock-lost bootstrap retry the same bounded admission that a superseded remote retry already has. > - The benefit is that the retry settles, the board gets a reconciliation action, and the sandbox lease is released instead of blocking the task. ## Linked Issues or Issue Description No public issue exists. Bug description: **What happened?** On a hosted instance, a native agent run failed at startup because the provider rejected `session.open` with a rate limit. Paperclip scheduled a same-run bootstrap retry. Seconds later, a new run for the task took its issue lock, most likely woken by a user comment. From then on, every restart recovery pass for the old run returned `blocked` / `issue_execution_lock_changed` and wrote nothing. The run recorded about 1,000 recovery events over 12 hours. Its coordinator stayed `retryable_failure` with `recoveryMode: bootstrap_retry`. The scoped orphaned-lease sweep found a pending workspace export for the lease. It could not preserve the workspace, because `preserveNativeFailedWorkspace` requires `terminal_failure`. So the lease stayed `active`, and 16 later runs for the task were cancelled as `workspace_busy`. Three more runs on the same instance were stuck in the same state. A board Stop on the failed run moved the coordinator to `terminal_failure`, and the sweep then released the lease within about a minute. **Expected behavior** A bootstrap retry that cannot get its task lock back settles within a bounded number of recovery passes. The coordinator reaches `terminal_failure`, the board gets a reconciliation action, and the existing orphaned-lease sweep can preserve the workspace and release the sandbox lease. **Steps to reproduce** 1. Fail a native remote run before provider work, so its coordinator is `retryable_failure` with `recoveryMode: bootstrap_retry`. 2. Move the task's `executionRunId` to a newer run for the same task, or clear it. 3. Run `claimNativeRestartRecoveries` for the failed run. 4. Before this change, it returns `blocked` with no `retryAt`, and the coordinator stays `retryable_failure` on every pass. The new tests in `native-remote-retry-admission.test.ts` reproduce this. **Paperclip version or commit** Observed on a hosted deployment. Reproduced in tests on `master` at `10a7d039c8e`. Related: #19190 (a `session.open` model rejection no longer schedules a doomed bootstrap retry), #19131 (preserve superseded observed workspaces) and #19281 (release the leases of a cancelled legacy run). None of them changes this admission path. ## What Changed - **Restart recovery (`server/src/services/native-runtime/native-restart-recovery.ts`)**: a new condition, `lockLostBootstrapRetry`, matches a failed run whose coordinator is `retryable_failure` with `recoveryMode: bootstrap_retry` and whose issue lock is not on the run. - A newer run for the task: the existing superseded check now covers this case too. It defers with `native_retry_source_superseded` as permanent, so admission is exhausted at once. - No newer run (the lock was released): recovery defers with `issue_execution_lock_changed` through the existing `deferRemoteRetry`. That path persists a deadline, backs off and exhausts after the system recovery budget. - When admission is exhausted, the existing hold path writes `terminal_failure` / `native_restart_recovery_blocked`, keeps the provider evidence and creates the board reconciliation action. This PR adds no new terminal path. - A bootstrap retry that still holds its own lock is unchanged. The exact-checkpoint remote retry path is unchanged. - **Docs (`doc/execution-semantics.md`)**: the bootstrap retry section now states the lock-lost rule. ## Verification - New test `settles a %s bootstrap retry that lost its task lock instead of re-checking forever` in `server/src/services/native-runtime/native-remote-retry-admission.test.ts`, for the `superseded` and `released` cases. The fixture is a genuine pre-provider failure: a `session.open` rejection with no session checkpoint, no runner checkpoint row and no provider evidence. - On `master` without the fix, both cases fail: recovery returns `blocked` without a deadline, and the coordinator stays `retryable_failure`. - With the fix, `superseded` settles on the first pass, and `released` settles after `SYSTEM_RECOVERY_MAX_RETRIES` deferred passes. Each deferred pass persists its `nextAttemptAt`. Both end in `terminal_failure` with one board recovery action, and later passes open no transaction. - `npx vitest run src/services/native-runtime/native-remote-retry-admission.test.ts` (from `server/`): 92 passed. - Related suites, each run on its own with exit code 0: `native-ingress-bootstrap-retry`, `native-startup-retry`, `native-scheduled-resume-lock`, `native-restart-recovery-isolation`, `native-runner-restart-recovery`, `native-runner-restart-preparation`, `native-runner-restart-diskless`, `native-warm-transition-restart`, `native-controller-recovery`, `native-failed-workspace-recovery`, `native-session-resumption`, `native-startup-cleanup-recovery`, `heartbeat-observed-workspace-preservation`, `heartbeat/recovery`, `runtime-selection`, `remote-workspace-migration` and `native-session-executor` (1,149 tests). Two other suites each had one local failure that this change does not cause: - `heartbeat-process-recovery`: "redacts opaque environment-bound credentials from Sentry diagnostics" fails the same way on this machine without the change. - `chat-channels.integration`: "settles a shared-endpoint accepted-response backlog without retrying its own locks" failed once under full-suite load. It passes alone, with and without the change. - `pnpm run typecheck` in `server/`: passed. ## Risks - Low to medium. The change only affects failed bootstrap retries whose lock is not on the run. Before this change, such a retry could never run, so no working path is removed. - A released lock that comes back to the same run during the backoff still resumes normally, because the condition reads the current lock on each pass. - When the retry settles, the task gets a board reconciliation action. Before this change, the task was silently stuck behind `workspace_busy`. - No migration and no API change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, code search, test runs), extended thinking, and a read-only investigation subagent for the root cause. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
74b11014c6 |
fix(server): stop a queued answer from blocking saved-work recovery (#19318)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native run can accept its result before the workspace copy-back finishes. If a server restart interrupts the copy-back, the run keeps an owner receipt for the server that stopped. > - Another server takes over only after the cloud platform proves that the old deployment stopped. After that proof, recovery checks that the task has not moved on, so old files never overwrite newer work. > - That check counted any queued run on the task as newer work. > - The user's answer to the run's question queues the assignee's next wake. That wake cannot claim while the saved run holds the task's execution lock, so it waits for the copy-back. > - Recovery waited for the wake and the wake waited for recovery. The task stayed stuck, and nothing was logged. > - This pull request ignores a later wake for the same assignee that has never started, and logs waits that the worker stops retrying. > - The benefit is that answering a question during an interrupted copy-back no longer strands the task. Every other newer-work check still applies. ## Linked Issues or Issue Description Refs #15824 (added platform-verified takeover and these binding checks) and #19111 (worker concurrency and ordering for the same copy-backs). This pull request keeps the rest of #15824's fence unchanged. **What happened?** On a hosted instance, a server restart interrupted a copy-back after the worker had written its owner receipt. The run had asked the user a question, and the user answered. The answer's wake was queued. Every later server got a valid stop proof for the old deployment and then got `binding_changed` from the shared binding check: the only failing condition was the queued answer wake. The worker treats `binding_changed` as a final wait and returns without logging. The run stayed `running` and the wake stayed `queued` for about 19 hours, across many restarts. **Expected behavior** A wake that is waiting for the saved run does not stop the saved run's recovery. Runs that could change the workspace still prevent takeover, and the publication check still stops a copy-back if such a run appears. **Steps to reproduce** 1. Accept a native result and leave a copy-back owner receipt from a stopped deployment. 2. Queue the assignee's next wake on the same task, for example by answering the run's question. 3. Let the reconciler or the workspace worker revisit the saved result with a verified stop proof. On `master` the takeover returns `binding_changed` and nothing is copied or committed. The new database tests run these steps against PostgreSQL. **Paperclip version or commit** `master` at `03eb49053fe`. **Deployment mode** Cloud-managed instances with platform stop verification. The same binding check also guards operator owner recovery, so self-hosted operator recovery had the same deadlock. ## What Changed - `server/src/services/native-runtime/native-workspace-recovery-binding.ts`: the newer-run check ignores a run when all of these hold: - It is `queued` and has never started (`startedAt` is null). - It belongs to the run's agent, and that agent is the task's assignee. - It was created after the saved run. - Its wake reason is not `source_scoped_recovery_action`. The run claim blocks such a run while the saved run holds the issue's execution lock (`lockIssueExecutionClaim` in `heartbeat/queue.ts`). Runs that have started, other agents' runs, recovery-action wakes and older queued runs still prevent takeover. - `server/src/services/native-runtime/native-workspace-finalization-work.ts`: log `runId` and `reason` when the worker stops retrying a wait (`binding_changed`, `owner_unverified`). It does not log `owner_active`. - `doc/native-workspace-finalization-recovery.md`: describe the waiting-wake rule in the shared binding section. - `server/src/services/native-runtime/native-workspace-queued-wake-recovery.test.ts` (new): the queued-wake tests, with a small fixture of their own. They are in a separate file so `native-workspace-finalizer.test.ts` stays under 1,000 lines. ## Verification - Failing first. These tests in `native-workspace-queued-wake-recovery.test.ts` fail against the original check and pass with this change: - Operator owner recovery with an answer wake queued. - Automatic takeover with the wake still queued. - Automatic takeover where the wake starts during copy-back. On `master` takeover is refused before this point; with the change, publication is stopped. - The real reconciler, from `retryable_failure`, with a stopped-deployment owner and a queued answer wake. It now copies once, commits the saved result and leaves the wake queued with its claim no longer blocked. - The fence is kept: three new cases (a requeued wake that already started, a recovery-action wake, another agent's queued run) still get `binding_changed`. They pass both before and after the change. - Changed test: "does not reopen an old result after a newer task execution exists" now inserts a started run instead of a never-started queued run of the same assignee. The queued case is now the new operator-recovery test above. - `cd server && npx vitest run src/services/native-runtime/native-workspace-queued-wake-recovery.test.ts src/services/native-runtime/native-workspace-finalizer.test.ts`: 2 files, 72 passed. With the original binding file restored, the 4 positive queued-wake tests failed and the 3 fenced cases passed. - Related suites (finalization work, platform proof, export retry, finalizer failure, recovery actions, owner-recovery routes, status reassessment, scoped deadline work): 8 files, 499 passed. - After rebasing onto `03eb49053fe`: finalizer and finalization-work suites, 97 passed. - `cd server && npx tsc --noEmit -p tsconfig.json`: pass. - Not verified on a live instance yet. The diagnosis comes from that instance's run, finalization, workspace-operation and recovery-action rows, plus a direct stop-proof request that returned a valid receipt. ## Risks - A copy-back can now finish while the assignee's wake is queued. The same-process retry path already does this, because it does not run the binding check. The takeover path now matches it for wakes that cannot start yet. - If some path starts such a wake despite the execution lock, `assertHeld` sees a running newer run and stops before publication. The window between that check and the write is the same as for the other binding conditions. - No schema migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use (shell, file edits, test runs, read-only database and log queries). The runtime does not expose the context window or reasoning configuration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
55389554c2 |
fix(runner): paginate workspace discovery before journal persistence (#19311)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents discover reusable workspaces through `list_workspaces`. > - The tool returned every authorized workspace in one result. > - A large inventory exceeded the durable command journal limit. The result stayed pending, and session cleanup blocked the next turn. > - This pull request limits discovery in SQL and returns explicit continuation metadata. > - Agents can inspect every authorized workspace without sending the full inventory in one command. ## Linked Issues or Issue Description Related: #15749 introduced the task workspace tools. No duplicate pagination fix was found. **What happened?** A native workspace discovery call returned a multi-megabyte inventory. Persisting its result failed with `command_payload_too_large`. The preceding answer could still be saved, but cleanup retained the unsettled tool call. A later turn failed before provider work with `native_session_cleanup_quarantined`. **Expected behavior** Workspace discovery must return a manageable page. Agents must be able to request the remaining authorized rows. Existing visibility checks and journal integrity checks must stay effective. **Steps to reproduce** 1. Create 2,000 task-selectable workspaces with ordinary repository paths. 2. Call `list_workspaces` from an admitted native run. 3. On the old code, the real API regression returns 869,781 bytes. The command journal allows about 510 KiB. 4. On the repaired code, follow `nextCursor` to inspect all rows. Pages contain at most 100 rows and fit within 128 KiB. **Paperclip version or commit** Reproduced on source commit `7351a9d74f` before the repair. **Deployment mode** Isolated source checkout with real PostgreSQL and authenticated native agent APIs. Hosted deployment and incident recovery remain separate. ## What Changed - Validate workspace list limits and UUID cursors with a JSON Schema pattern supported by strict validators. Return `workspaces`, `limit`, `returnedCount`, `hasMore`, and `nextCursor`. - Apply company, visibility, lifecycle, and cursor conditions before the SQL limit. Order pages by workspace ID. Fetch one extra row to detect continuation without a count query. - Fit encoded pages within 128 KiB, including JSON escaping and Unicode. If one summary cannot fit, return explicit string previews with `truncatedFields`. The authorized detail API retains its full record. - Keep existing unpaged summary callers on their array response and recent-use order. - Update the native tool descriptor, generated contracts, catalog fixture, and execution documentation. - Add regression coverage for large inventories, complete pagination, stable pages, private and foreign rows, invalid inputs, and legacy summary behavior. Add an opt-in two-turn live Codex acceptance check. ## Verification - Red before repair: `task-workspace-api.test.ts -t "pages a large authorized workspace inventory"` failed because the result was 869,781 bytes instead of below 128 KiB. The transport regression also failed at 2,284,671 bytes. - Green after repair and review changes: 120 tests passed across the workspace API, transport, routes, native authority, coordinator, historical-question, and Codex vertical-slice suites. They cover complete discovery, byte-sized pages, oversized individual summaries, full detail retrieval, private/foreign/lifecycle exclusions, and strict schema compatibility. - Journal safety: two focused durable control-plane tests passed. Oversized result rejection and lossless valid result delivery remain effective. - Tool catalog and reconciliation: 30 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed again on final commit `bc10880da6`. Server typecheck also passed after the review changes. The original affected suites passed on the rebased first commit (31 tests), and the expanded review regression set passed afterward (120 tests). - Real provider acceptance passed again on final commit `bc10880da6`: local macOS arm64, rebuilt native Runner, `gpt-6.1-sol`, high effort, per-turn lifecycle, and 2,000 persisted workspaces. Both turns succeeded. Persisted call inputs and results prove that the second page uses the first result’s cursor, both calls succeed, their IDs do not overlap, and the encoded results stay within 128 KiB. The second turn kept the provider conversation and file. Both task locks were clear after completion, and no recovered execution failure was recorded. Duration: 166.92 seconds. - Full local `pnpm test:run` stopped in the chat shard: 330 tests passed and one Slack registration test failed with `socket hang up`. That exact test passed in isolation without changes, and the final-head CI chat shards passed. A broader local managed-Slack follow-up produced no result after about eight minutes and was interrupted; it is not counted as a pass. The original full-suite failure remains recorded. - The first CI head failed native tool-schema compilation because strict validators did not recognize `format: uuid`. The revised descriptor uses a UUID pattern. The previously failing coordinator, native vertical-slice, authority, and history suites now pass locally. The stronger live cursor-receipt acceptance check passed. Final-head CI completed on `bc10880da6`: 57 checks passed and five optional checks were skipped, with no failures or pending checks. Greptile reviewed that commit at 5/5; both prior findings are resolved. - Dependency cost: paged discovery runs the existing visibility-scoped summary query with an ID cursor and at most 101 returned rows. It adds no count query, filesystem scan, provider call, or background worker. ## Risks - Native `list_workspaces` now returns a page object instead of an array. Individual summaries can be explicit previews when a single record exceeds the byte budget. Its advertised descriptor and generated contracts change together. Agents must follow `nextCursor` until null for a complete inventory. - Each page rechecks current visibility. Concurrent insertions, deletions, or access changes require a fresh scan when a complete point-in-time inventory is needed. - No migration or journal-limit change. An existing cleanup quarantine still requires normal recovery after deployment; this PR does not bypass it. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, shell tools, code editing, and test execution. The exact served model revision and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
a695398052 |
fix(server): preserve inactive rebase bookmarks in workspace snapshots (#19310)
Preserve a validated standalone REBASE_HEAD and its commit without changing the source workspace. Keep active Git operations, unmerged indexes, invalid markers and concurrent changes blocked. Verified 156 affected tests, workspace typecheck and build. All hosted CI checks passed, and Greptile scored the exact head 5/5 with no unresolved threads. The bounded full local suite stopped during its fifth chat shard; this incomplete result is recorded in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
5de779848b |
fix(recovery): save a reassignment-stopped writer's files from its stopped reusable sandbox (#19304)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A remote task workspace stays held by its last writer run until that writer's files are saved > - A reassignment stops the old owner's run. The stopped writer still holds the workspace with unsaved files. > - When the run used a reusable (warm) sandbox, the stop leaves its lease `released` under the reusable policy, with a stop-and-retain receipt > - Cancelled-workspace preservation only admitted `active` or `ephemeral` sources. It never recorded the save for this stopped writer, so no workspace recovery ran, and the hold stayed forever > - The new owner waited, then failed setup after the one-hour cutoff with "The workspace still has an unfinished writer or save operation" > - This pull request admits the stopped reusable source, so Paperclip saves the stopped writer's files and releases the hold. A waiter keeps waiting while that save is in progress. > - The benefit is that a reassigned task continues with the old owner's saved work, and the new owner does not fail setup ## Linked Issues or Issue Description Refs #19285 (parks a waiter behind an ended writer). Refs #19074, #18820. **What happened?** A board user reassigned a task during a native run on a reusable sandbox. The old run was cancelled with `issue_reassigned`. Its sandbox was stopped and its lease was released under `reuse_by_environment`. The workspace still named the stopped run as its dirty writer. No save operation and no workspace recovery action were ever recorded for that run. The new owner's runs were cancelled `workspace_busy` repeatedly. After more than an hour, a new run failed setup: "The workspace still has an unfinished writer or save operation." **Expected behavior** Paperclip saves the stopped writer's files from its stopped sandbox within a bounded time and releases the hold. The new owner waits for that save and then continues. If the save cannot finish, its retries end with a named board recovery action. **Steps to reproduce** 1. Give a native run a prepared remote workspace on a lease with policy `reuse_by_environment`. 2. Stop the run (for example by reassignment), so the lease is stopped and released with a stop-and-retain receipt. 3. Run cancelled-workspace preservation. Before this fix, it records nothing, and the writer hold stays. ## What Changed - `native-failed-workspace-recovery.ts`: Cancelled-workspace preservation (the delivery worker and the startup scan) now admits a `released` reusable lease with a stopped termination receipt and a stop-and-retain receipt. - `preserveNativeFailedWorkspace` retains such a reusable source only with the exact confirmed stop-and-retain receipt, and only while the workspace is still bound to that writer (`preparedRemoteWorkspaceSource`). Otherwise it writes nothing. - `remote-workspace-writer-hold.ts`: For an ended writer, the holder cutoff now uses the latest progress of an in-progress save of that writer's retained files (a workspace recovery that is not complete, exhausted, or recovered). Reservation and the parked `workspace_busy` wait both read this rule. A waiter waits for the save instead of failing setup. - `doc/execution-semantics.md`: Documents the stopped reusable source and the save-progress cutoff. ## Verification - Red before the fix, green after: - `npx vitest run src/services/native-runtime/native-cancelled-workspace-preservation.test.ts` — new cases "preserves a reassignment-stopped writer's files from its stopped reusable sandbox" (worker and startup scan) failed before the fix: no `workspaceRestoreRecovery` was recorded. - `npx vitest run src/__tests__/heartbeat-remote-workspace-busy.test.ts` — new cases "keeps a waiter parked while an ended writer's save is in progress" failed before the fix: the parked retry was released past the cutoff. - Safety cases: - A reusable source without a confirmed stop, a workspace now owned by another writer, and an already saved workspace are not retained, and the run row does not change. (The writer and saved cases fail if the new guard is removed.) - Preservation does not replay the stopped run, does not create a wake, keeps the stop receipt and lease policy unchanged, and keeps the writer hold until the save publishes. - An exhausted or complete save does not extend the cutoff. The existing past-cutoff test (no save in progress) still fails setup and keeps the held files. - Related suites: workspace recovery, remote workspace reservation, native workspace reconstruction, native failed-workspace recovery, legacy workspace restore recovery. - Not verified on staging: this change is not deployed. The staging task needs this fix deployed. Then the preservation worker records the save, workspace recovery saves the files, and the waiting owner continues. ## Risks - More cancelled runs are now admitted for preservation. The retention guard requires the exact stop-and-retain receipt and the workspace bound to this writer, so a reusable sandbox that another run resumed or that no longer holds unsaved work is not retained. - A stopped reusable source stays `reuse_by_environment` while Paperclip saves it. This matches the existing retention path for reusable native sources. - The writer-hold read adds one indexed query (`issue_recovery_actions_source_run_idx`) for an ended writer only. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, code editing, test execution) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
088f85c5fe |
fix: rename the managed environment to Paperclip Sandbox (#19296)
## Thinking Path > - Paperclip manages AI agents and their execution environments. > - Managed installations supply a default sandbox through managed configuration. > - Most selectors show the configured environment name. > - A fallback label, two errors, and examples still use Paperclip Computer. > - This pull request changes that stock copy to Paperclip Sandbox. > - Existing and custom environment names continue to come from the stored row. ## Linked Issues or Issue Description **What existing behavior does this improve?** The managed environment label in agent settings and the errors shown when the managed sandbox is unavailable. **Subsystem affected** The agent configuration UI, runner selection, and environment reconciliation tests. **Current behavior** Fallback copy and current examples call the managed sandbox Paperclip Computer. The actual environment name is already configurable. **Proposed behavior** Use Paperclip Sandbox for stock copy and current examples. Keep configured names unchanged in selectors, including legacy names and custom names. **Reason and benefit** The name identifies this environment as a managed sandbox and removes ambiguity. **Breaking changes** The displayed fallback and error text change. Error codes, execution behavior, and environment identity remain the same. No migration is needed. **Additional context** No duplicate rename issue or pull request was found. This is a copy update, with no new roadmap feature. ## What Changed - Update the managed fallback label and two unavailable-sandbox error messages. - Update current test and Storybook fixtures without changing story IDs or behavior. - Test configured names, fallback copy, and the unavailable-sandbox message. - Add a PostgreSQL regression test for a rename that retains the environment ID, instance default, agent binding, environment variables, and lease. ## Verification - `pnpm -r typecheck` passed. - `pnpm build` passed. - Focused coverage: 956 cases across nine changed suites. The new fixture initially tried to insert an existing singleton; its corrected upsert passed with all 52 environment-service cases. The updated runner suite also passed all 243 cases. - The two UI suites passed all 189 cases, including configured legacy/custom names and the missing-default fallback. - `pnpm check:token-gates` passed. - `node scripts/storybook-catalog.mjs --check` passed. The catalog is structural evidence only. - `node --test scripts/check-maps.test.mjs scripts/storybook-catalog.test.mjs scripts/storybook-feature-index.test.mjs` passed. - Full `pnpm test:run` is in progress. No live provider or browser journey is claimed by these tests. - CI passed on the current head after one server-shard rerun. The first attempt hit a 10 ms deadline assertion in the unchanged status-card timing test; its isolated local run and the complete rerun shard passed. The original failure and rerun are retained in the PR comment. - Reviewed the diff for secrets, personal data, and run artifacts. ## Risks Low risk: runtime edits change three display strings. Managed reconciliation is unchanged. The regression test exercises the existing rename against PostgreSQL. The configured name still controls normal display, so this code change alone does not rename existing database rows. A managed configuration update uses the existing reconciliation path. Roll back the copy by reverting this commit. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through Codex in T3 Code, xhigh reasoning, with repository tools and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, version, and capabilities - [x] I have checked ROADMAP.md; this copy update does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in the PR with the enhancement template fields - [x] No internal or instance-local issue references - [x] The branch name describes the change and contains no internal ticket ID - [x] Focused tests pass locally - [x] Relevant tests were added or updated - [x] Relevant copy and examples were updated; no API or schema documentation changed - [x] Risks are documented - [x] All CI gates passed on the current head, including the server 11/12 rerun and aggregate verify - [ ] Full local test suite completed (still running; six chat shards passed) - [x] Greptile is 5/5 on the current head with no unresolved findings - [x] I will address all reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7351a9d74f |
fix(native): restore retired backups after warm lease handoffs (#19299)
## Thinking Path > - Paperclip manages AI agents and keeps their work across turns. > - A native session can keep the same sandbox through several warm lease handoffs. > - Each handoff keeps the old lease row and records matching receipts on both owners. > - The retired-backup guard required exactly one row for a provider allocation. > - That rule rejected valid history after Paperclip destroyed the sandbox. > - This change uses the existing handoff proof to select the final owner. > - A verified backup can then restore without discarding ownership checks. ## Linked Issues or Issue Description Refs #19208. **What happened?** A follow-up received a new sandbox after its old sandbox was destroyed. The old allocation had a normal warm lease handoff. Both lease rows remained in PostgreSQL. The guard rejected them with `runner_harness_state_mismatch: backup_without_reusable_lease`. Its later same-run retry failed with `native_remote_recovery_lease_mismatch`. **Expected behavior** Restore the verified backup when one final owner has a valid destroyed receipt and every earlier owner has a matching handoff receipt. Keep uncertain ownership blocked. **Steps to reproduce** 1. Complete a native warm turn. 2. Hand the reusable sandbox to another turn. 3. Complete that turn and save its backup. 4. Close the workspace and destroy the sandbox. 5. Send a follow-up. The new allocation must restore the saved session. **Paperclip version or commit** Reproduced in the PostgreSQL regression on the parent of this change. The affected deployed build also contains #19208. ## What Changed - Select the single final lease owner. Validate all predecessor rows with `retiredReusableLeasePredecessors`. - Keep the destroyed receipt, exact backup stamp, and newer-provider-work checks. - Add database regressions for one and three handoffs. Reject missing or changed receipts, an active predecessor, missing history, an unrelated owner, a stopped source, another backup, and later provider work. - Document the valid lease-history shape. ## Verification - Red before the source change: the two valid-handoff cases returned false. The other 16 database cases passed. - Green after the change: all 18 database cases passed. They also passed on current head `d16e098b8c2e6c0dda70984c1df21e3dc792ab34`. - Command: `pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-retired-backup-source.integration.test.ts`. - The database, executor, retry-admission, and lease suites passed: 4 files, 1,308 tests. - `pnpm -r typecheck` and `pnpm build` passed. The build stamp names head `4ee733c3a979436196522f36537ff182e9df5398`. The full local suite stopped in chat shard 3 after a 15-second timeout in `materializes direct external-chat finals once and accepts the next turn without agent bookkeeping writes`. Earlier shards passed 702 cases; that shard passed 303 other cases. The isolated test passes on this branch and unchanged master `10a7d039c8efeac0fd655c24970df5ae3658414b`. All current-head CI chat shards also passed. The earlier full-suite attempt remains failed; its timeout cause is not established. No test or timeout was changed. CI is waiting for its last Runner job. - Current head `d16e098b8c2e6c0dda70984c1df21e3dc792ab34` removes four comment lines from the previously tested head. Its 18 database cases passed. Greptile reviewed this head at 5/5 with no actionable findings: https://github.com/paperclipai/paperclip/pull/19299#issuecomment-6665446098. CI remains pending. - Live inspection found the matching predecessor/successor receipts and the final owner's destroyed receipt. This was read-only evidence from the affected build. - A live provider run at the fixed revision has not been exercised. Local Daytona credentials and an immutable test image are unavailable. No deployment or original-task retry is claimed. - Initial wider tests could not resolve the unbuilt plugin SDK. The SDK build is now available. A sandboxed database launch could not bind loopback; the same regression passed with the disposable database outside that restriction. ## Risks - A wrong owner could allow an old backup to rewind a session. The shared validator must account for every lease row and match both sides of each handoff. - Live, unrelated, incomplete, or conflicting history stays blocked. The existing backup integrity and provider-progress checks remain. - No migration, permission, model, trust, timeout, or retry-policy change is included. - The original task still needs the fixed build and normal recovery. Hosted acceptance remains unverified. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code execution, and tool use. The session does not expose the exact model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
cb0b7525e0 |
feat(task-chat): warn at the top of a run when an MCP server failed to start (#19289)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each agent run shows its activity in the task chat. The Paperclip runner turns harness events into that activity. > - Codex reports `mcpServer/startupStatus/updated` when an MCP server fails to start. The runner dropped this event at the notification allowlist and in the Rust normalizer. > - When an MCP server fails, the agent runs without its tools. The agent then says "I could not reach the server", and the user cannot see why. > - This pull request maps a failed MCP start to a user-facing provider notice and shows it as one warning line at the top of the run. > - The benefit is that the cause of a "cannot reach my tools" answer is visible next to that answer. ## Linked Issues or Issue Description Refs #19252 (the plan card, which set the pattern for promoting a dropped harness signal to a dedicated task-chat surface). **Problem description** Codex sends `mcpServer/startupStatus/updated` (`name`, `status`, `error`, `failureReason`) during `thread/start`. The TypeScript allowlist (`codex-thread-normalization.ts`) and the Rust normalizer (`runner-core/src/provider_events.rs`) both dropped it. A run with a broken MCP server showed no sign of the failure. The agent only said that it could not reach the tools. **Proposed solution** Map `status: "failed"` to `provider.notice.recorded` with `category: "mcp_server_startup_failed"`, `severity: "warning"`, `scope: "session"` and `userActionable: true`. Show these notices as one run-health line under the runner header: "github MCP server failed to start: No such file or directory (os error 2)". **Alternatives considered** An inline notice row in the timeline. It scrolls away and sits among tool rows. A run-level line next to the header is easier to connect to the agent's answer. ## What Changed - Runner (Rust, production Codex path): `normalize_codex_notification` maps a failed `mcpServer/startupStatus/updated` to a `provider.notice.recorded` notice. `starting`, `ready` and `cancelled` stay out of the transcript. The new helper removes the Codex wrapper text ("MCP client for \`x\` failed to start: MCP startup failed:") so that the line shows the real reason. `reauthenticationRequired` becomes "needs re-authentication". - Runner (TypeScript): the same mapping in `canonicalProviderEventsFromCodex`, and the method is added to `isSupportedCodexNotificationMethod`. - Server: the `view=conversation` run-event query keeps `mcp_server_startup_failed` notices. A settled run can then still show the warning. Other notices stay out of that view. - UI: new `TaskChatRunHealthBanner`. `partitionRunHealthItems` takes MCP failures out of the run's items, with one line per server where the latest report wins. `TaskChatRunnerTurn` (live runs) and `TaskChatTurn` (settled, historical and merged turns) show it under the header. `coalesceSettledTurns` merges the failures when it merges turns. - Storybook: `Tasks and conversations/…/Task chat/Run health` shows one failure, several failures, a finished run and a healthy run, all through the production runner turn. ## Verification - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/provider-events.test.ts src/drivers/codex src/protocol`: 508 tests pass. - `cargo test --release -p paperclip-runner-core --lib -- provider_events`: 20 tests pass. - `pnpm --filter @paperclipai/ui exec vitest run src/components/task-chat src/components/transcript src/components/TaskChatThread.test.tsx`: 1340 tests pass, including the new `TaskChatRunHealthBanner.test.tsx` (persisted event to rendered banner, deduplication, settled historical turn, merged turns). - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/task-cold-open.test.ts`: the conversation view keeps the MCP notice and still excludes other notices. - `pnpm check:token-gates` is clean. `node scripts/typecheck-low-memory.mjs` passes. - Live test on a local instance with Codex 0.160: a Codex agent with a preconfigured Codex home whose `config.toml` has `[mcp_servers.github] command = "github-mcp-server-not-installed"`. Codex reported the failure during `thread/start`. The persisted event had the summary "github MCP server failed to start: No such file or directory (os error 2)". The banner showed while the run was live. It also showed on the settled turn after "More run output", above the agent's answer that it could not reach the GitHub MCP server. ## Risks - Low risk. The change adds one notice category. It does not change how existing notices map or render. - A settled run whose transcript is deferred on a cold open (a successful run with a posted agent comment) shows the warning only after "More run output". This is the same as all other run activity under the current compact history. - Claude does not report MCP startup failures over ACP. `claude-agent-acp` reads MCP status only for OAuth. This change is therefore Codex-only. - Provisional task titles from provider session names are not part of this change. Codex app-server does not generate thread names (only its TUI does). Claude generates its title after the turn ends, and ACPX drops the title text. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code with tool use (shell, file edits, browser automation for Storybook and the live test). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandClaude Opus 5.5
|
99c68455e6 |
fix(workspaces): omit an untracked Python virtual environment from workspace snapshots (#19274)
`python -m venv` links its interpreter by absolute path. The portable workspace archive rejects absolute symlinks, so a workspace with an untracked .venv could never be saved and the task blocked with native_workspace_sync_out_retry_exhausted. Treat an untracked directory with a regular PEP 405 pyvenv.cfg at its root like an untracked node_modules: record it as excluded and do not archive it. Tracked environments are still archived; other absolute links are still rejected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
10a7d039c8 |
fix(dispatch): keep a workspace wait parked while an ended writer's save settles (#19285)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A run that finds its task workspace held by another run waits as a `workspace_busy` scheduled retry. #19143 keeps that wait as one parked retry while the holder is live. > - A remote task workspace stays held by its last writer run until the writer's save and sandbox release settle. This hold continues after the writer run ends, for example when a reassignment stops the writer. > - The parked-wait check counted only a `running` holder. For an ended writer whose save had not settled, it released the waiter at each due check. Preparation deferred again, cancelled the run, and created a new retry run. > - After a mid-run reassignment, the new owner got five cancelled `workspace_busy` runs in eleven minutes, and the task did no work. > - This pull request makes the parked wait use the same writer-hold rule as workspace reservation, so the waiter stays parked until the hold releases. > - The benefit is one durable wait, one start when the hold releases, and no churn of cancelled runs. ## Linked Issues or Issue Description Refs #19143 (parks a workspace-busy wait). Refs #19184 (releases a recovered hold for a new assignee after reassignment). **What happened?** A board user reassigned a task in the middle of a run. The old run was cancelled with `issue_reassigned`. The new assignee's runs were each created, started and cancelled with `workspace_busy`: "Shared execution workspace is busy: run … ended, but its workspace save has not settled". This repeated every 1–2 minutes (five cancelled runs in 11 minutes). Every cancellation named the stopped run of the previous assignee as the holder. The new assignee's other tasks were not the holder. **Expected behavior** The reassigned task waits as one parked retry. It starts once when the stopped writer's save and sandbox release settle. The wait stops at the one-hour holder cutoff. **Steps to reproduce** 1. Give a task a remote execution workspace whose writer run has ended, but whose save has not settled (`writerRunId` set, `dirty: true`). 2. Wake the task's assignee. The run defers with `workspace_busy` and schedules a retry. 3. Make the retry due and promote scheduled retries. Before this fix, the retry is promoted and dispatched, defers again, and creates another run. ## What Changed - New `server/src/services/remote-workspace-writer-hold.ts`: `readRemoteWorkspaceWriterHold` holds the rule for who holds a remote workspace (a running writer, or an ended writer whose save or sandbox release has not settled). The code is moved from `reserveRemoteWorkspace` without a behavior change. - `reserveRemoteWorkspace` uses that function. It throws the same errors as before. - `parkWorkspaceBusyRetryInTx` (from #19143): when the recorded holder is not running, it reads the same hold for the waited execution workspace. If the recorded holder is an ended writer that still holds it, the retry stays parked. The cutoff clock starts at the writer's end, the same value that reservation measures. Earlier waiters keep their arrival order. - New `WORKSPACE_BUSY_SETTLING_RECHECK_MS` (60 s). Settling has no single completion point that can wake the waiter, so a wait behind a settling writer re-checks every minute. Each check creates no run. In the promotion transaction it reads the workspace, the writer run, its task, and the writer's leases (one row per sandbox lease of that run). The checks last only while the wait exists, up to the one-hour cutoff. - `doc/execution-semantics.md`: documents the parked wait behind a settling writer. ## Verification - Red before the fix: `cd server && npx vitest run src/__tests__/heartbeat-remote-workspace-busy.test.ts`. The two new tests failed: the due retry was promoted (`expected [ … ] to not include '<retry id>'`). - Green after the fix, with: - `parks one waiter while an ended writer's save settles, then admits it once the hold releases`: three due checks keep one parked run, each re-check is bounded by the writer's end plus one hour, there is one `workspace_busy` cancellation in total and no new retry run. After the hold releases, the next check promotes the same run. - `stops parking behind an ended writer once its unsettled save reaches the holder cutoff`: parked before the cutoff, promoted after it (preparation then fails with the existing unfinished-writer error). - Existing suites pass: `heartbeat-workspace-busy.test.ts` (live-holder parking, arrival order, stop, workspace move), `modules/run-dispatch`, `remote-workspace-reservation.test.ts`, `remote-workspaces.integration.test.ts`, `isolated-remote-workspace-source.test.ts`, `heartbeat/workspace-preparation.test.ts` (476 tests in all). - `tsc --noEmit` for `server` shows no errors in the changed files. - Not done: a live run on staging. The fix is not deployed. ## Risks - Low. Parking applies only to a retry that recorded a `workspace_busy` wait on an execution workspace whose recorded writer is that holder. If the hold settles, moves to another run, or the task changes workspace, the retry is released to normal dispatch as before. - The wait stays bounded by the existing one-hour holder cutoff from the writer's end. After that, preparation fails as before. A user can still stop the parked run. - One writer per workspace is unchanged. Reservation still decides admission with the same rule. - The one-minute re-check runs only while a wait exists. It is not idle polling. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser reads of staging diagnostics). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
b05d3db2b3 |
fix(recovery): continue a pre-turn interrupt after recovery, not after the destroy (#19292)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A board user can interrupt a native run with a comment. The next turn should start soon after the run stops > - When the stop leaves sandbox files unsaved, recovery saves them, re-stops the sandbox, and then queues the destroy for an ephemeral lease > - #19224 kept the verified stop on that lease as `workspaceRecoveryCleanup.stopReceipt`, and taught the continuation gates to accept it > - One gate was missed: `acknowledgedNativeStopExecutionHasStopped`. An interrupt that lands before the provider turn opens leaves no gating hold, so the saved comment goes through this gate, not through the explicit continuation gate > - That gate still wanted the raw `remoteExecutionTermination` receipt, so the comment waited about 5 minutes for the destroy retry > - This pull request lets that gate accept the carried-forward stop receipt too > - The benefit is that an interrupt during startup continues right after recovery, as #19224 intended for every lease policy ## Linked Issues or Issue Description Refs #19224 Refs #19206 **What happened?** On hosted staging, an `interrupt: true` comment reached a native run of an ephemeral-lease agent while the run was still in startup (the provider pack upload). The run stopped before any provider turn. Recovery saved its files at once. The follow-up turn started about 6 minutes after the interrupt. The stopped run's lease was released by `pending_cleanup_retry`, and the follow-up started 2 seconds later. Agents whose interrupt landed after the provider turn opened continued in 30 to 50 seconds on the same build. **Expected behavior** The saved comment starts a new turn right after recovery saves the files and the sandbox is stopped. The destroy of a stopped sandbox is cleanup and must not hold the next turn. **Steps to reproduce** 1. Use an agent whose sandbox lease is ephemeral (destroy after each turn). 2. Post an `interrupt: true` comment while its native run is still starting, before the provider turn opens. 3. Let recovery save the files. See the follow-up start only after the destroy retry releases the lease. **Paperclip version or commit** Hosted staging at `e37ab23d` (includes #19206 and #19224). ## What Changed - `server/src/services/acknowledged-native-stop.ts`: `acknowledgedNativeStopExecutionHasStopped` accepts a remote lease that has `hasRemoteTerminationReceipt` or `hasRecoveredCleanupStopReceipt`. Before, it accepted only the first one. - `server/src/services/workspace-recovery.test.ts`: new regression for a pre-turn interrupt on an ephemeral lease. Recovery completes, the destroy is queued, and the saved comment is handed to `resumeRemoteStopComments` twice. With a verified carried stop, exactly one turn starts while the lease is still `pending_cleanup`. Without the stop receipt, the comment stays deferred and no turn starts. - `doc/execution-semantics.md`: one sentence that extends the #19224 paragraph to the pre-turn Stop case. ## Verification - Red before the fix: `cd server && npx vitest run src/services/workspace-recovery.test.ts -t "starts the interrupt redirect once after recovery"` failed the `verified-stop` case with `expected [] to have a length of 1 but got +0`. The `no-stop-receipt` case passed. - Green after the fix: the same command passes both cases. - `npx vitest run src/services/workspace-recovery.test.ts`: 382 passed. - Related suites pass: `acknowledged-native-stop`, `explicit-native-continuation`, `heartbeat/run-control`, `heartbeat/recovery`, `heartbeat/native-stop-evidence-read`, `native-session-executor`, `heartbeat-observed-workspace-preservation`, `native-startup-cleanup-recovery`, `execution-control-work`. Two `run-control` cases timed out at 30 s under parallel load and passed when run alone. - `pnpm --filter @paperclipai/server typecheck` passes. - Not done: a live re-run on staging. This change is not deployed. ## Risks - Low risk. `hasRecoveredCleanupStopReceipt` is the same check #19224 added to the other gates. It is true only for a lease whose recovery cleanup belongs to its own run, which was released, and whose carried stop receipt matches the company, run, lease, provider, and provider lease id. A lease without that receipt still blocks. - The no-replay hold is unchanged. The successor still goes through ordinary admission with all task gates. The test calls the resume twice and sees one turn only. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file edits, browser reads of staging). Extended reasoning was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
b400f76647 |
feat(ui): submit task composers with Enter by default (#19272)
Use Enter to submit task replies and new tasks, with Shift+Enter for a new line. Add an experimental preference to require Command/Ctrl+Enter. Preserve autocomplete selection and IME confirmation, and retain the modifier shortcut until saved settings load. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
629abc8509 |
fix(recovery): keep pending Retry unclickable and scope failure causes (#19268)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - #19260 shows the saved-work recovery step on a task, and records why
a recovery attempt failed.
> - The review of #19260 found three gaps after the PR merged.
> - A scheduled recovery offered Retry while a Retry preparation was
already pending.
> - Classified failures could get a misleading `provider_error` cause.
> - The new cause text in the board handoff had no test.
> - This pull request closes these gaps.
> - The benefit is a Retry button that matches what is allowed, and
causes that name the correct subsystem.
## Linked Issues or Issue Description
Refs #19260
**What happened?**
After #19260, a task with a scheduled recovery and a pending (`issued`
or `processing`) Retry preparation showed an active Retry button. The
server only coalesces the second request. Also, a failure that recovery
classified as `archive_unsafe`, `source_unavailable` or
`snapshot_verification_failed` still recorded a provider cause. The
handoff text could then read "last failure: archive_unsafe, cause:
provider_error".
**Expected behavior**
Retry is not offered while a Retry preparation is pending. Only the
generic transient fallback records a provider cause.
**Steps to reproduce**
1. Request a chat Retry on a task while its saved-work recovery is
scheduled.
2. The task projection returns `retry_scheduled` with `canRetry: true`.
**Paperclip version or commit**
master at
|
||
|
Devin FoleyandPaperclip
|
6293e581f6 |
fix(recovery): resume cancelled workspace retries from verified waits (#19276)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A shared workspace can make a run wait before it calls a provider. > - The scheduler saves a retry for that wait. > - The control plane can cancel the retry while it is still queued. > - Recovery then asks for a process stop receipt for a run that never claimed execution. > - This pull request verifies the retry and its exact workspace-wait parent before it accepts a later user instruction. > - The benefit is that Retry and saved messages can continue through normal admission without replaying old work. ## Linked Issues or Issue Description Refs #13640 and #15483. Related fixes: #18838 covers dispatcher cancellation codes; this change covers a system-cancelled scheduled workspace retry. Related open PR #15383 covers claimed conversation-adapter preparation. This change requires an unclaimed retry with no run profile or owner. **What happened?** A scheduled workspace retry was cancelled before runtime selection. The run retained an expected system cancellation and a `startupCancellation.beforeNativeSelection: false` marker. It had no start timestamp, execution stage, run profile, controller, process, lease or provider events. Recovery recorded an unknown-action hold. A later user message and Retry could not pass that hold. **Expected behavior** A later authorized instruction can start a fresh turn when the saved retry and its exact parent prove that no provider work started. Missing or conflicting evidence must retain the hold. **Steps to reproduce** 1. Record a terminal workspace wait before provider dispatch, with `executionRecovery.kind: workspace_wait` and `providerWorkStarted: false`. 2. Save a scheduled retry for that same company, task and agent. 3. Cancel the retry through the control plane before it claims execution. 4. Send a new user message or request Retry after the cancellation. 5. On the base commit, admission returns no continuation. With this change, normal admission creates one fresh successor. **Paperclip version or commit** Reproduced in source based on master at `499712c68b3ca95801b6114d5f2f0f0ea6759318`. **Deployment mode** Source checkout with isolated embedded PostgreSQL. Hosted rollout and a real provider run remain to be verified after merge. ## What Changed - Extend the existing cancelled-startup proof with one source-bound workspace-retry case. Require the canonical retry parent and reason to match the saved context. - Verify the parent's company, task, agent, time order, immutable Runner claim and typed workspace-wait record. Reject runtime, process, lease, coordinator, result and output evidence on either run. - Keep validation of the full saved result in the database predicate. The bounded execution projection can add empty receipt fields without changing eligibility. - Reuse the existing admission, Retry and deferred-message paths. Preserve cancelled history and unknown action outcomes. Keep permission, Stop, budget and recovery-limit gates. - Add 23 regression cases, including heartbeat message and Retry admission, concurrent saved-message recovery, and 19 conflicting-evidence cases. Document the contract. The cancelled-startup service owns this proof. No provider protocol, schema, UI or shared contract changes. Reads use exact run IDs and return only narrow metadata. The new reads occur only for a cancelled, unstarted Runner retry candidate. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/services/explicit-native-continuation.test.ts src/services/legacy-execution-recovery.test.ts src/services/execution-blocker-read.test.ts src/services/task-retry.test.ts` — 452 passed. - `pnpm -r typecheck` — passed. - `pnpm build` — passed. - `node scripts/check-module-boundaries.mjs` and `git diff --check` — passed. - Local diff review and secret/private-context scan — passed. Test identities are synthetic. - Regression history: the original positive case failed on unchanged source while 11 negative guards passed. Initial test fixtures then exposed ownership/contract constraints and a missing saved-wake envelope; the fixtures now use valid persisted shapes. The heartbeat Retry test exposed extra keys in the bounded evidence projection. Full-record validation now runs in SQL, and both assembled heartbeat paths pass. The unknown action outcome stays unchanged. - Platform: macOS arm64, Node 25.6.1, pnpm 9.15.4. Tests use real isolated PostgreSQL. They stop at admission and do not call a live provider. - Exact-head GitHub CI: 54 successful checks, 3 skipped, no unfinished or failed checks. Greptile scored `cfbf55cea68e3c69f65f50afcba98f775fa6fa3d` 5/5, with no actionable findings and no review threads. - The full repository `pnpm test:run` was not run locally. A previous full run in this repair effort stalled and was stopped after 2h45. The four complete affected suites pass; full PR CI must pass before merge. ## Risks - False admission could allow work before a predecessor is safe. The proof fails closed on missing parent records, different ownership, a changed retry binding, later parent completion, restore obligations, any execution resource, or any provider/output evidence. - This deliberately covers only system-cancelled, unclaimed workspace retries. Other cancellations retain their current reconciliation path. It does not grant automatic replay, certify unknown external writes, or reset exhausted recovery limits. - No migration. Verify the serving version and an existing saved-message continuation after deployment. ## Model Used - OpenAI `gpt-6.1-sol`, xhigh reasoning, through the Codex harness in T3 Code. Used code editing, shell tools and test execution. The harness does not expose its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
34d308a331 |
fix(native): let a run retire the prior owner it reserved when its session rotates (#19263)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native Runner agent keeps a warm provider session per task. A follow-up run on the same task can reuse that session and its managed instruction directory. > - When a follow-up run cannot resume the saved session (for example, the effective run configuration changed), it first reserves the prior owner's instruction directory. Then it starts a new session identity and must retire the prior owner. > - Retirement treated the run's own directory reservation as another executor's busy turn. The run failed at startup with `native_session_supervisor_busy`. > - That failure happened after the remote workspace was prepared, so failure handling retained the sandbox and blocked the task. A board follow-up comment left the task `blocked` with no agent progress. > - This pull request lets a run retire the prior owner that it reserved itself, and keeps the directory that it adopted. Any other reservation or turn is still busy. > - The benefit is that a follow-up on a task with a rotated session runs once, and the task does not become blocked. ## Linked Issues or Issue Description **What happened?** A board follow-up comment with reopen woke an agent on a task that had just finished a successful run. The new run reused the task's own sandbox. The saved session could not resume because the effective run configuration changed, so the run started a new session identity. At startup (right after the native execution claim) the run failed with `native_session_supervisor_busy`. The run got the error code `native_continuation_requires_reconciliation`, the sandbox was retained for workspace repair, and the task became `blocked`. The agent had another live run on a different task at the same time. That run used a different sandbox and a different workspace, so it was not the cause. The busy owner was the task's own prior warm session. The new run had reserved that session's instruction directory itself (`reserveWarmNativeInstructionDirectory` sets `preparingRunId` to the new run). `retireSupersededWarmNativeSessions` then saw `preparingRunId` and rejected the run. **Expected behavior** The follow-up run retires the prior warm owner of its own task, keeps the instruction directory it adopted, starts the new session, and runs once. The task does not become blocked. **Steps to reproduce** 1. Run a native Runner (Codex app-server) task to completion with a warm session and a managed instruction directory. 2. Change the agent's effective run configuration, so the saved session cannot resume. 3. Wake the same task again (for example, a board comment with reopen). 4. Before this change, the run fails at startup with `native_session_supervisor_busy` and the task becomes blocked. **Paperclip version or commit** Observed on staging at `7fd05e13`. The code is unchanged on `master`. ## What Changed - `retireSupersededWarmNativeSessions` now receives the current run id. When the prior owner of the same task (same logical owner scope) is reserved by this run (`preparingRunId === runId`), retirement continues instead of throwing `native_session_supervisor_busy`. - That retirement closes the prior owner with `preserveInstructionsForRunId`, so the directory that this run adopted is not collected. This matches the existing configuration-change branch in warm reuse. - After a successful close, the reservation is cleared. A failed close keeps the reservation, so its `release()` retries retirement as before. - A reservation by any other run, a busy owner, and an executing scope still throw `native_session_supervisor_busy`. ## Verification - Red before the fix, green after (`server/src/services/native-runtime/native-session-executor.test.ts`, "managed directory warm checkpoints"): - "retires its own reserved directory owner when the session identity rotates": before, the run rejected with `native_session_supervisor_busy`. After, the prior owner closes once, the new session runs once, the adopted directory is not collected during retirement, and it is collected once when the new owner retires. - "retires its own reserved directory owner after the Runner claim when the session identity rotates" (the production `useRunnerd` path, where retirement happens after the claim): before, admission rejected with `native_session_supervisor_busy`. After, admission retires the prior owner and keeps the adopted directory. - Safety case (green before and after): "still rejects a session identity rotation over another run's directory reservation". The provider does not start and the other run's owner is not closed. - Existing safety case (green): "does not retire an active turn when a new session identity arrives". - `npx vitest run src/services/native-runtime/native-session-executor.test.ts src/services/heartbeat/instruction-working-copy`: 1163 passed. - Staging evidence was read only. This change is not deployed. ## Risks - Low risk. The change applies only when the prior owner has the same logical owner scope (company, agent, task, workspace, environment) and its reservation names the current run. That reservation exists only after this run validated reuse of the prior directory. - The claim-path instruction copy (`claimWarmNativeInstructionCopy`, which sets `busy`) is unchanged. That path is not used by Codex app-server agents. - Two related points are unchanged and are out of scope here: a startup failure after the workspace is prepared still retains the sandbox and blocks the task, and the "Instruction edits could not be recovered" warning on that failure is advisory only. With this fix, this flow no longer reaches that failure. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, file edits, browser read of staging run records) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Refs #18893 (shared-sandbox ingress retirement). Refs #19143 (workspace-busy wait). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
01dd6ab336 |
fix(recovery): show the saved-work recovery step and record why it failed (#19260)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Sometimes a failed run left files in a sandbox. Then Retry first
runs system recovery to save those files before the task continues.
> - Recovery has steps: confirm that the previous attempt stopped, save
its files, then finish.
> - The task page did not show these steps. People saw "Queued" or
"Trying again in…" with a retry time that kept moving.
> - Each unclassified failure of the stop-confirmation step was recorded
only as `provider_unavailable`, so a repeated deferral could not be
diagnosed.
> - This pull request shows the current recovery step on the task. It
also records why each failed attempt failed, in closed labels.
> - The benefit is that people can see what Retry waits for, and
operators can see why recovery deferred.
## Linked Issues or Issue Description
Refs #19249
**What happened?**
A person clicked Retry on a task whose failed run had a retained
sandbox. For about 7 minutes the task showed only "Queued", and the
retry time moved forward twice. Recovery deferred twice in the
stop-confirmation step. Each deferral was recorded only as
`provider_unavailable` with an opaque fingerprint, so the cause could
not be found.
**Expected behavior**
The task shows the recovery step that it waits for, such as "Checking
that the previous attempt stopped. Trying again in 30s…". Each failed
attempt keeps a closed-label cause, for example `sandbox_not_running` or
`transport_timeout`, with an optional errno-style code and HTTP status.
**Steps to reproduce**
1. Get a remote task run to fail after its sandbox was placed, so that
recovery owns saved work.
2. Click Retry while the sandbox provider does not respond to the stop
check.
3. The task shows "Queued" or "Trying again in…". The recovery evidence
records only `provider_unavailable`.
**Paperclip version or commit**
master at
|
||
|
DottaandClaude Opus 5.5
|
cb5d7e440c |
fix(server): retry setup when the plugin worker was not running (#19249)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote agent runs call a sandbox provider plugin, such as Daytona,
through a plugin worker process.
> - Sometimes that worker is not running. It can be stopped, restarting
after a crash, or retired by the host.
> - Then the host refuses the call before it sends anything to the
worker.
> - Setup recorded this refusal as a generic `setup_failed`. It
scheduled no retry, so the task became stranded and needed a person.
> - This pull request types the refusal and gives it its own transient
setup code. The bounded bootstrap retry then runs.
> - The benefit is that the task continues on its own when the worker is
available again. Users do not see a blocked task.
## Linked Issues or Issue Description
Refs #19150 (removed one trigger for the worker retirement)
Refs #19245 (restarts a worker after a fail-closed retirement)
**What happened?**
A plugin worker stopped. Each run that then called `environmentExecute`
during setup failed with `Cannot call "environmentExecute" — worker for
"<plugin>" is stopped`. The run got `setup_failed`, no retry was
scheduled, and the task moved to board recovery. On one staging
instance, 15 runs failed this way in 12 minutes.
**Expected behavior**
No provider work started, so the failure is safe to retry. The run must
retry within the bootstrap budget, as it does for a reusable sandbox
resume failure or a preparation timeout.
**Steps to reproduce**
1. Stop the sandbox provider plugin worker, or let the host retire it.
2. Wake an agent whose run needs that provider during setup.
3. The run fails with `setup_failed`. No retry is scheduled.
**Paperclip version or commit**
master at
|
