mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 18:26:40 +01:00
92271da8991cbfc144240ab37a12b40d8a00c603
1005
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
92271da899 |
fix(native): keep a chat turn's message batch when a fresh-session retry continues it (#22413)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - External chat turns (GitHub, Slack, Discord) wake an agent on a task
with an admitted message batch.
> - When a task is in review, dispatch checks that the run is bound to
that reviewed chat batch. A run without a batch fails with
`reviewed_chat_execution_binding_not_authorized` (`context_mismatch`).
> - Automatic fresh-session retries (tool-receipt limit, provider
capacity, environment blocker, agent blocker, and safe replacement)
remove all consumed wake input from the failed run's context. This also
removes the chat message IDs, but the context keeps `source:
chat:<provider>`.
> - Thus every automatic retry of a reviewed chat turn fails at setup.
The one-shot review-path recovery that follows copies the same empty
context and fails the same way.
> - This pull request keeps the admitted chat message batch on those
retries and removes the failed run's dispatch proofs, so dispatch checks
ownership and current access again.
> - The benefit is that a reviewed chat turn can continue after a
transient failure, and access checks still apply.
## Linked Issues or Issue Description
Refs #13818 (the same approach for review-path recovery). Related:
#20689, #20697, #20842 (other causes of the same denial).
**What happened?**
A GitHub PR review turn on a task in review stopped on a transient
failure (tool-receipt limit or sandbox environment blocker). Paperclip
scheduled an automatic fresh-session retry. The retry failed at setup in
less than one second with
`reviewed_chat_execution_binding_not_authorized` and denial reason
`context_mismatch`. The review-path recovery wake then failed with the
same error. On staging this occurred 5 times on 3 tasks after #20689,
#20697 and #20842 were deployed.
**Expected behavior**
An automatic retry of a chat turn keeps the admitted message batch.
Dispatch then checks the retry against current task ownership and
current endpoint, conversation and principal access. If access is still
valid, the retry runs. If access was revoked, dispatch refuses the
retry.
**Steps to reproduce**
1. Put a task in review with a run-bound external chat turn (for example
`source: chat:github` with `wakeCommentIds`).
2. Fail that native run with a fresh-session retry lane (for example
`native_environment_blocker`).
3. Let dispatch claim the scheduled retry. The retry context has
`source: chat:github` but no `wakeCommentIds`.
4. Dispatch refuses the retry with `context_mismatch`.
**Paperclip version or commit**
`master` at
|
||
|
|
5db2ccdae6 |
fix(sentry): keep a continued native tool-receipt-limit stop out of Sentry (#22399)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server sends unexpected run failures to Sentry as run-failure events. It keeps known, recovered failures local > - The runner stops a native turn when the turn holds too many tool results (`semantic_tool_turn_receipt_limit`). The finalizer then continues the task in a fresh session > - The runner's stop has no recorded cancellation request. So the cancelled run keeps `unknown`, unexpected cancellation provenance, and `reportRunFailure` sends it to Sentry as "Native session cancelled" > - The continuation owns recovery, so this event is noise. The native capacity retry had the same problem, and #19051 and #19320 fixed it with the successor run as the proof > - This pull request uses the same successor-run proof for the tool-receipt-limit continuation > - The benefit is that a continued receipt-limit stop does not make Sentry noise. Exhausted continuations still report ## Linked Issues or Issue Description Refs #19051, #19320 **What happened?** On staging, a native Codex run stopped at its tool-result receipt limit. The finalizer committed the `semantic_tool_turn_receipt_limit` decision and scheduled a fresh-session continuation run (`retryOfRunId` = the stopped run, `scheduledRetryReason` = `semantic_tool_turn_receipt_limit`). The continuation succeeded and the task finished. Sentry still got a run-failure event: `Error: Native session cancelled`, `run_status=cancelled`, cancellation `source: unknown`, `expected: false`. **Expected behavior** A receipt-limit stop that has a scheduled continuation stays local. The continuation reports itself if it fails. A stop with no continuation (exhausted, or the user stopped it) stays reportable as before. **Steps to reproduce** 1. Run a native turn that reaches the semantic-tool receipt limit. 2. Let the finalizer schedule the fresh-session continuation. 3. With `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected`, see that the cancelled run is sent to Sentry. ## What Changed - `server/src/services/run-failure-report.ts`: `isScheduledNativeCapacityRetry` becomes `isScheduledNativeRetry`. It reads the successor run for the lane that can own the run: `native_provider_overloaded` for a failed capacity run (unchanged), and `semantic_tool_turn_receipt_limit` for any other failed or cancelled native run. The query, task binding, setup-phase, signal, and read-failure behavior do not change. - `server/src/services/__tests__/run-failure-report.test.ts`: regression tests with the persisted shape of the staging run, plus negative cases (no successor, other retry lane, other task, no native task, legacy, signal, timed out, setup phase, `all` mode, failed read). - The shared `buildRun` fixture in `run-failure-report.test.ts` now has `createdAt`, like a persisted run row. Without it, the wider scheduled-retry read failed on in-memory test runs and logged a spurious warning. Real report inputs are full database rows, so production code does not change for this. - `server/src/services/native-runtime/native-tool-receipt-limit.integration.test.ts`: the real native executor and finalizer path now also asserts the Sentry outcome. A continued stop sends no report. A cancelled turn without a continuation still sends one. The fixture run now has `startedAt`, as a real started run does. Without it, no cancelled-run report can occur, and the new assertion would prove nothing. ## Verification - Red before the fix: the two new unit cases failed (`captureRunFailure` was called), and the integration assertion failed (`expected [...] to have a length of +0 but got 1`). - Green after the fix: - `npx vitest run src/services/__tests__/run-failure-report.test.ts src/__tests__/run-failure-sentry.test.ts src/__tests__/run-failure-sentry-real-sdk.test.ts src/services/native-runtime/native-provider-overload.integration.test.ts src/services/native-runtime/native-tool-receipt-limit.integration.test.ts src/services/native-runtime/status-arbiter.test.ts` → 383 passed, 13 skipped. - `node scripts/typecheck-layers.mjs` in `server` → no errors. - `pnpm verify` did not finish in the local sandbox in the time limit. The only failures were 3 cases in `server/src/__tests__/archived-nested-repository.test.ts`. They also fail on unchanged `master` in the same sandbox, and they do not use this code. CI runs the full suite. ## Risks - Low risk. The change only removes Sentry events. It does not change run status, task status, retries, or the user's view. - A failed or cancelled native run that is not a capacity failure now does one more indexed read before its report. That read is bounded by company, task, and agent, and it happens only when a report is about to be sent. - A successor with this retry reason exists only when the status arbiter proved the receipt-limit stop from a persisted runner diagnostic. A user Stop refuses the continuation, and an exhausted lane creates none, so these runs still report. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, code search, and test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c3260b1c03 |
fix(native): name the missing precondition when a bootstrap restart refuses an unopened provider (#22406)
Adds a closed diagnostic suffix to the provider_identity_incomplete refusal raised when a same-run bootstrap restart meets an unopened provider: bootstrap_restart=<same_run|other_run>,provider=<unopened|opened>,proof=<held|not_recorded|stop outcome>. Recovery behaviour is unchanged. Co-Authored-By: Paperclip <noreply@paperclip.ing> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9874608d0a |
fix(native): retry a workspace failure write that loses a deadlock (#21507)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The native runtime finalizes each agent run after the provider turn
ends. It copies the workspace back and records the result.
> - When the workspace copy-back fails, `recordRetryableFailure` records
the failure and a bounded export retry in one transaction.
> - That transaction locks the coordinator, issue and run rows. Then the
recovery-action insert takes `FOR KEY SHARE` on the agent row for its
foreign keys.
> - A different transaction that holds the agent row and waits on the
run row deadlocks with it. Postgres aborts the failure write with 40P01.
> - The run then fails with `Database query failed (40P01)`, and the
export retry is never recorded.
> - This pull request runs the whole failure write through the existing
`retryTransientLockContention` helper.
> - The benefit is that a lost deadlock no longer fails the run or drops
its workspace export retry.
## Linked Issues or Issue Description
Refs #21494 (same mechanism, but for the provider-turn failure write in
`native-session-executor.ts`. This PR covers `native-run-finalizer.ts`.)
Refs #21506 (a different agent-row deadlock in run setup. It does not
change this write.)
**What happened?**
On a hosted staging server, two runs finished their provider turn, and
then the workspace copy-back failed. The server tried to record that
failure and lost a Postgres deadlock on `insert into
issue_recovery_actions`. The stack was `recordNativeFinalizationFailure`
-> `recordRetryableFailure` -> `upsertSourceScoped`. Each run failed
with `Database query failed (40P01)`.
**Expected behavior**
A lost deadlock in the failure write is retried. The run records
`retryable_failure` and an active `active_run_watchdog` recovery action
for the bounded workspace export retry.
**Steps to reproduce**
1. Hold `FOR UPDATE` on the agent row in one transaction.
2. Start `recordRetryableFailure` for a run of that agent. It locks the
run and issue rows, then waits on the agent row.
3. In the first transaction, request a lock on the run row. Postgres
detects the deadlock and aborts one side with 40P01.
The new integration test does these steps with embedded Postgres.
**Paperclip version or commit**
`master` at
|
||
|
|
b5c53a996b |
fix(native): stop a repeated startup rejection when a run resumes its predecessor's provider session (#22393)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs on a remote sandbox start a Runner, and the Runner opens
the provider session with `session.open`
> - When the provider refuses `session.open` at startup, no provider
work began. The executor records a startup failure signature, so the
next attempt can stop a repeat of the same refusal (#20874)
> - Only bootstrap retries recorded that signature. A run that continues
its predecessor's provider session (for example a provider-limit Retry)
recovers as an exact checkpoint resume, so it never recorded one
> - Such a run retried every 30 seconds against the same provider rate
limit, spent all 3 attempts, and ended as
`provider_session_open_rejected`. That outcome is reported as a run
failure
> - This pull request records the signature for every remote startup
rejection
> - The benefit is that the second identical refusal ends the run at
once. With #22386, a repeated rate-limit refusal then becomes the
usage-limit hold that retries the task at the reset
## Linked Issues or Issue Description
Refs #22386
Refs #20874
**What happened?**
A staging run resumed its predecessor's provider session. The provider
refused `session.open` with HTTP 429 ("would exceed your account's rate
limit") on each attempt. The recovery mode was
`exact_checkpoint_resume`, so no startup failure signature was saved.
Each attempt looked new. The run made 3 attempts in about 3 minutes and
ended as `provider_session_open_rejected` (retry budget spent).
**Expected behavior**
The second identical startup refusal stops the retries, as it does for a
bootstrap retry. A rate-limit refusal waits for the reset (with #22386)
and does not use the full retry budget.
**Steps to reproduce**
1. Give a remote native run a session checkpoint with an established
provider session.
2. Make the provider refuse `session.open` at startup with the same
error on two attempts, with the runner exit proven.
3. See that the second attempt is retried again, and the run ends only
when the attempt budget is spent.
**Paperclip version or commit**
`master` at
|
||
|
|
4d954646eb |
fix(native-runtime): refresh a runtime asset lease once for concurrent turns (#22384)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Before a turn starts, the server copies each assigned skill into a
content-addressed runtime asset bundle. Retention garbage collection can
delete old bundles.
> - `withRuntimeAssetUse` keeps a bundle safe from deletion with an
hourly access lease. It records the lease under the same directory
publication lock that deletion uses. The lease is held in memory, so it
is cold after a server restart and again each hour.
> - When the lease is cold, each concurrent turn that uses the same
bundle takes the exclusive lock in sequence and verifies the full bundle
again. One large skill has 1,350 files (24 MB). On a busy server, the
turns at the end of that queue wait more than the 30 s lock budget. The
run then fails with "Timed out waiting for workspace restore lock". The
lock diagnostics show that the same process holds the lock
(`knownLocalHolder: true`).
> - This pull request lets concurrent uses of one digest share one
locked lease refresh.
> - The benefit is that a restart or an hourly lease expiry no longer
makes the preparation of parallel turns fail.
## Linked Issues or Issue Description
No public issue exists. This is the bug description.
**What happened?**
Agent runs on a hosted staging server failed during preparation with
`ERR_WORKSPACE_RESTORE_LOCK_TIMEOUT` ("Timed out waiting for workspace
restore lock at <instance>/locks/directory-merge/<sha256>.lock"). The
stack is `materializeSelectedSkills` (Promise.all) ->
`materializeSkillAsset` -> `withCanonicalDirectoryMergeLock` ->
`acquireDirectoryMergeLock` -> `waitForLock`. The lock diagnostics were
`knownLocalHolder: true` and `waitMs: 30052`, and the owner sidecar read
timed out. The server process had started 48 minutes before the failure.
Eight events occurred in two days. One agent ran many turns in parallel
with the same large skill.
**Expected behavior**
Concurrent turns that use the same runtime asset bundle start without
waiting for each other to verify the bundle again.
**Steps to reproduce**
1. Start a server with an empty in-memory lease map (a new process).
2. Start many turns at the same time. Assign the same large skill to all
of them.
3. Each turn takes the publication lock for that bundle in sequence. The
last turns time out after 30 s.
The new unit test shows the cause directly: 8 concurrent cold uses of
one digest take the lock 8 times on `master`. With this change, they
take it once.
**Paperclip version or commit**
Release `24132dac169fafc616615b4f3144e921aae8ad54`. The code is
unchanged on `master`.
**Deployment mode**
Hosted (staging).
**Agent adapter(s) involved**
Not adapter-specific (core bug).
## What Changed
- `withRuntimeAssetUse`
(`server/src/services/native-runtime/runtime-asset-retention.ts`): the
first non-mutation use of a digest with a cold lease runs the locked
refresh. It registers the refresh in a per-process map before its first
`await`. Concurrent non-mutation uses of the same digest wait for that
refresh. When the refresh records the lease, they use the asset without
the lock, as warm turns already do. When it records no lease (for
example, the refresh failed or no manifest exists), each waiter takes
the lock as before.
- Mutation uses (`captureSkills`, rebuilds) always take the lock. Their
behavior does not change.
- The locked body moved to a `useLocked` helper without changes.
- Two regression tests in `runtime-asset-retention.test.ts`: one refresh
serves 8 concurrent cold uses, and waiters fall back to the lock when
the shared refresh records no lease.
## Verification
- Red: on `master` the new test fails with `expected [ true, true, true,
true, true, …(3) ] to have a length of 1 but got 8`.
- Green: `npx vitest run
src/services/native-runtime/runtime-asset-retention.test.ts` passes 4/4
with this change.
- Related suites pass: `runtime-context.test.ts`,
`runtime-skill-preservation.test.ts`, `managed-skill-content.test.ts`,
`ephemeral-server-runtime-skill-startup.test.ts` (with the retention
suite: 103 of 104 tests pass). The single failure is
`remote-launch-assets.test.ts` "delivers the selected Codex skills…"
(`expected 256 to be 292`). It fails the same way on unmodified `master`
in this sandbox because the sandbox umask is 077.
- `node server/scripts/typecheck-layers.mjs`: all layers pass (below,
cycle, above, tests, test-tree-0..2), exit 0.
## Risks
- Low risk. The change only removes repeated lock acquisitions while one
refresh for the same digest runs in the same process. The lease
semantics and the deletion lock do not change.
- A waiter uses the asset without the lock only after the shared refresh
recorded a fresh lease. This is the same condition that warm turns use
today.
- If a refresh never settles, its waiters wait with it. Before this
change they waited on the lock and failed after 30 s. The lock
acquisition in the refresh keeps its own 30 s budget, so a refresh
cannot wait for the lock without limit.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
1196a3402d |
fix(native): do not report a provider usage-limit run that a restart ended (#22385)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs report an unexpected terminal failure to Sentry through the run-failure reporter > - The reporter keeps an expected provider usage-limit failure (`native_provider_usage_limit`) out of Sentry, because the task takes its provider-limit wait and runs again after the reset > - A server restart can end such a run after its provider turn failed on the limit, but before its executor saves `native_provider_usage_limit`. Restart recovery then blocks the run as `provider_checkpoint_permanently_failed`, and the reporter sends it to Sentry > - On staging, one deploy restart sent seven such reports at once. Each task then took its provider-limit wait and finished after the reset, so no operator action was necessary > - This pull request re-derives the usage-limit fact from the run's last committed runner turn terminal and keeps that expected failure out of Sentry > - The benefit is that a deploy during a provider limit does not create false error reports, and real checkpoint failures are still reported ## Linked Issues or Issue Description **What happened?** A deploy restarted the server while several native runs were settling a provider turn that failed on a weekly allowance (`quota_exhausted`). Restart recovery blocked each run as `native_restart_recovery_blocked` / `provider_checkpoint_permanently_failed`. The run-failure reporter sent one Sentry event per run. **Expected behavior** A run that ended on a provider usage limit is an expected failure. The reporter does not send it to Sentry when the executor settles it normally. A restart during settlement must not change that. **Steps to reproduce** 1. Start a native run whose provider turn fails with a `quota_exhausted` or `rate_limit` provider failure. 2. Restart the server before the executor saves `native_provider_usage_limit`. 3. Restart recovery blocks the run as `provider_checkpoint_permanently_failed`, and the reporter sends a Sentry event. Refs #20149 ## What Changed - `native-provider-failure.ts`: move `nativeProviderUsageLimitFromEvent` here from `native-session-executor.ts` so that the evidence reader can use it. The executor re-exports it, so its callers do not change. - `native-provider-failure-evidence.ts`: add `readPersistedNativeTerminalUsageLimit`. It reads the run's last committed turn lifecycle event. It returns true only when that event is a `turn.failed` from the run's own runner instance, the PRP event validates against the stored row (run, session, source, sequence and payload hash), and the failure is a provider usage limit. - `run-failure-report.ts`: for an unexpected `native_restart_recovery_blocked` / `provider_checkpoint_permanently_failed` failure, skip the report when that reader returns true. A read error logs a warning and keeps the report. - Add a real-process restart-recovery test with four cases: usage limit and rate limit are not reported; an unknown failure and a later turn after the limit are still reported. ## Verification - `cd server && npx vitest run src/services/native-runtime/native-runner-restart-recovery.integration.test.ts -t "provider usage limit|pending retry|unreported"`: 5 passed. - Red proof: with only `run-failure-report.ts` reverted, the `usage-limit` and `rate-limit` cases fail (`expected ... to have a length of +0 but got 1`). The two report cases still pass. - `cd server && PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm run typecheck`: passed. - `npx vitest run src/services/run-failure-report src/services/native-runtime/native-provider-failure src/services/native-runtime/native-session-executor.test.ts`: 1619 passed, 1 failed. The failure (`compares export process compatibility without changing conversation identity (changed: false)`, `db.transaction is not a function`) also fails on unchanged `master` (`ce8006de3c`), so this change does not cause it. ## Risks - Low risk. The change only removes a Sentry report. It does not change run status, task status, recovery, or the provider-limit wait. - The filter needs a validated, run-bound runner terminal. A missing, tampered, foreign or later event keeps the report. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in an agent harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
62bc0519d9 |
fix(native): bound per-turn harness backups by inactivity, not total time (#22382)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - After a remote native turn is accepted, the executor saves the provider's harness state to durable storage, so another web server can continue the same conversation. > - That save runs under a fixed 300 s total deadline (`AbortSignal.timeout(300_000)`). The deadline covers archive capture, digest, encryption, the upload, any storage-outage wait and a full verifying read-back. > - On staging, five accepted runs failed this save 337–388 s after it began. The run then reported `native_workspace_sync_out_failed` with "The operation was aborted" or tar-stream's "already finalized or destroyed", and workspace copy-back went to retry. > - #19906 fixed the same fixed-deadline rule on the restore side. The write side still uses it. Since #20772/#20871 the save also waits out storage throttling and outages for up to five minutes, which cannot finish inside a five-minute total. > - This pull request bounds the upload and the read-back by inactivity instead of total time. A slow transfer continues while the store makes progress. A transfer that stops fails with a named code. > - The benefit is that an accepted turn no longer loses its provider backup only because the backup is large or the store is slow. ## Linked Issues or Issue Description Refs #19906 (same idle-deadline rule for failover-backup restores), #20772 and #20871 (file-source resends and storage-outage waits during the save), #19108 (stops tar-stream from hiding the real abort), #18932 (durable harness backups). **What happened?** On staging, five native per-turn runs failed within one hour, all after the provider turn was accepted. In each run, `harness_state.backup.persist` failed after 359,588 / 365,422 / 387,471 / 383,801 / 337,335 ms, and `session.checkpoint.persist` failed with it. The only fixed limit on that path is `AbortSignal.timeout(300_000)` around `persistNativeHarnessBackup` in `native-session-executor.ts`. The durations sit just above 300 s, plus a variable sandbox sync-out step before the save. Errors were reported as "The operation was aborted" and, before #19108, as "already finalized or destroyed". **Expected behavior** An accepted turn saves its provider backup while the store keeps making progress, even when the total time is more than five minutes. A write or read-back that stalls fails with a named, diagnosable code. A failed save publishes no backup. **Steps to reproduce** 1. Persist a native harness backup through a store that takes upload chunks slowly, so the full save takes longer than the deadline. 2. With the executor's fixed total deadline, the save fails with "The operation was aborted" and publishes nothing. 3. A store that stops taking bytes, with no caller deadline, waits indefinitely. ## What Changed - `native-harness-backup-storage.ts`: generalize `withDownloadProgressDeadline` into `withTransferProgressDeadline`. It still wraps `getObject` exactly as before, and it now also wraps `putFile` with a five-minute idle timer: - A provider that reads the stream body (local disk) renews it with each chunk it takes. - A provider that sends the file source itself (S3 with `sourceFile`, which never reads the stream) renews it with each request the store answers, each multipart part it takes, and each resend or outage wait that starts. - After the last of them the timer stays armed until the store answers, so a write that never completes is named too. - `persistNativeHarnessBackup` gains `transferIdleTimeoutMs` and `persistMaxMs` (default 30 minutes). The outer bound wraps the whole save, including the storage-outage wait. With an idle limit, a stalled upload throws `native_harness_storage_upload_stalled`, a stalled read-back throws `native_harness_storage_download_stalled`, and the whole save throws `native_harness_storage_persist_deadline_exceeded` at the outer bound. Caller cancellation keeps its own reason, and a storage outage that outlasts its own budget still surfaces as the storage error. Callers that pass no idle limit (controller archive, offline migration, preservation) are unchanged. - `storage/request-stats.ts`: add `observeStorageProgress(onProgress, work)`, an `AsyncLocalStorage` observer called from the existing `recordStorageRequest`, `recordStoragePart` and `recordStorageBackoff` hooks. It is scoped to the wrapped `putFile` call only, and a throwing observer is ignored. Stats collection is unchanged. - `native-session-executor.ts`: the per-turn backup passes `NATIVE_HARNESS_UPLOAD_IDLE_TIMEOUT_MS` (five minutes) instead of `AbortSignal.timeout(300_000)`. - Restore behavior and error codes are unchanged: `withDownloadProgressDeadline` now delegates to the shared helper with the same defaults. - `doc/database-backups.md`: document the write-side idle limit, what counts as progress, and the new codes. Previous-release story (`rules/upgrade-safe-run-state.mdc`): no persisted shape changes. Backup rows, references, receipts and checkpoints are unchanged. A deploy that lands during a save stops the process, and the pending row is never published, as before. ## Verification Run from `server/` (provider env vars unset): - `pnpm exec vitest run src/services/native-runtime/native-harness-backup-upload-deadline.integration.test.ts`: 6 tests passed. They use embedded PostgreSQL, local-disk storage and an incompressible 1 MB acpx backup. - "keeps saving a backup whose upload outlasts the idle window while the store keeps taking bytes". Under the former fixed-total policy the same slow upload fails with "The operation was aborted" and publishes nothing. With the idle limit it completes after more than twice the window and restores. - "names an upload that stops taking bytes and publishes no backup", then saves the same source once the store responds. - "keeps a file-source upload going while the store answers its requests, and names one that stops". The fake store never reads the stream and records each request answer the way the S3 provider does. - "names a read-back that stops sending bytes and publishes no backup". - "stops at the outer bound while an upload still progresses". - "lets Stop end a progressing upload with its own reason". - RED, part 1: with `native-harness-backup-storage.ts` and `request-stats.ts` reverted to `master`, 4 tests fail. Three stall tests hang until the 10 s test timeout, and the outer-bound test resolves instead of rejecting. - RED, part 2: with only the progress observer disabled, the file-source test fails with `native_harness_storage_upload_stalled` on an upload that is still progressing. - `native-harness-backup-*` suites, `native-harness-online-preservation`, `src/storage/`, `storage-request-stats` and `storage-s3-multipart-backoff`: 200 of 201 passed. The one failure, `native-harness-backup-cleanup.test.ts` ("removes retired backups containing immutable skills…", expected mode 256 to be 292), fails the same way on unmodified `master` in this sandbox. - `native-session-executor.test.ts`: 1541 of 1542 passed. The new assertion checks that the per-turn backup passes `transferIdleTimeoutMs` and no fixed signal, and flipping it fails the `opencode-new-durable*` cases. The one failure, "compares export process compatibility without changing conversation identity (changed: false)" (`db.transaction is not a function`), fails the same way on unmodified `master`. - `node scripts/typecheck-layers.mjs`: no errors in `server/`. The local filtered install lacked some CLI/UI/demo packages, which reported unrelated errors. CI covers the full typecheck. Not verified: a live staging run with this build. The staging evidence shows the save failing just after 300 s on several runs. It does not show which step was slow (upload, read-back or local capture), so the fix removes the fixed total for every step on that path. ## Risks - A save that progresses very slowly can now take longer than five minutes. Before, it failed. The settle step waits for it, up to the 30-minute outer bound. - A local step that hangs (for example a stalled disk during capture) is now cut off at 30 minutes instead of 5 minutes, with `native_harness_storage_persist_deadline_exceeded`. - For S3 file-source uploads, progress is measured per request, not per byte. A single request that takes more than five minutes counts as stalled. For a 16 MiB single PUT, that is a store slower than about 55 KB/s. Multipart parts are 8 MiB, eight in flight. - If a successor takes over during a long save, the existing fence checks before and after the upload reject it with `owner_superseded`, and nothing is published. This is unchanged behavior. ## Model Used - Provider: Anthropic. Model: Claude Opus 5.5 (`claude-opus-5-5`), with tool use (shell, file edits) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fe0c4aa95d |
fix(recovery): settle a late workspace hold when a remote-epoch continuation already took over (#22387)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Restart recovery must continue a task after a server is killed. A person must never need to press Retry. > - When Paperclip cannot resume a cut-off run exactly, it records a "fresh continuation". For a remote workspace, the continuation abandons the old run's workspace epoch and restores the last save into a new root. It does not apply the old run's unsaved files. > - Workspace recovery can try to save the old run's files after that. The epoch is already abandoned, so the source is "unavailable". Recovery then wrote an active hold that asked a person to press Retry to continue in a new workspace, which the continuation had already arranged. > - That hold made the dispatch gate withhold every queued run of the task, and the task rested on a Retry banner. > - This pull request settles that hold into the recorded continuation at the moment workspace recovery would leave it exhausted. Dispatch and continuation delivery are unchanged. > - The benefit is that the queued continuation starts automatically after a hard kill, with no Retry banner. ## Linked Issues or Issue Description **What happened?** A restart-chaos release-swap wave hard-killed the old server. One task (of 6) then rested on a Retry banner for about 5 minutes. The other 5 tasks passed. The sequence for the failed task: 1. A run started on the incoming server. The wave's injected database delay held its process-identity write for 480 s, so its runner attach timed out. The run saved a same-run resume. 2. Restart recovery found a newer queued run for the task (a child-completion wake). It treated the same-run resume as superseded, and the run ended `failed` with `native_restart_recovery_blocked`. 3. The dispatch gate withheld the queued run (`execution_reconciliation_required`). In the same transaction, it recorded a fresh continuation with `remote_epoch` isolation, which abandoned the old run's remote workspace epoch. 4. 4 s later, workspace preservation for the old run found the abandoned epoch (`source_unavailable`, `use_alternative_strategy`). It left an active hold: "Paperclip cannot recover the unsaved files from the previous run ... Retry continues the task in a new workspace". The retained-source fallback also refused it later, with `workspace_authority_changed`, because the continuation owned the epoch. 5. The gate withheld both bounded retries because of that hold. The retries were spent, and the task showed a Retry banner. 6. A person pressed Retry. Retry started a run in a new workspace, which the recorded continuation had already decided to do. **Expected behavior** The queued continuation starts automatically. No Retry banner shows. **Steps to reproduce** 1. Run the restart-chaos K1 kill variant: 2 release swaps, and the old server is hard-killed 120 s after SIGTERM. 2. Use an OpenCode task on a remote sandbox whose run starts just before the second swap and that gets a child-completion wake while it runs. 3. Watch the task after the kill. Workspace recovery writes the unsaved-files hold, the gate cancels each queued run with `execution_reconciliation_required`, and the task rests on a Retry banner. `server/src/__tests__/heartbeat-isolated-continuation-hold.test.ts` reproduces the database state at step 5. **Paperclip version or commit** `master` at |
||
|
|
25b5cad7f9 |
fix(native-runtime): start first OpenCode turns from any saved launch-file copy instead of waiting on a stalled one (#22367)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - OpenCode agents in remote sandboxes must have a saved, verified copy of their launch files before their first turn (#19622, #21287) > - The first runs of a company share one copy of these files per server. Other runs wait for it > - In a restart chaos wave, the shared copy out of one sandbox stalled for 330 s. Seven other runs waited silently for it. Two runs then failed before their first turn with the cause `unknown` > - The copy speed out of a sandbox differs from one sandbox to the next. When the first copy failed, the other sandboxes copied the same 314 MB in 15 to 24 s > - This pull request lets every waiting run use the first copy that any run saves, starts a second copy when the first one is slow, and names the cause of a stalled copy > - The benefit is a shorter cold start for the first runs of a company, no failed run when another run already saved the files, and a named cause instead of `unknown` ## Linked Issues or Issue Description Refs #19622 Refs #21287 **What happened?** A restart chaos wave started 15 OpenCode runs on new Daytona sandboxes (1 vCPU) in a new company. The first run (A) started the shared copy of the 314 MB launch files out of its sandbox. The copy did not finish. The plugin call timed out after 330 s, and run A failed before its first turn. The other runs waited for that copy without any run event, so the task looked dead for 284 to 343 s. When run A failed, every waiting run started its own copy at the same time. Five copies finished in 15 to 24 s. One copy (run B) stalled again and failed after 330 s, although a saved copy existed after 15 s. One copy (run C) took 5 min and started its turn 10 min after the run started, although it could have used the saved copy 5 min earlier. Both failures had the technical cause `unknown`. Each failure also logged "native result could not be marked workspace_failed; the reconciliation loop will retry persisted results", but the run had no result, so there was nothing to retry. **Expected behavior** A run uses the first saved copy as soon as one exists. A slow copy does not hold all first runs. A stalled copy out of the sandbox has a named cause. A run with no result does not log a result-finalization failure. **Steps to reproduce** 1. Use a company with remote OpenCode agents on Daytona sandboxes with durable storage and no saved launch-file copy. 2. Start 8 or more runs together on new sandboxes. 3. Make the copy out of the first sandbox stall (for example, a slow sandbox link). 4. See that all other runs wait for the full plugin timeout, and see the failures with the cause `unknown`. **Agent adapter(s) involved** OpenCode (native runtime, remote sandbox). ## What Changed - `shareNativeProviderArtifactCapture` (`native-provider-artifact-capture.ts`): - The first capture that saves a ready copy ends the wait for every run. A run whose own capture is still copying uses the saved copy and aborts its own copy. - A run whose own capture fails uses a saved copy when one exists, and fails only when none exists. - When the running capture is slower than 90 s, one waiting run starts one more capture from its own sandbox. At most two captures run at once. A capture keeps its slot until its copy out of the sandbox ends. - The start decision is owned by one `SharedArtifactCapture` coordinator (`tryStart()` returns a release lease). - When the shared capture fails, one waiting run starts the next capture. The other runs keep waiting for it. Before, every waiting run started its own copy at the same time. - The capture callback now gets its own abort signal, so a superseded copy stops before upload. - A failed copy out of the sandbox is now `NativeProviderArtifactExportError` with the code `native_provider_artifacts_export_timeout` (plugin call timed out) or `native_provider_artifacts_export_failed`. The run's technical cause shows this code instead of `unknown`. A timeout shows "Setting up the computer took too long ...". - `runtime-execution.ts`: do not try to mark the workspace as failed on a native result when the run has no result. Before, this always threw `native_finalization_missing` and logged a false "will retry" warning. - `doc/opencode-cold-recovery.md`: describe the new sharing, the second capture and the new cause codes. ## Verification - Red/green: with the old `native-provider-artifact-capture.ts`, these 5 new tests fail. With this change, they pass: - `native-provider-artifact-gate.test.ts`: "starts a run's turn from another run's saved artifact while its own capture is still copying", "uses a saved artifact when the run's own capture fails after another run saved one", "starts one more capture when the shared capture runs past the hedge delay, not one per waiting run", "does not start a capture per waiting run when the shared capture fails", "names a stalled copy out of the sandbox instead of an unknown cause". - Red/green: `heartbeat-native-predispatch-retry.test.ts` "continues on a fresh ... sandbox when the provider artifacts fail to save before any turn" now also asserts that no "could not be marked workspace_failed" warning is logged. It fails before the `runtime-execution.ts` change and passes after it. - `cd server && npx vitest run src/services/native-runtime/native-provider-artifact-gate.test.ts src/services/native-runtime/native-provider-artifact-capture-confinement.test.ts src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-harness-backup-storage.integration.test.ts src/services/native-runtime/native-provider-artifact-capture.test.ts src/services/heartbeat/runtime-execution.test.ts src/services/task-interruption-presentation.test.ts src/__tests__/heartbeat-native-predispatch-retry.test.ts`: all pass. - `npx tsc --noEmit -p server`: pass. ## Risks - A slow first capture can now cause one more 314 MB copy after 90 s. At most two copies run at once for one company and manifest. A run whose saved copy cannot be reused (for example, an ownership fence mismatch) starts its own copy at once. Before, a failed burst started one copy per waiting run at once, which was more. - A superseded copy never uploads, but the provider call that copies files out of the sandbox continues until it ends or times out. Its local temporary directory is removed when it ends. - Sharing is still per server. Two servers can still copy the same files. - No change to the rule from #19622: a verified saved copy still exists before the first turn. - No runner or image change. No migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file edit, test runs). Default context window; extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
24132dac16 |
fix(tasks): verify PR merges with the managed run identity (#22357)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native task completion independently verifies that declared PR deliverables are merged. > - Runs can already use an authorized managed GitHub connection to merge private PRs. > - Completion used the background credential resolver, which cannot select that run's managed identity. > - This pull request reuses operation-time identity and grant checks for completion verification. > - Agents can finish delivered work while unavailable grants and unverified merges remain blocked. ## Linked Issues or Issue Description Related: #22244 introduced the native PR merge gate. No duplicate fix was found in the open PR search. The roadmap's personal GitHub identity capability already covers the intended identity model. **What happened?** A native agent with a managed GitHub connection merges a private PR, then completion rejects it as not verified merged. The independent server-side check only tries legacy company token names or an instance token; it does not use the agent run's managed connection. **Expected behavior** Verify the merge with the run's current accepted GitHub identity, retaining current grant, membership, trust and company boundaries. GitHub must confirm the merge regardless of saved work-product metadata. **Steps to reproduce** 1. Configure a managed GitHub identity with access to a private repository and no legacy company GitHub token. 2. Use a native task run to merge and register a PR deliverable. 3. Finish the task. Before this change, the merge verifier cannot authenticate with the managed identity and rejects completion. ## What Changed - Bind completion credential lookup to the persisted company, native task and run. - Add a server-side completion credential path that shares managed grant, membership and trust checks with command credentials. Both active and succeeded runs capture identity under the existing task/run locks and reconcile pending steering acknowledgements before credential lookup. The command credential API still requires a running run. - Resolve credentials once for all PRs in a verification attempt; do no credential work for tasks without PR deliverables. - Preserve legacy fallback only for runs without a managed identity, and preserve the public-only restriction on server-environment credentials. - Add 23 PostgreSQL-backed authorization regression cases and document completion's identity behavior. ## Verification - Local checks cover revision `da2a3608b0375ead646bbbad35af853191488f07`. [Greptile review](https://github.com/paperclipai/paperclip/pull/22357#issuecomment-7023243798) is 5/5 on that revision with both prior threads resolved. [Required CI](https://github.com/paperclipai/paperclip/actions/runs/44625662403) passed on attempt 2 for the same revision, including typecheck, build, canary, server/runner suites and all eight e2e shards. - Red/green: the private-PR and identity-selection cases failed before the initial fix. Review exposed a succeeded-run reassessment failure and then an unsettled-steering gap in that path; each has a regression that failed on the reviewed commit and passes after its correction. Unresolved steering performs no token lookup, and durable acknowledgement selects the new person. The full worker test completes the task after automatic-review withdrawal while command credentials remain unavailable to the finished run. - `pnpm exec vitest run server/src/services/native-runtime/native-pull-request-completion.test.ts server/src/services/native-runtime/native-completion-feedback.test.ts server/src/__tests__/github-operation-credentials.test.ts server/src/__tests__/native-status-reassessment-work.test.ts server/src/__tests__/run-identity.test.ts server/src/__tests__/run-identity-projections.test.ts` — 222 tests pass across six files, on macOS arm64 with Node 25.6.1 and embedded PostgreSQL. - `pnpm --filter @paperclipai/server typecheck` — passes. - `pnpm verify --explain` — exits 2 because the subsystem documentation selects broad CI coverage. Required CI supplies the broader test/build coverage; this exit is not a full-suite pass. - The credential selection, steering, grant and company checks use real PostgreSQL state. Only the secret vault and GitHub HTTP transport are mocked; test requests never reach GitHub. - `git diff --check` and the staged secret/PII scan pass. No runtime artifacts are committed. - Earlier CI revision `4e42f64`: the [Build job](https://github.com/paperclipai/paperclip/actions/runs/44624163145/job/135914203042) ended after the runner received a shutdown signal and the workflow was cancelled. Its log does not show a compiler error before interruption; this interrupted attempt is not build verification. The current revision subsequently passed its own CI build on attempt 2. - Current revision canary: the [Canary Dry Run job](https://github.com/paperclipai/paperclip/actions/runs/44625662403/job/135919504493) was interrupted by runner shutdown before completion. The repository runner-loss classifier returns `lost: true` for its actual annotation and log. A targeted retry was refused while the workflow remained active. The same revision subsequently passed Canary Dry Run and the full workflow on attempt 2; the earlier interrupted attempt remains recorded here separately. - Live acceptance remains pending deployment; the change has not been deployed. ## Risks Completion now reads managed credentials through a server-side path shared with the existing authorization checks. Successful finished runs can be reassessed without granting them command execution credentials. Denied or unavailable identities must remain blocked without falling through to a different identity; regression tests cover revocation and company/task isolation. Credential lookup is shared within one bounded PR verification attempt, not cached across attempts. The existing HTTP deadline and merge proof requirements remain in place. No migrations or API contract changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning, with repository inspection, code editing and test execution. Context-window size is not exposed by the harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
cf89f0436c |
fix(native-runtime): name the closest assigned tools when a tool name is mistyped (#22327)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runners give agents their assigned app tools through two
wrapper tools: `paperclip_search_assigned_tools` and
`paperclip_call_assigned_tool`.
> - Search returns each tool with a hashed alias
(`app_<readable>_<digest>`). #22213 also accepts an unambiguous gateway
name or display name.
> - Some models copy the alias with an error: they drop or change one
digest character, change the characters of the bot id, or attach the
digest of a different tool.
> - The wrapper correctly rejects these names, because a guess could
select a write tool when the agent meant a read tool.
> - But the error only said "name must be an exact name". The agent
searched again and often repeated the same mistake.
> - This pull request adds the closest names that the agent can use to
the error. It does not resolve any name by approximation.
> - The benefit is a faster recovery after a mistyped name, with no
change to authorization.
## Linked Issues or Issue Description
Refs #22213
**What happened?**
An agent searched its assigned tools and then called
`paperclip_call_assigned_tool` with a name that is almost the returned
alias. For example, one digest character was missing, or the agent used
the digest of the read tool with the name of the submit tool. The call
failed with `assigned_mcp_tool_unknown: name must be an exact name
returned by paperclip_search_assigned_tools. Search assigned tools again
and use the returned name.` In observed runs, the agent made more failed
calls with other wrong names.
**Expected behavior**
The call is still rejected. The error also shows the closest assigned
names that the agent can use, so that the agent can correct the name
directly.
**Paperclip version or commit**
`master` at
|
||
|
|
806440d12c |
feat(runner): relaunch a released task as the next run after a restart (#22343)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An environment plugin can keep a warm task for an agent session. After each turn, Core releases the task's Runner, and the next run of the session resumes it > - #22288 added `resumeIdentityVersion: 1`: for a provider that stops a released Runner, Core rotates the session to the next run and the provider starts the Runner again directly as that run > - Core takes that path only while it still holds the released session in memory. After a Core restart, it recovers the released task through the older hand-off: it resumes the task as the previous run, then switches the Runner's authority to the new run > - So a provider that advertises the capability still has to keep the hand-off for every follow-up after a Core restart, and cannot drop it > - This pull request makes restart recovery take the same path when the provider advertises the capability > - The benefit is that a provider with the capability gets one start shape for a released task, before and after a Core restart, and can retire the hand-off ## Linked Issues or Issue Description Follows #22288. **What existing behavior does this improve?** The first run of a warm plugin session after a Core restart, when its task was released and its provider advertises `resumeIdentityVersion: 1`. **Subsystem affected** The native session executor and the plugin Runner task owner. **Current behavior** After a restart, Core finds the released source on disk (`inspectRemoteWarmSessionRecovery`), creates a backend that adopts the task, and calls `recoverSession`. The task owner calls `resume` without `runner`, the provider starts the Runner as the previous run, and Core hands it to the new run. **Proposed behavior** When the provider advertises the capability, Core recorded the build the task was released on, the process configuration is unchanged, and the session is not moving to another task, Core treats the recovery like a cached released session. It revalidates the recovery, loads the warm checkpoint, and opens a new backend on the same task with `relaunchReleased`. A new task owner takes over the submitted task without adopting a Runner process, prepares the claimant run with the released source as `priorIdentity`, and calls `resume` with `runner`. Every other recovery keeps the hand-off. **Reason and benefit** A provider that advertises the capability no longer needs the hand-off for its own tasks, after a restart or not. ## What Changed - Executor: a remote warm recovery with a released source, a recorded build, a reusable process configuration and no session move, on a provider that advertises the capability, revalidates the recovery and becomes a relaunch: `relaunchReleased` with the released source and build, the warm checkpoint, and no `remoteWarmRecovery` for the backend. The startup context ends as for a cached relaunch. - Executor: the relaunch's task owner can now come from the new backend (`onPluginTaskOwner`), so the unavailable-reuse fallback and the rotation revert work without a retained owner. Without an owner, no relaunch can have started, so the revert applies. - Backend: a relaunch neither stops a resumed task that has no durable binding nor resolves a `resumeFrom` task. - Task owner: an owner created with `relaunch` and the capability takes over an already submitted task, as an adopting owner does, but adopts no process: its prepare sends the released source as `priorIdentity`, and its launcher calls `resume` with `runner`. - `doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md`: Core prepares the claimant run before the resume, also after a restart, and a provider must accept that prepare without state kept in memory. ## Verification - `server`: `npx vitest run src/services/native-runtime/plugin-runner-release-resume.integration.test.ts`: new `adopted restart relaunch` cases for Codex and Claude, with real runnerd processes. Core restarts while the task is released; the next run relaunches the task as itself, with no claim, one inspection and no second submit. The existing `adopted relaunch` case now relaunches both follow-ups, since the idle close also recovers through this path. - `server`: `plugin-runner-task.test.ts`: a new owner relaunches without submitting or adopting, and prepares with the released source; without the capability it prepares a new task. - `server`: `plugin-runner-release-resume.integration.test.ts` and `plugin-runner-eviction.integration.test.ts`: 61 passed. `native-session-executor.test.ts`: 1535 passed, 7 skipped. `plugin-runner-task.test.ts`: 118 passed. - Typecheck: `server` passes. - Red-green: with the executor lane turned off, both `adopted restart relaunch` cases fail. Without the task owner change, the new-owner relaunch test fails. ## Risks - Only providers that advertise `resumeIdentityVersion: 1` see a change. Recovery without a recorded build, with a changed process configuration, or for a session moving to another task keeps the hand-off. - The provider sees a `prepare` for the claimant run on a released task right after a Core restart. A provider that tracked unclaimed released tasks only in memory must handle that prepare (the doc now says so). - A Core restart in the middle of this path is still not qualified, as for the cached relaunch. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code with tool use (code search, editing, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
575ef12575 |
feat(responsibilities): file company skills under responsibilities (#22275)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A responsibility shows the skills of the agents doing its work. People cannot say which skills a responsibility needs. > - The maintainer chose to file skills under responsibilities like projects, tasks and routines. > - Agents also need to set a skill's catalog details without rewriting its SKILL.md. > - This pull request lets people and agents file, move and remove company skills under a responsibility, and adds `update_skill_details`. > - The benefit is that a responsibility lists the skills it needs, beside the skills its agents already hold. ## Linked Issues or Issue Description Refs #22253, #22261 and #22274; this pull request is stacked on them. **Subsystem affected** Database (`responsibility_links`), shared types and validators, server responsibility service and routes, Dashboard versions, native runner tools, and the responsibility page UI. **Problem or motivation** Skills under a responsibility are only computed from the agents doing its work. A person cannot record that "Hiring needs interviewing", and an agent cannot file a skill when it maps work for a person. **Proposed solution** Add a nullable `skill_id` to `responsibility_links` and allow it as the one target of a link. Show filed and computed skills together. Let the file, unfile, lookup and link routes and tools take skills. Keep filed skills in Dashboard versions so undo covers them. Add a small live tool for a skill's tagline and categories. **Alternatives considered** Keeping skills computed only, and giving the skill to the agent that runs the responsibility, was the other option. It was not chosen. Extending `update_skill` would change its scenario contract, whose mock needs `markdown`, so the details tool is separate. ## What Changed - Review fixes: the default responsibility view's Skills "+" adds an existing skill, removes a filed one, or opens an agent's skills page. A newest Dashboard version without a skill list is upgraded in place, so the first skill filed after it is undone. Undo files again, or removes, only skills the restoring person can see. A read scope without a skill condition sees no filed skill. `update_skill_details` with only `skillId` is a no-op, as its schema allows. - Migration `0499`: add `responsibility_links.skill_id` (FK to `company_skills`, on delete cascade). Widen the single-target check to four targets. Add a unique `(responsibility_id, skill_id)` index and a `skill_id` index. The migration is idempotent and online-compatible (NOT VALID then VALIDATE, and the indexes are built CONCURRENTLY). It is listed in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`. - Shared: `ResponsibilityLink.skillId`; `ResponsibilitySkillSummary.linkId` and `computed`; `skillId` in the link and lookup schemas; `skill` in `RESPONSIBILITY_FILE_ITEM_KINDS`; optional `skillIds` in Dashboard snapshot links. - Server: filing checks the reader's skill visibility (`skillAccessService`). The detail merges filed and computed skills into one card each. Lookup accepts `skillId`. Dashboard capture, restore and summary include filed skills. A version without `skillIds` leaves filed skills as they are when it is restored. - Tools: `file_under_responsibility`, `remove_from_responsibility` and `find_responsibilities_for` accept `kind: "skill"`. `get_responsibility` returns `linkId` and `computed` for each skill. Add `update_skill_details` (tagline and categories). - UI: the responsibility page has "Add existing skill". A filed skill shows "Filed here" and a remove button. Computed skills keep their holders. ## Verification - Server: the `responsibilit*` tests (new: file, find, move and unfile a skill; one card for a skill that is both filed and computed; undo of a filed skill from the Dashboard history), `skill-tools.test.ts` (new `update_skill_details` cases), `responsibility-tools`, `routine-tools`, `routines-routes`, `paperclip-runner-tool-authority`, `runner-api` and `openapi-routes`: 24 files, 2723 tests passed. - UI: responsibility page and component tests (16 files, 158 tests, including a new filed-skill test) pass. `pnpm --filter @paperclipai/ui typecheck` passed. - `@paperclipai/db` `check:migrations` passes and reports the override. The snapshot drift test passes. `tsc` passes for `db`, `shared` and the server layers. - Runner: catalog and semantic-tool tests (317) pass. All generator checks pass. ## Risks - The migration changes a check constraint on `responsibility_links`. The replacement accepts every row that the old check accepted. It is swapped as NOT VALID and then validated. The previous release never sets `skill_id`. - Deleting a company skill removes its links (cascade). - Filed skills follow the reader's skill visibility. A skill the reader may not see is not listed. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b7319eb581 |
feat(routines): let agents list and change their person's routines in plain terms (#22274)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines are recurring work that an agent runs on a schedule, webhook or request. People file them under their responsibilities. > - #22253 and #22261 let agents read and change areas and responsibilities. Agents still have no dedicated routine tools, and they can change only routines assigned to themselves. > - When a person says "move my weekly digest to Monday mornings" or "map my routines to my responsibilities", the agent must use the generic API, write cron, and it is refused for the person's routines. > - This pull request adds `list_routines` and `update_routine`. It also lets an agent change routines that the person its run works for is responsible for. > - The benefit is that an agent can find routines under no responsibility, file them, and change their schedule in plain terms. Each change is a routine revision the person can restore. ## Linked Issues or Issue Description Refs #22261 (phase 2; this pull request is stacked on it), #22253 (phase 1). **Subsystem affected** Server (routine routes, native runner tool authority), the native runner tool catalog, and the `paperclip` skill. **Problem or motivation** An agent can list and change only its own routines through the generic API. It cannot change the schedule of a routine its person is responsible for. It must write cron expressions, and it cannot find the routines that are filed under no responsibility. **Proposed solution** Make the existing scenario operation `list_routines` live, with filters for words, status, agent, project, responsible person, a responsibility and "unfiled". Add `update_routine` for the title, instructions, agent, project, status, priority, a plain `schedule`, and filing under a responsibility. Let an agent manage a routine when the person its run works for is responsible for it. Handing the routine to another agent needs that person to have `tasks:assign`. **Alternatives considered** A new `routines` permission grant would add a setting with no current need. Routine revisions already give an undo path, so this change adds no new history. ## What Changed - `list_routines` is now live (it was a scenario-only mock) and keeps its scenario mock. - Add `update_routine`. A `schedule` such as `{ frequency: "weekly", dayOfWeek: "monday", time: "09:00", timezone }` maps to the routine's schedule trigger through the existing routine-offer cadence helpers. `schedule: null` turns the schedule off. `fileUnder` files the routine under a responsibility. - Add `server/src/services/routine-tools.ts`. It calls the routine routes with the run token. Fields and schedule go in one `PATCH /routines/:id` (one transaction and revision, guarded by `baseRevisionId`); filing follows, and if it fails after the routine changed the tool reports what was saved. Each part reports its own undo: the routine revision, or the Dashboard history for filing. - `PATCH /routines/:id` takes `schedule` (`{ cronExpression, timezone? }` or `null`) and replaces the schedule under the routine row lock, so racing changes leave one schedule. Without a zone it keeps the current schedule's, else uses the person's saved time zone, else answers 422. - `GET /companies/:companyId/routines` filters `q`, `status`, `assigneeAgentId`, `responsibleUserId`, `responsibilityId` and `unfiled=true` in SQL. - In `routes/routines.ts`, an agent may manage routines assigned to itself or routines its person is responsible for. On a routine the agent does not run, taking it from another agent (or handing it on), turning it on, changing its schedule (also through the trigger routes), starting a run, or restoring a revision requires that person's `tasks:assign`, as for a person. The person must have directed the run (see #22261). `routine.updated` activity records `onBehalfOfUserId`. - The routine and responsibility routes share one rule, `agentPersonInCompany` in `routes/agent-person-authz.ts`. - The catalog keeps each live action's `requiredClaims` (`list_routines` is `routines:read`). - The routine tools stay visible when Responsibilities is off. Only `responsibilityId`, `unfiled` and `fileUnder` need it. - Update `search_api` tool pointers, the routines reference, the skill section, the chat directive, the operation-groups spec and the pinned counts. ## Verification - Server: `routine-tools.test.ts` (new), `routines-routes.test.ts` (new: an agent changes its person's routine, hands it off only with `tasks:assign`, and is refused for another person's routine), the `responsibilit*` tests, `paperclip-runner-tool-authority` and `runner-api`: all pass. The first test in `routines-routes.test.ts` (Fireflies) times out at 15 s in my sandbox on a cold import, with or without this change. It passes with `--testTimeout=60000`. - Server `node scripts/typecheck-layers.mjs`: passed. - Runner: catalog, semantic-tool, eval, scenario and permission-policy tests (316) pass. All generator checks pass. ## Risks - Medium-low. Agents get write access to their person's routines. The route still checks the company, the routine revision guard and `tasks:assign` for a hand-off. The person can restore any earlier routine revision. - `list_routines` changes from a scenario-only mock to a live tool, so the reconciled catalog counts change. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
c0aa7751f8 |
feat(responsibilities): let agents change areas and responsibilities for their person, with undo (#22261)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People keep their areas and responsibilities in Paperclip (experimental, `enableResponsibilities`). #22253 lets agents read them. > - Agents can still only propose. They cannot rename or archive an area, change an active responsibility, let one go, or move work between responsibilities in one step. > - A person who says "file my weekly routines under Sales" must then do it by hand. > - The maintainer chose this rule: an agent may make each change that its person could make, the person can undo it from the Dashboard history, and a permanent delete still needs the person's yes on a card. > - This pull request adds eight change tools and the server rules behind them. > - The benefit is that an agent maps work to responsibilities for the person, and the person keeps control through undo and the delete card. ## Linked Issues or Issue Description Refs #22253 (phase 1; this pull request is stacked on it). **Subsystem affected** Server (responsibility and area routes, Dashboard versions, native runner tool authority), shared validators and Dashboard types, the native runner tool catalog, the `paperclip` skill, and the Dashboard history UI. **Problem or motivation** An agent that works for a person can only propose areas and responsibilities. It cannot change active ones, archive or delete them, or move filed work in one step. Unfiling needs link IDs. There is no way for the person to undo an agent's change. **Proposed solution** When an agent acts for the person its run works for (`onBehalfOfUserId`) and that person directed the run (the run's own task is one they made, such as their chat, or their own message woke it; parent tasks do not count), apply that person's own rights: their own responsibilities, or anyone's for a company owner or admin; areas for any active member. Save each such change as a version of the person's Dashboard, so the existing history and Undo line show it under the agent's name. Make a permanent delete wait for an accepted confirmation card that the agent posted. Add `file` and `unfile` routes that take items by kind and ID, with an optional move. **Alternatives considered** One card for every change, or propose-only. The maintainer chose "act for me, with undo". A separate undo journal would need a new table and a new UI; Dashboard versions already have both. A new version cause needs a check-constraint migration; this change uses `change_set` with no `interactionId` instead. ## What Changed - Add tools `create_area`, `update_area`, `delete_area`, `create_responsibility`, `update_responsibility`, `delete_responsibility`, `file_under_responsibility` and `remove_from_responsibility`. They are standard-mode only and hidden while Responsibilities is off. - Add the writer rule for agents in `routes/agent-person-authz.ts` (`agentPersonInCompany`, `agentActsForResponsibilityOwner`), used by `routes/responsibilities.ts` and `routes/responsibility-areas.ts`. The person must have directed the run: a person can name anyone as a task's responsible person, and a run can also work for a routine's person or the company default, so those runs do not borrow that person's rights. An agent with no such person keeps the old propose-only rules. - The response names only the version of the responsibility's owner (for area changes, the acting person's), and `X-Paperclip-Dashboard-Owner` says whose history holds it; tools return `dashboardOwnerUserId`. A change Undo cannot reverse (the name, description, order or un-archiving of an area other people's proposed or active responsibilities share) saves no version and is not reported as undoable. - `/file` without `moveFrom` follows the `/links` rule, so `create_responsibility` with `fileItems` works for a proposal. - A title-only create is allowed for an acting agent. It is active by default and records the agent in `proposedByAgentId`. - Add `responsibility-agent-changes.ts`. In one transaction that holds each affected owner's Dashboard lock, it saves the dashboard before the change (when a person changed it by hand), runs the change on services bound to that transaction, and saves the result as a `change_set` version by that agent. The response names the version in `X-Paperclip-Dashboard-Version`. The tools say Undo only when a version was saved. - Dashboard snapshots now also hold each responsibility's filed items, description, "run by" agent, evidence, health override and note, and each area's description (optional fields). Restore puts them back. Older versions still restore as before. A matching older version is upgraded in place, so the first change after it compares and undoes in full. - Create and update accept `areaTitle`. The server finds or makes that area in the same transaction, so a refused change leaves no area. Agent creates reuse a live responsibility with the same title under a lock, so retries and parallel calls make no copies. - Add `POST /api/responsibilities/:id/file` (`items`, optional `moveFrom`: an ID or `all`) and `POST /api/responsibilities/:id/unfile` (`items`), each in one transaction. - Agent deletes need `confirmationId`: an accepted `request_confirmation` card that this agent posted, with target key `responsibility-delete:<id>` or `responsibility-area-delete:<id>`. Someone who may manage the owner must accept it, on the card or by a verified chat reply. The tools post the card (named after the item), return `awaiting_confirmation`, and delete on the next call after acceptance. - `DELETE /api/responsibility-areas/:id` accepts `moveResponsibilitiesTo`. Area changes and deletes save versions for everyone whose Dashboard shows the area. - Agent activity details now include `onBehalfOfUserId`. The Dashboard history shows "<agent>, for you" for these versions. `ResponsibilityDashboardVersionSummary` now has `interactionId`. - Update the OpenAPI registry and metadata, the `search_api` tool pointers, the skill, the API reference, the chat directive, the operation-groups spec, and the pinned counts and SDK source pins. ## Verification - Server: `vitest run` of the `responsibilit*` tests, `responsibility-tools.test.ts`, `openapi-routes`, `runner-api*`, `paperclip-runner-tool-authority` and `task-markdown` all pass. New tests cover these cases: acting-agent create, update and undo; file, move and undo; the rule for a person who may not manage the owner; delete only after an accepted card, including a verified chat reply; area delete with move and a version for the other owner; undo of a health override, an area description and an un-archived empty area; upgrade of an older version; an area by title that a refused change rolls back; parallel creates. - UI: `DashboardEditModule.test.tsx` and `responsibility-dashboard.test.ts` passed. `pnpm --filter @paperclipai/ui typecheck` passed. - Server `node scripts/typecheck-layers.mjs` passed. - Runner: catalog, semantic-tool, eval, protocol-action and permission-policy tests (284) passed. All generator checks pass. - I did not run the browser e2e suite locally. ## Risks - Medium. Agents that work for a person can now change and archive that person's responsibilities and areas without a card. Every change is a Dashboard version that the person can restore, and a delete still needs the card. - Undo restores titles, sentences, areas, order, status, filed items, description and the "run by" agent. It does not restore a bar or metric change; those stay in the metric history. - A restore of a version from before this change does not touch filed items, which matches the earlier behavior. - Undo of a rename or archive of an area that other people also use keeps its name, as the existing restore rule does for shared areas. - Manual edits do not take the Dashboard lock. A person's edit that commits while an agent's change is open can appear in that agent's version. - Changes to proposed or let-go responsibilities are not on a Dashboard, so they cannot be undone there. The tools say so. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ffcd49f6aa |
feat(responsibilities): let agents read a person's areas and responsibilities (#22253)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People keep their areas and responsibilities in Paperclip (experimental, `enableResponsibilities`). Agents chat with them about this work. > - Agents have no dedicated tool to read a person's responsibilities. The only path is the generic `search_api` and `call_api` fallback. > - That path needs the person's user ID, and nothing in a chat tells the agent that responsibilities exist. When the feature is off, every route returns a 404 that looks like "nothing there". > - So a chief-of-staff agent asked the person "what are your responsibilities, or where are they written down?" when Paperclip already had them. > - This pull request adds three read tools, makes the owner default to the person the run works for, and tells agents in chats how to use the tools or that the feature is off. > - The benefit is that an agent reads and maps a person's responsibilities itself. This is phase 1 of 4. Later pull requests add the change tools for areas, responsibilities, routines and skills. ## Linked Issues or Issue Description **Subsystem affected** Server (responsibility routes, native runner tool authority, chat prompt), native runner tool catalog, and the `paperclip` skill. **Problem or motivation** An agent in a chat cannot find a person's responsibilities. It has no dedicated tool. The list route requires `ownerUserId` from an agent, and the chat does not give the agent that ID. The chat prompt does not mention responsibilities. When the experiment is off, the 404 does not tell the agent that the feature is off. The agent then asks the person to list their responsibilities again. **Proposed solution** Add three optional native tools: `list_responsibilities`, `get_responsibility` and `find_responsibilities_for`. They call the existing routes with the run's agent token, so all route checks and the activity log stay the same. The owner defaults to the person the run works for. The routes return a `responsibilities_disabled` code while the feature is off. The tools are hidden while the feature is off, and the chat prompt then says that the feature is off. **Alternatives considered** Better `search_api` guidance alone still needs the user ID and many calls. A new read service in the tool authority would copy the route permission rules. The tools use the routes instead. ## What Changed - Add `list_responsibilities` (areas with their responsibilities, responsibilities with no area, totals, and company work filed under no responsibility), `get_responsibility` (one responsibility with its filed items, link IDs, skills and latest status report), and `find_responsibilities_for` (where a project, task or routine is filed; a task can be a `PAP-12` identifier). - Add `server/src/services/responsibility-tools.ts`. It calls the routes over HTTP with the run token and returns compact results with page links. - When an agent sends no `ownerUserId`, the responsibility list, create, summary, automation and dashboard routes use the person the run works for (`onBehalfOfUserId`). A run that works for no person still gets a 422 with a clear message. - The nine "not enabled" checks now throw one shared error. The 404 and its message do not change. The body now also has `code: "responsibilities_disabled"` and a remediation line. One `RESPONSIBILITIES_OFF_GUIDANCE` text is the remediation, the tool error and the chat prompt note. - `list_responsibilities` with `areaId` counts only that area. Archived areas are listed only on request. When only proposals are hidden, the tool says so. - `services/paperclip-api-loopback.ts` holds the tools' authenticated API call; the project tools use it too. - Show the tools only while the instance has Responsibilities on. A hidden tool that is called by name returns "Areas and responsibilities are turned off". - In standing chats, add an "Areas and responsibilities" prompt block: use the tools before you ask the person, or say that the feature is off. - `search_api` results for these routes now name the dedicated tools. - Add an "Areas and responsibilities" section to `skills/paperclip/SKILL.md` and to `references/api-reference.md`. Add one line to the chief-of-staff persona. - Register the tools in the runner catalogs. Regenerate the semantic contracts, the action catalog, protocol coverage, the operation-groups spec and the capability inventory. Update the pinned counts. ## Verification - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/catalog src/semantic-tools src/tools src/eval/workflow-evals.test.ts src/protocol-actions src/evals/native-execution.test.ts src/drivers/acpx/permission-policy.test.ts`: 276 passed. - All runner generator checks pass: protocol manifest, capability contract, capability inventory, protocol coverage, semantic contracts, semantic action catalog, operation groups, replay goldens, workflow traceability. - Server: `vitest run` of `responsibilit*` route tests, `responsibility-tools.test.ts` (new), `task-markdown.test.ts`, `task-context*`, `paperclip-runner-tool-authority.test.ts`, `runner-api.test.ts`, onboarding first-task tests and `user-facing-wording-guidance.test.ts`: 20 files, 2620 tests passed. - `node scripts/typecheck-layers.mjs` for the server: passed. - New tests cover these items: the owner default for an agent and the 422 without a person, the `responsibilities_disabled` code on every route, the tool projections, an identifier lookup, the "turned off" mapping, the tool visibility switch, and the chat prompt block. ## Risks - Low risk. The routes keep their permission checks. Agents can still only read and propose in this phase. - An agent that sends no `ownerUserId` now gets the responsible person's list instead of a 422. Agents that name an owner see no change. - The chat prompt gets one more short block in standing chats. - The flag is pinned when a turn opens. If an admin changes it during a turn, the tool list changes on the next turn. The routes always check the current setting. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1c67687998 | feat(runner): start a released Runner again as the next run (#22288) | ||
|
|
440f96247b |
fix(cloud): resolve or name the remaining online move blockers (#22294)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A hosted server can move its durable data to shared storage while it serves work, and then run without a volume > - A bounded online pass moves the data, and a readiness check names what still blocks the move > - Some servers stay blocked after many passes, and the pass reports only `skills:blocked` or `native_harness:source_not_successful`, or reports nothing for run logs > - Operators cannot see why, so they cannot tell recoverable data from data that needs a maintenance step > - This pull request recovers the cases that are safe to recover online and gives every other case a precise code > - The benefit is fewer blocked moves and a clear next step for each one that stays blocked ## Linked Issues or Issue Description Refs #19719 **What happened?** A serving server that runs the online durable-data pass many times stays ineligible. The pass reports `skills:blocked` without a reason, `native_harness:source_not_successful` for several different causes, and no reason at all for run logs that stay on the local disk. **Expected behavior** The pass moves every item that it can move safely while the server serves work. For each item that stays, the pass reports a code that names the cause. **Steps to reproduce** 1. Install a catalog skill on a volume-backed server. Remove its installed files and do not keep a saved version. 2. Keep a settled native run whose local log file has no final size or digest and whose events do not prove full coverage. 3. Run the online durable-data pass. 4. See that the skill is reported only as `skills:blocked` and the run log stays local with no reason in the pass result. **Additional context** The settled native log archive path from #19719 had no online caller. ## What Changed - Skill installation conversion returns a `code` for each blocked installation (for example `source_files_unavailable`, `saved_version_incomplete`, `catalog_origin_unavailable`, `legacy_source_unsafe`, `source_changed`). The pass reports `skills:<code>`. An unknown cause stays `skills:blocked`. - A spent budget during conversion now stops the page. Before, it was reported as a blocked skill and the page moved past it. - A catalog skill whose files are gone converts from the bundled catalog package only when the catalog content hash equals the recorded origin hash and the last audit recorded the installed bytes as equal to that origin with no local changes. - A modified catalog skill whose reset origin is gone gets its origin version from the bundled catalog package only when the catalog content hash equals the recorded origin hash. - Native harness pre-copy replaces `source_not_successful` with `source_run_active`, `source_run_unfinished`, `source_coordinator_missing`, `source_coordinator_unsettled`, `source_session_not_selected` and `source_lease_missing`. The settled-source rules do not change. - The run evidence worker archives a settled native log by run id when the canonical import cannot prove event coverage. The existing settled-source guard still decides. The worker records the reasons from its last completed sweep, and the pass reports them as `run_logs:<code>` and `provider_traces:<code>`. - `migrateRunEvidence` accepts a `runId` filter. ## Verification - `cd server && npx vitest run src/__tests__/managed-skill-content.test.ts` (embedded PostgreSQL): new catalog recovery, hash mismatch, unproven install and budget abort tests fail before the change and pass after it. - `npx vitest run src/services/native-runtime/native-harness-online-preservation.integration.test.ts src/services/native-runtime/native-harness-backup-storage.integration.test.ts`: the new reason tests fail before the change and pass after it. - `npx vitest run src/__tests__/run-evidence-migration-work.test.ts src/services/native-log-archive.integration.test.ts`: the worker archive test fails before the change and passes after it. The integration test shows a real settled source that is blocked as `native_log_source_coverage_unproven` and then archived by run id. - `npx vitest run src/services/online-durable-data-migration.test.ts src/services/online-durable-data-migration-postgres.test.ts` - `pnpm --filter @paperclipai/server typecheck` ## Risks - A settled native log with unproven coverage now moves to the read-only historical archive instead of staying local. The archive keeps the exact bytes. It does not claim canonical event coverage. This is the same result that the offline path gives. - The catalog fallback reads bundled or pinned catalog bytes. Each file is checked against its pinned SHA-256, and the package must match the recorded origin hash. - Pass deferral codes change. Readers that match `native_harness:source_not_successful` must use the new codes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
170e1b0c07 |
feat(runner): pin Codex 0.170.0 and Agent SDK 0.3.328 so Fable 5.2 and GPT-6.2 run (#22278)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs Claude and Codex agents from one shared provider pack. The pack pins an exact Claude Agent SDK (which bundles Claude Code) and an exact Codex CLI. > - #22222 added Claude Fable 5.2 (minimum Claude Code 2.1.321) and GPT-6.2 Sol and Astra (minimum Codex 0.169.1). The pack still bundles Claude Code 2.1.286 and Codex 0.160.0. > - Because of this, Paperclip Runner stops these models with a named version error, and native remote Codex accepts only `>=0.149.0 <0.161.0`. > - This pull request moves the provider pack to Agent SDK 0.3.328 (Claude Code 2.1.328) and Codex 0.170.0. It moves every constant that depends on these pins, and the remote Codex window to `<0.171.0`. > - The benefit is that Fable 5.2 and GPT-6.2 run on Paperclip Runner after a sandbox image with this pack is promoted. ## Linked Issues or Issue Description Refs #22222 (adds the models and the version floors; this pull request makes them runnable on Paperclip Runner). Refs #18488 (moves only the Codex pin to 0.166.0 and the window to `<0.167.0`; it is not updated since November 1 and has conflicts with `master`). Refs #18486 (GPT-6.1 Terra needs Codex 0.165.0; Codex 0.170.0 satisfies it). **Overlap with #18488.** #18488 changes the same Codex pin locations. This pull request uses a newer pin (0.170.0) and also moves the Claude runtime. If this pull request merges first, #18488 has no remaining change. I did not change the #18488 branch. ## What Changed - Claude runtime: `@anthropic-ai/claude-agent-sdk` 0.3.286 → 0.3.328 (Claude Code 2.1.286 → 2.1.328). This changes the root `pnpm.overrides`, `pnpm-workspace.yaml`, the ACPX `claude` profile, and the qualified Claude runtime in `installation-integrity.ts`, with new executable digests for Linux x64, Linux ARM64, macOS ARM64, and macOS x64. - Codex runtime: `@openai/codex` 0.160.0 → 0.170.0. This changes the runner dependency, the `codex-acp@1.6.2>@openai/codex` override, the `codex-acp` patch dependency line, the ACPX `codex` profile, and the qualified Linux x64 runtime with its new executable digest. - The ACP bridges do not change: `claude-agent-acp` 0.73.0 and the patched `codex-acp` 1.6.2. Their `commandDigest` values and profile versions do not change. - Generated declarations: `generated-profiles.ts` and `generated_acpx_profiles.rs` are regenerated with `generate:acpx-profiles`. - Remote native Codex window: `REMOTE_CODEX_MAXIMUM_VERSION_EXCLUSIVE` 0.161.0 → 0.171.0. The minimum stays at 0.149.0. Images with Codex 0.149.0 through 0.170.x are accepted. - Codex company observability: the qualified-version check moves to `codex-cli 0.170.0`. The `otel` configuration keys that Paperclip writes (`exporter`, `trace_exporter`, `metrics_exporter`) are present in the 0.170.0 configuration schema. The `OtelExporterKind`, `OtelHttpProtocol`, and `OtelTlsConfig` definitions are identical to 0.160.0. - Daytona runner image probes (`Dockerfile`, `Dockerfile.release`) expect `codex-cli 0.170.0` and `2.1.328 (Claude Code)`. The remote npm install spec in `server/src/adapters/registry.ts` is `@openai/codex@0.170.0`. - Tests: fixtures and assertions that mirror the pins use the new versions. The window tests accept 0.161.0 through 0.170.99 and reject 0.171.0. The ACPX gate tests now show that Fable 5.2 and GPT-6.2 are admitted. A new test mocks an older bundled runtime and keeps the named version-gap errors covered. The fallback test shows that GPT-6.2 stays on 0.170.0 and still falls back on 0.160.0. - Opt-in qualification scripts (`scripts/tests/native-cleanup-paginated-codex.mjs`, `native-provider-startup-codex.mjs`) require Codex 0.170.0. - Documentation: `docker/daytona-runner/README.md`, `doc/architecture/paperclip-runner-compatibility.md`, `doc/agent-observability.md`, `docs/deploy/environment-variables.md`, and the deferred item in `doc/adapter-model-audit-2026-11-18.md`. - `pnpm-lock.yaml` is not part of this pull request. The refresh bot regenerates it on `master`, as for the previous pin refreshes (#14942, #14993). ## Verification Sources, checked on 2026-11-19: - npm registry: `@openai/codex@0.170.0` (published 2026-11-18T03:45Z) and `@anthropic-ai/claude-agent-sdk@0.3.328` (published 2026-11-18T23:13Z, `claudeCodeVersion` 2.1.328). Both are the `latest` tags. - [Codex releases](https://github.com/openai/codex/releases) 0.161.0 through 0.170.0. The notes record no app-server protocol break. 0.169.0 keeps project configuration disabled for `codex exec` in untrusted directories; Paperclip's CLI path already uses `--skip-git-repo-check`, and the native path uses `app-server`. - [Claude Code changelog](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md) 2.1.287 through 2.1.328. See "Risks" for the headless changes that I reviewed. Digests and package shape: - I downloaded each platform archive with `npm pack`, which verifies registry integrity, and computed the SHA-256 of the executable. As a method check, the same command on the 0.3.286 Linux x64 archive gives the digest that is on `master` (`fe503f65…`). The Codex 0.160.0 archive also gives the `master` digest (`12eb3e81…`). - The extracted executables report `codex-cli 0.170.0` and `2.1.328 (Claude Code)`. - The Codex 0.170.0 Linux x64 archive no longer contains `codex-resources/zsh/bin/zsh`. Paperclip does not reference that path. - The Agent SDK 0.3.328 `optionalDependencies` have the same platform set as 0.3.286. The Codex 0.170.0 `optionalDependencies` use the same `npm:@openai/codex@<version>-<platform>` aliases. - Bridge link check: I installed `@agentclientprotocol/claude-agent-acp@0.73.0` with an override to Agent SDK 0.3.328 and imported its entry module. Node linked all of the named SDK imports, and the import completed. Real Codex qualification (no model turn, no credentials): - `PAPERCLIP_TEST_CODEX_BINARY=<installed 0.170.0 Linux x64 binary> node --import ./server/node_modules/tsx/dist/loader.mjs scripts/tests/native-cleanup-paginated-codex.mjs` passed: `green: true`, with the expected red control and paginated `thread/resume` through the real 0.170.0 `app-server`. The installed binary digest matches the pinned digest. Focused suites (Linux x64, Node 25.9.0, local `pnpm install --no-frozen-lockfile`): - `cd packages/paperclip-runner && pnpm run typecheck:typescript` passed, including `check:acpx-profiles`. - `npx tsc --noEmit` passed in `packages/adapters/codex-local` and `packages/adapters/claude-local`. - `packages/adapters/codex-local`: 625 passed. `packages/adapters/claude-local`: 419 passed, 1 skipped. - `node --test`: `acpx-codex-package-contract` 9 passed, `controlled-evals/controller` 77 passed, `scripts/acpx-patch-packaging` 26 passed, `scripts/__tests__/grok-public-install-sandbox` 27 passed. - `server`: `acpx-qualification`, `codex-runtime-compatibility`, and `codex-model-fallback` 110 passed. `native-session-executor` 1541 passed, 1 failed. The failure (`compares export process compatibility … db.transaction is not a function`) also fails on unchanged `master` on this host. - `packages/paperclip-runner` vitest (`installation-integrity`, `qualified-profiles`, `codex-security-config`, `native-execution`, `native-backend-factory`): 283 passed, 25 failed. All 25 failures are in `installation-integrity.test.ts` with "ACPX provider lifetime guardian failed to start". The same 25 tests fail on unchanged `master` on this host (master fails 30 here). Rust (toolchain 1.97.1 from `rust-toolchain.toml`, Linux x64): - `cargo test -p paperclip-runner-core --lib`: 658 passed, 2 failed, 4 ignored. `warm_run_attachment_reuses_the_provider_ingress_listener` also fails on unchanged `master` on this host. `reaping_stops_a_dead_runners_provider_and_its_escaped_tool_job` is flaky here: on this branch it passed 2 of 3 reruns. - `cargo test -p paperclip-runner-core --test acpx_warm_attachment --test native_provider_backend`: 7 and 45 passed. - The new and changed saved-session tests pass: `a_session_saved_by_the_previous_agent_runtime_is_refused_by_name`, `unrestorable_saved_sessions_keep_strict_bindings_and_name_their_class`, and `a_settled_session_saved_by_the_previous_acpx_release_continues`. `rustfmt --check` passes. - `server`: `native-session-resume.test.ts` and `acpx-qualification.test.ts` 110 passed, 2 skipped. Both new resume cases run and pass. Not run here: - `native-provider-startup-codex.mjs`, because it needs a compiled `paperclip-runnerd`. The trusted CI runner workflow covers startup and session-resume probes. - The Docker image build and any live model turn. A live Fable 5.2 and GPT-6.2 test follows the staging image rollout. ## Saved sessions from the previous release This change keeps the ACPX bridge command digests and profile versions. It changes `agentRuntimeVersion` in the `claude` and `codex` ACPX profiles. That field is part of the saved session identity. The previous release saved sessions with Agent SDK 0.3.286 or Codex 0.160.0. This build refuses those saved sessions. It does not migrate them. - **Next run (the usual case):** the saved provider descriptor names the old runtime. `rebindNativeSessionCheckpoint` does not match it, so the next run starts a fresh provider session in the same workspace with the complete handoff prompt. No run fails. The earlier provider conversation does not carry over. - **A run in flight at the deploy (settled or mid-turn):** the runner restore refuses the saved descriptor with the permanent class `provider_state_incompatible`. The server reports it as `native_provider_state_incompatible` and stops automatic retries. The runner opens no provider and keeps the saved state bytes and the workspace. The next run on the task starts a fresh provider session. - **Native Codex (`codex_app_server`) and OpenCode** sessions do not record this field, so their resume decision does not change. After a new image is promoted, native Codex resumes a saved thread with Codex 0.170.0 instead of 0.160.0. This pull request does not test that cross-version thread resume. New tests: - `acpx_provider_backend.rs`: `a_session_saved_by_the_previous_agent_runtime_is_refused_by_name` covers settled Claude and Codex sessions and a Claude turn interrupted mid-run by the deploy. `unrestorable_saved_sessions_keep_strict_bindings_and_name_their_class` has a new `agent_runtime_release` case. Each case checks the named class, that no provider session opens, and that the saved bytes do not change. - `native-session-resume.test.ts`: "a session saved by the previous Claude or Codex ACPX runtime" checks that a same-release session resumes, and that a session saved with Agent SDK 0.3.286 or Codex 0.160.0 starts fresh in the same workspace. ## Risks - Medium. Every Paperclip Runner Claude and Codex run uses the new executables. The digests, generated profiles, image probes, and window must agree. They agree locally, and CI runs the Rust and runner suites again. - Claude Code 2.1.287 through 2.1.328 change some headless defaults. I checked each one against Paperclip: - 2.1.302: `-p` and Agent SDK sessions with no permission mode start in `default` mode on third-party providers. Paperclip always sets a mode (`bypassPermissions` or `default`), so this does not apply. - 2.1.299: MCP allow rules now match only the whole server name. Paperclip already writes `mcp__<server>__*` and exact `mcp__paperclip__<tool>` rules. - 2.1.302: fork subagents are on by default in `-p` and the Agent SDK, as in the terminal. This pull request keeps the upstream default and does not set `CLAUDE_CODE_FORK_SUBAGENT=0`. Reviewers can ask to keep them off. - 2.1.299 and later: stdio MCP connections negotiate MCP protocol 2026-07-28 first and fall back to the older protocol for servers that do not answer. - Agent SDK 0.3.328 is less than one day old. If a regression appears, revert this pull request to return to 0.3.286 and Codex 0.160.0. - Sandbox images built before this change still carry Codex 0.160.0 and Claude Code 2.1.286. They stay accepted. On those images, Fable 5.2 and GPT-6.2 keep the existing named errors and fallbacks until a new image is promoted. - Until the refresh bot updates `pnpm-lock.yaml` on `master`, a local `--frozen-lockfile` install fails, as with the previous pin refreshes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude, model ID `claude-opus-5-5`, through Claude Code with tool use (shell, file edit, `gh`) and code execution. The context window and reasoning mode are not reported in this session. - The registry archives, digests, release notes, configuration schemas, and changelog were read directly. No model output was used as a source for those facts. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ebb98120dc |
Own the issue admission lock and dispatch after commit in wake-queue (#22272)
## Thinking Path
> - Paperclip admits work for a task while other agents and users may
also act on it.
> - A fresh run and a wake behind an active execution both require the
same issue lock.
> - Heartbeat previously opened that transaction and passed handles
across the module boundary.
> - The wake-queue module can bind both admission operations to the
locked company and issue.
> - The result of the database commit determines whether to emit a
queued event, dispatch a run, or return a saved result.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The transaction and handoff used by task-bound wake admission.
**Subsystem affected**
Server task admission.
**Current behavior**
Heartbeat opens the transaction, builds a scope for the busy path, and
dispatches based on the result. Its contention handler can reach
buffered side effects after a rollback.
**Proposed behavior**
Wake-queue owns the lock and the durable acknowledgement. The
application hands off a committed result. A rolled-back contention
attempt returns without publishing its buffered effects.
**Additional context**
Depends on #22268 and #22266.
## What Changed
- Bind the busy and fresh admission operations to a single company and
issue in Postgres. Remove the transaction scope object.
- Keep specialized heartbeat gates and continuation writes on the
acquired transaction.
- Buffer the workspace preflight activity publication until the
transaction commits.
- Centralize the handoff for new, coalesced, replayed, deferred, skipped
and durable results.
## Verification
On
|
||
|
|
e2cee049f9 |
fix: show instruction saves in history, credit the real writer, and speed up the agent tool-rule list (#19515)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Operators and agents change agent instructions (AGENTS.md), and they
must see who changed them and when
> - Agent files replaced instruction revisions. Saves no longer write
revision rows, so instruction history is empty
> - The current revision is stamped with the reader and the date
1970-01-01, so it gives false audit data
> - Separately, an agent that lists tool rules gets a timeout when many
rules name the same resources
> - This pull request reads history from the save records that Paperclip
already writes, credits the real writer, and checks each tool-rule
resource one time
> - The benefit is a correct audit trail for instruction changes and a
tool-rule list that agents can read
## Linked Issues or Issue Description
No public issue exists. I searched open and closed issues and PRs for
these problems and found none.
**What happened?**
1. `get_agent_instruction_history` (and the history API) returns 0
revisions for an agent whose AGENTS.md was saved many times. The
activity log shows each save as `agent.files_updated` with the path,
content hash, actor and run.
2. `read_agent_instructions` returns a current revision with `createdAt`
1970-01-01. Its actor is the agent that reads it, and its `sourceRunId`
is the run that reads it.
3. `GET /api/companies/{companyId}/tools/policies` from an agent fails
with a transport timeout most of the time when the company has many tool
rules. A board user gets the list without a problem.
**Expected behavior**
1. History lists each save, newest first, with its writer, run, source
and time.
2. The current revision names the agent or user that wrote those bytes,
and the time of that save. If no save is recorded, it names no writer.
3. The agent gets the tool-rule list in normal time, with the same rules
hidden as before.
**Steps to reproduce**
1. Save an agent's instruction entry file (board editor,
`update_agent_instructions`, or a run that edits the file). Read the
history. It is empty.
2. As a different agent, call `read_agent_instructions` on that agent.
The revision shows the reader and 1970-01-01.
3. Create about 40 tool rules that name the same connection and agent.
As an agent, list `/tools/policies`. Each rule does a new set of access
checks for each resource, one after another.
**Paperclip version or commit**
`master` at `819420a60a`.
**Deployment mode**
Hosted. The new tests reproduce all 3 problems on `master`.
## What Changed
- `agent-instruction-revisions.ts`: history merges the legacy revision
rows with the `agent.files_updated` save records for the entry file. The
order is `(created_at, id)` descending, and cursors keep PostgreSQL
timestamp precision.
- `agent-instruction-revisions.ts`: the current revision takes its
writer, run, source and time from the latest save of the same content
hash. A reader is never the writer. Bytes with no save record have no
writer.
- `agent-instruction-revisions.ts`: an identical re-save over bytes with
no record is marked `source: "adopted"`. It repairs the runtime journal
but is not listed as a save.
- `agent-instruction-revisions.ts`: a history ID can be a legacy
revision, a save record, or the current content token. A save record
keeps only metadata. If its bytes are no longer current, a read returns
404 with `INSTRUCTION_REVISION_CONTENT_NOT_RETAINED`.
- `agent-file-store.ts`: saves record `byteLength`. A run collection
that changes the entry file records its path, hash and size, so history
shows saves from runs.
- `connection-management-authz.ts`: `filterConnectionManagementPolicies`
keeps one read decision for each distinct resource in a request, and
loads each application or catalog entry one time. A rule is still hidden
when any resource that it names is hidden. Board callers do not use this
path.
- Tests: new tests for save history, paging, writer credit, adoption,
and the tool-rule list. Old assertions that expected empty history now
check the exact saves.
## Verification
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/agent-instruction-revisions.test.ts
src/__tests__/agent-instruction-working-copies.test.ts
src/__tests__/agent-directory-working-copies.test.ts
src/__tests__/built-in-agents.test.ts
src/services/native-runtime/agent-instruction-tools.integration.test.ts
src/__tests__/tool-policy-list-authz.integration.test.ts`
- With the fix: all pass (revisions: 28 passed, 1 skipped on Linux;
other 5 files: 230 passed).
- With the new tests on `master` code: 13 tests fail, as expected.
- `src/__tests__/connection-management.integration.test.ts`: 90 passed.
- The new tool-rule test lists 40 rules that name 1 connection and
counts the connection reads. It needs fewer than 10.
- I could not run the full server typecheck in my sandbox because of a
memory limit. CI runs it.
- CI on head `a2f0b825`: all checks pass, including `ci / verify` and
`ci / e2e`.
## Risks
- History now shows entries whose ID is an activity row ID, not a
revision row ID. Clients that read or restore an older save get a clear
404 because Paperclip no longer keeps those bytes. The latest save and
legacy revisions still read and restore as before.
- Saves recorded before this change have no `byteLength`. For those,
history shows the current size when the hash matches, and 0 otherwise.
- Stock updates of built-in agents do not record a file path, so they do
not show in history.
- No migration. No change to the stored data model.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code with
tool use (shell, file edits, test runs). Standard context window.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: lucktastic <13757243+lucktastic@users.noreply.github.com>
|
||
|
|
d4b7ae5532 |
fix(tasks): require merged PR deliverables before native completion (#22244)
Keep native tasks with declared pull request deliverables active until
GitHub verifies every deliverable is merged. Preserve delivery guidance
in follow-up contracts and recheck the current outputs under task locks.
Use bounded incomplete-work feedback during reassessment, including for
saved non-Done reports. Serialize work-product restoration and identity
changes with completion. Preserve configured reviews and conversations.
Add regression coverage and repair the CI fixtures exposed during
qualification. All CI checks pass on
|
||
|
|
28bdb4d778 |
feat(cloud): migrate durable data online and name volume-free config blockers (#22240)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted Paperclip servers can move from a server with a volume to a volume-free server, which allows deploys without downtime > - A server with history can move online only after its durable data is off the local disk > - Today only an offline maintenance process moves that data, so each existing server takes an outage of many minutes > - Most data owners already work safely while the server keeps serving, but nothing drives them online > - This pull request adds a signed, bounded pass that drives those owners online and reports progress, and it names configuration blockers > - The benefit is that existing servers can move to volume-free with no maintenance window ## Linked Issues or Issue Description **What existing behavior does this improve?** An existing volume-backed server can move to volume-free only through offline maintenance, because only the offline process migrates its local data. **Subsystem affected** Server: ephemeral enrollment, durable data owners (agent files, skill packages and installations, runtime skill assets, native harness backups and controllers), Cloud control endpoints. **Current behavior** - Online preparation refuses while local data remains, and no online path moves that data. - When the volume-free configuration is incomplete, preparation fails with a server error that does not say why. **Proposed behavior** - `POST /api/instance/ephemeral-enrollment/migrate` (Cloud control action `ephemeral-enrollment:migrate`) runs one pass with a 60-second budget. The pass drives the online-safe owners, defers items that are busy, and returns counts, deferral reasons and fresh eligibility. - Preparation and the eligibility read report an incomplete configuration as a named blocker: `volume_free_storage_flags_required`, `volume_free_master_key_required`, `volume_free_decision_key_required`, `volume_free_database_listener_required` or `volume_free_configuration_required`. Preparation returns 409 with these blockers. **Reason and benefit** Cloud can call the pass until a server is eligible, and then move it online with no outage. **Breaking changes** None. The offline maintenance behavior is unchanged. ## What Changed - New `services/online-durable-data-migration.ts`. The pass: - requires S3 storage and `PAPERCLIP_DURABLE_AGENT_FILES=true`, so the server does not keep creating local data; - runs these owners in order, with live writers: `migrateAgentFiles` (`writersStopped: false`), the local skill object copy, skill snapshots and versions, skill installations, runtime skill assets, and the native harness and controller prestage; - stops between pages when the budget is spent; - runs one pass at a time in each process. - `skill-package-storage-migration.ts`: a new `online` mode. In this mode a row that another owner holds is skipped and counted as `busy`. The row claim and the publication already use compare-and-swap. The offline mode is unchanged and still fails closed on a busy row. - `services/ephemeral-enrollment.ts`: a configuration check maps the volume-free contract errors to blocker codes. Preparation and the eligibility read both use this check. - `routes/ephemeral-enrollment.ts`, `middleware/cloud-control.ts`, `services/cloud-runtime-identity.ts`: the new route and action. The route accepts only an empty body. ## Verification - `npx vitest run src/routes/ephemeral-enrollment.test.ts src/services/ephemeral-enrollment.test.ts src/services/online-durable-data-migration.test.ts src/services/skill-package-storage-migration.test.ts src/__tests__/openapi-routes.test.ts src/services/cloud-runtime-identity.test.ts`: 102/102 pass. - New tests: - the pass drives every owner with live writers, aggregates counts and deferrals, and reports eligibility; - the pass does nothing without S3 or durable agent files; - the pass stops at the budget; - one pass runs at a time; - the skill object copy refuses outside maintenance unless the mode is online, skips a busy row online, and still fails closed offline; - each missing setting becomes a named blocker in preparation and in the eligibility read; - the migrate route accepts only a signed migrate assertion and an empty body. - `tsc --noEmit` reports no errors in the touched files. A full local `pnpm verify` needs the Runner build, so CI covers it. ## Risks - The pass runs real migrations while the server serves. Each owner keeps its own locks and compare-and-swap rules. Busy items are deferred, not forced. The budget limits each call. - The skill object copy can now run online. It copies immutable objects, verifies the hashes, and publishes only if the row did not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, browser). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d1ca0cc218 |
fix(native): stop retrying a run that filled its sandbox disk (#22239)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native agent runs execute in a managed sandbox with a fixed disk size > - An agent can fill that disk. The runner then exits with "No space left on device". The disk reserve restarts the runner, but the turn in progress can still be lost, and the run fails > - The run failure did not carry the full-disk fact. The run was classified as a generic provider or runner failure. Automatic recovery then started the same work on a fresh sandbox of the same size, which filled the disk again. After three attempts the task stopped in "Recovery needed" with no useful explanation > - This pull request names the cause `sandbox_disk_full` when the runner reported a full disk during the run, and stops automatic retries for that cause > - The benefit is that Paperclip no longer spends sandboxes and time on retries that cannot succeed, and the task tells the user what to do in plain language ## Linked Issues or Issue Description **What happened?** A native run in a managed sandbox filled the 10 GiB disk. The runner exited with "No space left on device (os error 28)". The runner was restored, but the in-flight turn was lost and the run failed as `native_provider_terminal_failed`. Automatic recovery repeated the same work on a fresh sandbox of the same size three times. Each attempt filled the disk again. The task ended in "Recovery needed" with an unrelated cause. **Expected behavior** When a run ends because its sandbox disk is full, the run failure names that cause. Automatic recovery does not repeat the same work on a sandbox of the same size. The task shows a plain-language explanation and a working Retry. **Steps to reproduce** 1. Run a native agent task in a managed sandbox whose work installs more than the free disk space. 2. Wait until the runner exits with "No space left on device". 3. See the run fail as a generic provider failure, and see automatic retries start fresh sandboxes that fail the same way. **Relevant logs or output** `paperclip-runner` exited with "No space left on device (os error 28)". The run then failed with `provider_turn_lost_on_restore`, classified as `native_provider_terminal_failed`. Refs #22068 (disk reserve that restarts the runner). Refs #22235 (continues a turn that a runner restore lost, once, in the same sandbox). This PR covers the case where the run still ends. ## What Changed - Server: each run's diagnostic lease now records when a runner diagnostic reports a full disk (`ENOSPC`, `EDQUOT`, "No space left on device", "Disk quota exceeded"). The runner's exit detail already reaches the server as a diagnostic, so no runner change is needed. The fact belongs to that run's diagnostic epoch only. A later run on the same warm session, or a diagnostic after the epoch retires, does not see it. - Server: native failure classification returns `sandbox_disk_full` when either (a) the run's lease saw a full-disk report, or (b) the error chain carries the runner's existing `sandbox_disk_full` launch code (a relaunch that cannot stage on the full disk). A fixed list of failures keeps its own meaning: permission, approval, usage limit, model, runtime, integrity, cleanup quarantine, ownership, saved-state and admission failures. - Server: `sandbox_disk_full` is a permanent native failure. The run does not take another same-run attempt. Native safe replacement does not pick it up, because it is not one of its failure codes. The task becomes blocked with a board-owned recovery action and a plain-language next action. - Server: the bounded retry scheduler (`scheduleBoundedRetryForRun`) refuses `sandbox_disk_full`. It does not schedule a `transient_failure` retry on a fresh sandbox of the same size. - UI: the task chat shows "Disk full" and a plain explanation instead of the raw failure code. - Retry stays available. `sandbox_disk_full` is not a reconciliation cause, so the board's Retry starts a new run when the user chooses. ## Verification - Red before the fix (captured on the unfixed code): - `heartbeat-retry-scheduling.test.ts` "does not repeat a run that filled its sandbox disk on a fresh sandbox of the same size": expected `not_scheduled`, received `scheduled`. - `native-sandbox-disk-full.test.ts` "names the full disk and stops instead of retrying the same work": expected `sandbox_disk_full`, received `native_provider_terminal_failed`. - `native-sandbox-disk-full.test.ts` "names the full disk when the runner could not be relaunched on it": expected `sandbox_disk_full`, received `native_session_interrupted`. - Green after the fix: - `cd server && npx vitest run src/services/native-runtime/native-sandbox-disk-full.test.ts src/__tests__/heartbeat-retry-scheduling.test.ts src/services/native-runtime/remote-runner-disk-reserve.test.ts` (90 passed). - `cd ui && npx vitest run src/components/TaskChatThread.test.tsx` (426 passed). - `cd server && npx vitest run src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-session-diagnostics.test.ts`. - Safety cases in the tests: - A runner that exited for another reason still gets its bounded retry. - A turn lost on restore without a full-disk report keeps `native_provider_terminal_failed`. - Integrity and approval failures keep their codes when a full disk was reported. - A full-disk report belongs only to its own run's diagnostic epoch; run B on the same session does not see run A's report. - After a full-disk report, other runner and provider loss codes (session.open timeout, initialize timeout, protocol error, process status failure) also become `sandbox_disk_full`. - Worst-case automatic run starts per hour across a company: before, each disk-full task started up to 3 automatic runs, each on a fresh sandbox (3 × the number of affected tasks). After, a disk-full failure starts 0 automatic runs. Only a person's Retry starts a new run. Real transient failures keep their existing bounded lanes, so their worst case is unchanged. - Not run locally: the full server suite, the full typecheck and build. Required CI covers them. - Not verified live on staging yet. A live check needs a run that fills its sandbox disk after this change deploys. ## Risks - A run whose runner reported a full disk, and which later fails with a runner or provider loss for an unrelated reason, is now labeled `sandbox_disk_full` and is not retried automatically. The board's Retry still works. This is the safer error: a full disk is the most likely cause, and an automatic retry would repeat the same disk use. - The full-disk report lives on the run's in-memory diagnostic lease. A server restart between the report and the failure loses it, and the run falls back to the previous classification. That is the previous behavior, not a new failure. - Low risk otherwise. No schema or API change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
eaac8ed0f7 |
feat(adapters): add Claude Fable 5.2 and GPT-6.2 with CLI floors, and refresh sandbox CLI pins (Nov 18 audit) (#22222)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents run on coding harnesses. Each adapter has a model catalog, and sandboxes contain pinned CLIs. > - Anthropic released Claude Fable 5.2 and OpenAI released GPT-6.2 Sol and GPT-6.2 Astra for Codex. Both need newer CLIs than the Paperclip Runner provider pack ships. > - A stale catalog hides valid models. A model that needs a newer CLI fails with an opaque backend error. > - This pull request adds the verified model IDs with their CLI floors, names the version gap in Paperclip Runner, and refreshes the sandbox CLI pins. > - The benefit is that operators can select current models and get a named version error instead of a backend failure. ## Linked Issues or Issue Description Refs #20774 and #21636 (both add parts of Fable 5.2 support; see "Overlap" below). Refs #18486 (adds GPT-6.1 Terra; this PR does not duplicate it). Refs #16708 (adds Claude Haiku 5.5 to `claude_local`; this PR does not duplicate it). **What happened?** Claude Fable 5.2 (`claude-fable-5-2`) and GPT-6.2 Sol / Astra (`gpt-6.2-sol`, `gpt-6.2-astra`) are generally available. The `claude_local`, `codex_local`, and OpenCode pickers do not list them. Claude Code 2.1.321 is the first release with Fable 5.2. Codex 0.169.1 is the first release whose bundled catalog lists GPT-6.2. The Paperclip Runner provider pack bundles Claude Code 2.1.286 and Codex 0.160.0. **Expected behavior** The pickers list the new models. A run on a CLI below the model minimum stops with the detected and required versions, or falls back to a compatible model with a visible message. **Proposed change** Add the models and their per-model CLI floors. Gate ACPX Claude in Paperclip Runner on the Claude Code version that its Agent SDK bundles. Refresh the sandbox CLI pins with verified digests. Record the audit. ## What Changed - Claude Code (`claude_local`): - Add `claude-fable-5-2` (label "Claude Fable 5.2") at the top of the picker. `claude-fable-5-1` stays selectable after the current-generation models. - Add Bedrock `global.anthropic.claude-fable-5-2`. Anthropic documents only this global ID for Bedrock. - Fable 5.2 offers all five efforts (`low` to `max`). - `minimumClaudeCliVersionForModel("claude-fable-5-2")` returns `2.1.321`. The existing CLI-lane Test and launch checks use this floor. On a managed sandbox, the CLI lane installs a private Claude Code at this version, as it does for the other floors. - Paperclip Runner (ACPX Codex): `resolvePaperclipRunnerProviderProfile` rejects a model whose Codex floor is above the qualified ACPX Codex runtime (`0.160.0`). The error code is `paperclip_runner_codex_cli_version_incompatible`, and the message gives both versions. - Paperclip Runner (ACPX Claude): `resolvePaperclipRunnerProviderProfile` rejects a model whose Claude Code floor is above the version that the qualified Agent SDK bundles. The error code is `paperclip_runner_claude_cli_version_incompatible`. The message gives both versions. A new helper, `claudeCodeVersionForAgentSdk`, maps SDK `0.3.N` to Claude Code `2.1.N` (the SDK's `claudeCodeVersion` field) and returns `null` for any other line. - Codex (`codex_local`): - Add `gpt-6.2-sol` and `gpt-6.2-astra` at the top of the picker, in the bundled catalog order. - Both offer Fast mode and the `low` to `ultra` efforts. - Both have a CLI floor of `0.169.1`. On the native Codex provider with the pinned Codex 0.160.0, the existing fallback moves `gpt-6.2-sol` to `gpt-6.1-sol` and `gpt-6.2-astra` to `gpt-6-astra` with a visible message. - The default stays `gpt-6.1-sol`. - OpenCode fallback catalog: add `openai/gpt-6.2-sol`, `openai/gpt-6.2-astra`, `anthropic/claude-fable-5-2`, `google/gemini-4-carbon`, and `google/gemini-4-hydrogen`. - Evaluation image sandbox layer (`docker/daytona-runner`): | CLI | Before | After | | --- | --- | --- | | Grok CLI | 1.0.70 | 1.0.89 | | Gemini CLI | 0.65.0 | 0.68.0 | | Cursor CLI | 2026.10.22-27dcf5f | 2026.11.17-f532c45 | | GitHub CLI | 2.103.3 | 2.105.0 | - Add `doc/adapter-model-audit-2026-11-18.md` with sources, compatibility findings, and deferred items. Add a note on above-window Codex floors to `doc/architecture/paperclip-runner-compatibility.md`. No saved agent changes model, effort, or spending settings. No picker entry is removed. ### Managed sandboxes On Paperclip Runner, Fable 5.2 and GPT-6.2 stay unusable until the provider pack moves to Claude Code 2.1.321 or later and Codex 0.169.1 or later. A sandbox image that carries that provider pack must also be promoted. This PR does not change the provider pack (see "Deferred items" in the audit). Native remote Codex accepts only `>=0.149.0 <0.161.0`, so GPT-6.2 also needs that window to move. ### Overlap - #21636 sets the Fable 5.2 floor to `2.1.290`. The [Claude Code changelog](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md) adds Fable 5.2 in 2.1.321. `git merge-tree` merges the two without a text conflict, but #21636's line becomes unreachable after this PR's line, and its test then fails. Rule: keep `2.1.321`. Merge this PR first. Then close #21636 or change its floor and test to `2.1.321`. - #20774 (draft) lists `claude-fable-5-2`. It conflicts in `packages/adapters/claude-local/src/index.ts` and `server/src/__tests__/adapter-models.test.ts`. Rule: keep this PR's catalog order and Bedrock ID. - #18486 adds `gpt-6.1-terra`. It conflicts in `packages/adapters/codex-local/src/index.test.ts` and `src/server/codex-args.test.ts` (the ordered ID list and the Fast-mode message). Rule: keep both sets of models in the bundled catalog order (`gpt-6.2-sol`, `gpt-6.2-astra`, `gpt-6-astra`, `gpt-6.1-sol`, `gpt-6.1-terra`, ...). Either PR can merge first; the second rebases. - #16708 adds Claude Haiku 5.5. It conflicts in `packages/adapters/claude-local/src/index.test.ts` (adjacent test cases) and `doc/DEVELOPING.md`. Rule: keep both test blocks. ## Verification Research date: November 18, 2026. Sources are listed in the audit doc. - `vitest run src/index.test.ts src/server/model-order.test.ts src/server/cli-capabilities.test.ts src/server/test.remote.test.ts` in `packages/adapters/claude-local`: 40 passed. `execute.remote.test.ts` (29), `test.probe.test.ts` (21), `sandbox-cli-version.test.ts` (15), `acp.auth.test.ts` (25), and `acp.test.ts` (47, with `--testTimeout=60000`) also passed when run one file at a time. - `vitest run src/index.test.ts src/server/codex-args.test.ts src/ui/build-config.test.ts src/server/test.remote.test.ts` in `packages/adapters/codex-local`: all passed (155). `acp.test.ts` (49), `acp.cli-version.test.ts` (4), and `execute.test.ts` (10) also passed one file at a time. - `vitest run` in `packages/adapters/opencode-local`: 94 passed. - `vitest run src/services/native-runtime/acpx-qualification.test.ts src/services/native-runtime/codex-model-fallback.test.ts src/__tests__/adapter-models.test.ts src/__tests__/heartbeat-model-fallback-warning.test.ts` in `server`: all passed. The new cases cover the ACPX Claude and ACPX Codex version errors and the native GPT-6.2 fallback on Codex 0.160.0. - `vitest run src/lib/agent-setup-fields.test.ts src/lib/codex-reasoning-effort.test.ts` in `ui`: 11 passed. - `tsc --noEmit` in the three adapters and `pnpm typecheck` in `server`: passed. - Cursor archive SHA-256 `189ed41d8963cd22dcdafbbd62f072cc55ac08374c922dbd04aee525f4018ab6` is computed from the versioned Linux x64 tarball. The extracted binary reports `2026.11.17-f532c45`, the version that `https://cursor.com/install` resolves. - GitHub CLI SHA-256 `79e7ab5db738c48777b696e374f8b428f0647debfb424f44d2f426370407fe2f` matches `gh_2.105.0_checksums.txt`. - Grok CLI 1.0.89 and Gemini CLI 0.68.0 installed from npm report those versions. The flags that Paperclip uses are in `--help` (`-m`, `--effort`, `grok models`; `-m`, `-p`, `--output-format`, `--approval-mode`, `--yolo`). - Host baseline: tests that create an AI provider need the host's empty `ANTHROPIC_BASE_URL` unset. Under full-package parallel runs, SSH and sandbox fixture tests time out at 5 seconds on this host; each file passes alone. The audit doc records both. - Not run locally: the full `pnpm -r typecheck`, `pnpm build`, browser suites, and the image build. CI runs these. No live model inference ran. ## Risks - Low risk for saved agents: no default, saved model, effort, or spending setting changes. - The new Paperclip Runner checks reject `claude-fable-5-2` on ACPX Claude and `gpt-6.2-*` on ACPX Codex (and any later model whose floor is above the bundled runtime) before launch. Without this PR, the turn would start and fail at the backend. - The `0.3.N` → `2.1.N` mapping covers only the current SDK line. Any other line returns `null`, and the check is skipped. - The sandbox CLI pins change only the evaluation image. The fleet image change is separate. - Rollback: revert this PR. No data migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, web fetch, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
46e38e1387 |
fix(native): let the outgoing server checkpoint its idle warm session while its follow-up has not started (#21887)
**HOLD: do not merge until the coordinator confirms (merging resets the zero-downtime gates).** ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A deploy swaps servers with an overlap. A warm agent session stays with the outgoing server that ran its last turn. > - When that turn finishes during the overlap, the outgoing server saves its idle session and closes it. A checkpoint fence decides whether the save is still allowed. > - The task's follow-up run usually starts on the incoming server at the same moment. It waits for the outgoing owner (#21774), and #21791 lets the outgoing server save while the follow-up is parked on it. > - The fence still refused the save (`native_runner_checkpoint_owner_lost`) in every other state of the same unstarted follow-up. In release-swap waves the close then fails at least once and retries, and the follow-up waits longer. If every attempt hits such a window, the owner stays until the outgoing server stops, up to the 510 s overlap. Speed is a requirement. > - This pull request makes the fence accept the outgoing server's final save in every state in which the follow-up has not started: no process, no own process start, no own checkpoint, `attempt = 0`, same session. It still refuses when the follow-up names another owner or has started. > - The benefit is that the outgoing server closes its session on the first attempt, and the follow-up continues at its next owner check. ## Linked Issues or Issue Description Refs #21791, #21774, #21782 **What happened?** Release-swap waves logged `idle warm native session close failed … native_runner_checkpoint_owner_lost`. The close worked on a later attempt. Two recent waves, traced with the wave's 2-second run samples: | Wave | Outgoing run | Attempt | Follow-up at that moment | Old fence | | --- | --- | --- | --- | --- | | 13:48Z | `480c86e4` (finished 14:29:04.7) | 1, 14:29:07.7 | `5396e5ab`, started 14:29:05.6 on the incoming server: `preparing` (legacy, incoming boot) at 06.2, native `observed` with no owner at 08.2 | refused: a preparing follow-up was accepted only on the outgoing server's own boot, and a native follow-up only when parked or re-claimed with `waitingForOwner` | | 15:58Z | `87de6f54` (finished 16:16:14.1) | 1, 16:16:19 | `2009f6bb`, started 16:16:18.7: `preparing` (legacy, incoming boot) at 19.8 | refused: foreign preparation | | 15:58Z | `87de6f54` | 2, 16:16:27 | `2009f6bb`: native `observed`, no owner from 21.8 to 25.8, owner stamped at 27.8 | refused: selected follow-up before its first owner check, or a park or re-claim without `waitingForOwner` | In each case the follow-up was the task's current run and had no process, no process-start event and no checkpoint rows. No successor claim or stale epoch held the session: the outgoing server's own run had committed and released its lease. The fence refused only because it did not recognize these states of the same unstarted follow-up. **Expected behavior** The outgoing server saves and closes the session on the first attempt while the follow-up is in any unstarted state. The follow-up then sees the saved session at its owner check and continues without parking (or within one recheck if it already parked). **Steps to reproduce** 1. Run a warm native session on a remote sandbox, and let a run finish on server A during a swap to server B. 2. Let B claim the follow-up right away. 3. While B prepares it, selects native execution, parks it, or re-claims it, A saves and closes its idle session. The fence refuses the save. **Paperclip version or commit** master `f61b919701` (merged in). The waves ran on master builds without this pull request. ## What Changed - `native-runner-checkpoint.ts`: the parked-follow-up check is now `unstartedSuccessorOfOwner`. It takes `FOR SHARE` on the follow-up's finalization row (if any), then on its run row, in that order, inside a savepoint. The claim and defer paths use the same order. If native selection created the finalization row while the fence waited for the run lock, the fence rolls the savepoint back, which releases the run lock, and takes both locks again in the same order (at most twice). A last statement then reads the current state again and accepts one of these states: - **Preparing on any server** (new): the run is still `legacy`, `preparing`, on this task, has a live controller lease and no process, and has no finalization row. Before, only the outgoing server's own boot was accepted. - **Selected, before its first owner check** (new): native, finalization `observed`, `attempt = 0`, no lease, no failure, empty recovery history. - **Parked**: as before, but `waitingForOwner` may be absent. - **Re-claimed**: as before (live `same_run_resume` claim by the current controller), but `waitingForOwner` may be absent, as in claims from a previous release. - These conditions apply to every state and are unchanged: running; no own process-start event; no own checkpoint rows. For the native states, also: same company, task and native session; `attempt = 0`; no result; no process, unless it is the identity inherited from the outgoing run (`inheritedProcessIdentityRunId`). The fence still requires an accepted result and no new work. - A `waitingForOwner` that names another run or boot is still refused. - `SAME_RUN_RESUME_DISPOSITION` moves to the leaf module `same-run-resume-disposition.ts`. The fence no longer imports `scheduled-resume-continuation.ts`, so the runner E2E server does not load the executor and `@paperclipai/plugin-sdk` through it (`server-startup-import-graph.test.ts`). - The run-row share lock makes native selection (`runtime_selection` locks the run `FOR UPDATE`) and process start wait for the save to commit. A save that waited reads the committed state in its last statement and is refused if the follow-up has started. - Tests, real Postgres: - `native-controller-recovery.integration.test.ts`, new: - The outgoing server saves and closes while the incoming server prepares the follow-up. - Native selection of that follow-up waits for the save. - A save that waited while native selection held the run lock and created the coordinator is accepted (Greptile review). - It saves and closes while the follow-up is selected but before its owner check, also with an inherited process identity, and the follow-up's check resolves at once. - It saves and closes while the follow-up is parked or re-claimed without `waitingForOwner`, including a claim entry in the previous release's shape. - Still refused, preparing: expired controller lease, `dispatching`, own process, own process-start event, other task. - Still refused, selected: own process, a process inherited from another run, own process-start event, own checkpoint, other session, admitted (`attempt = 1` with a lease). - The outgoing-server fixtures share one helper. - `native-harness-backup-storage.integration.test.ts`: the harness backup fence shares this check. A released owner whose follow-up is still preparing on another server now replaces its selected backup. `expired_preparation`, `dispatching_successor` and `foreign_native` still keep the old backup. - `native-runner-restart-recovery.integration.test.ts`: the `foreign_preparation` early-retirement case now expects the save to succeed. `expired_preparation`, `dispatching_successor`, `wrong_issue`, `unclaimed_successor` (another session) and `foreign_native_preparation` still expect a refusal. - From the earlier commits of this pull request: the re-claimed state, `sameRunResumeClaimEntry` recording `waitingForOwner`, the run-row lock and the post-lock recheck. ## Verification - Red/green for this round (only the fence source changed between runs): - With the fence at `ed7227eeda` (this pull request before this round), 9 tests fail with `native_runner_checkpoint_owner_lost`: preparing close (+ `foreign_preparation` ×2), native-selection ordering, selected-before-check ×2, waiting-without-owner ×3. The 11 new refusal cases pass both before and after. - With the fence change: `native-controller-recovery.integration.test.ts` and `native-runner-restart-recovery.integration.test.ts` 306 passed, 14 skipped. `native-harness-backup-storage.integration.test.ts` 73 passed. `scheduled-resume-continuation.test.ts`, `native-runner-checkpoint-save.test.ts` and `native-controller-recovery.integration.test.ts` 110 passed. - Savepoint restart: the new waited-on-native-selection test fails with `native_runner_checkpoint_owner_lost` without it, and passes with it. - `tests/runner-e2e/server-startup-import-graph.test.ts` failed in CI at `cc513d9434` (import path through `scheduled-resume-continuation.ts`). It passes locally after the move. - Earlier rounds (unchanged): red on master for the re-claimed close, the process-start ordering and the claim ordering; red without the run-row lock; red without the recheck. - `tsc --noEmit` for `server` passes. - `pnpm verify` (base `f61b919701`) selected 2842 of 3683 files, because the checkpoint module has a wide import graph. The reusable job passed: 2 files, 12 tests. The fresh job was cut off locally (exit 144, no test failure printed). Required CI supplies that coverage. - `node scripts/agent-preflight.mjs` stops at `pnpm install --frozen-lockfile`: on master, `server/package.json` lists `posthog-node` (#22006) and the lockfile does not have it yet. CI resolves the lockfile inline. Locally, dependencies were installed without the frozen lockfile, and `pnpm-lock.yaml` was not committed. - Not run: a release-swap wave. The coordinator runs the gates. ## Risks - The fence is wider. It now accepts the outgoing server's final save while the follow-up is in any state before it starts. The save needs an accepted result and no new work. The follow-up must have no process (except the identity inherited from this run), no process-start event, no checkpoint of its own, and `attempt = 0`. A follow-up that names another owner, holds an expired claim, has been admitted, or is in another session is still refused. - A follow-up that another server is preparing is now accepted. It has not touched the session: it runs the owner check after native selection, and native selection waits for the save to commit. The earlier same-boot-only rule was stricter than needed. - The fence takes share locks on the follow-up's rows whenever it reaches this check, also when it then refuses. The save transaction is short. The lock order is the finalization row, then the run row, as in the claim and defer paths. - Previous-release state: a claim or park written by a release without `waitingForOwner` is now accepted, not refused. No stored shape changes. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c36e791bd9 |
fix(native): back off and retry a provider rate limit that reports no reset (#22216)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for work
> - Native runs drive ACP agents, such as Claude, through the runner. The runner types each provider failure for the server.
> - When a provider limit stops a turn, the server holds the task. It retries the task automatically at the reset that the provider reports (#20692).
> - Claude can reject a request at the account rate limit with "This request would exceed your account's rate limit. Please try again later." This text has no reset time and no window.
> - The server then plans no automatic Retry. The task stays blocked until a person retries it, and the case work stops before a final reply.
> - This pull request gives a `rate_limit` failure without a reported reset the default provider-limit cooldown, and then requests the task's Retry.
> - The benefit is that a short account rate limit no longer strands a task. The existing Retry admission keeps the saved work and all gates.
## Linked Issues or Issue Description
Refs #20692, #19246, #21100
**What happened?**
A native task ran on a Claude agent. In the middle of the turn, Claude returned HTTP 429 with "This request would exceed your account's rate limit. Please try again later." The runner typed the failure as `rate_limit`, with no `retryAt`. The server put the task on hold with "Restore model provider usage capacity, then retry the task. The provider did not report when its usage limit resets." It did not plan an automatic Retry. The task stayed blocked until a person retried it.
**Expected behavior**
A rate limit passes by itself. Paperclip should wait a bounded time, and then retry the task. It should keep the saved work and should not start another request before the wait ends.
**Steps to reproduce**
1. Run a native task on a Claude agent whose account is at its rate limit.
2. Let Claude reject a request in the turn with the 429 account rate-limit text above.
3. See that the task hold has no reset time and that no automatic Retry is planned.
**Paperclip version or commit**
`master` at
|
||
|
|
f472991647 |
fix(native-runtime): resolve assigned tool names to the authorized alias (#22213)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for work.
> - Native runners give agents their assigned app tools through two wrapper tools: `paperclip_search_assigned_tools` and `paperclip_call_assigned_tool`.
> - Search returns each tool with a hashed alias (`app_<readable>_<digest>`). The description also shows the display name.
> - Agents sometimes pass the gateway tool name or the display name instead of the alias, in `schemaTool` or in the call `name`.
> - The wrapper rejected these values with a bare `assigned_mcp_tool_unknown`. The agent had to search the catalog again before it could call the tool.
> - This pull request resolves an unambiguous name to the authorized alias, and gives a corrective error when the name is not known.
> - The benefit is fewer failed tool calls and fewer repeated catalog searches, with no change to authorization.
## Linked Issues or Issue Description
**What happened?**
An agent searched its assigned tools and then called a tool with the gateway name or the display name (for example `sql_query`). The search with `schemaTool` and the call both failed with `assigned_mcp_tool_unknown`. The call error did not tell the agent what to do.
**Expected behavior**
When the name identifies exactly one tool in the session catalog, the wrapper uses that tool. When the name is not known or is ambiguous, the error tells the agent to use the exact name from search.
**Paperclip version or commit**
`master` at
|
||
|
|
84a529df4f |
fix(monitors): let low-trust reviewers schedule a self-wake on their own task (#22212)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use the `set_task_monitor` tool to save a timed follow-up
check on their task
> - Agents that run with the `low_trust_review` preset (for example,
pull request reviewers) cannot save this check
> - The tool requires company-wide `runtime:manage` access, and the
preset denies that access
> - Thus a reviewer must wait for a different event before it can
continue its review
> - This pull request lets the assignee save or clear a self-wake
monitor on its own task, and keeps the company-wide restriction
> - The benefit is that reviewers can continue their own review on time,
with no new company-wide access
## Linked Issues or Issue Description
**What happened?**
A `low_trust_review` agent calls `set_task_monitor` on its own
in-progress task. The call fails with this error: "low_trust_review
agents cannot use company-wide or privileged runtime:manage APIs by
default." The tool is advertised to the agent, but the agent cannot use
it. The agent cannot schedule its own follow-up check.
**Expected behavior**
The assignee can schedule or clear a monitor that only wakes itself on
its own task inside its trust boundary. Monitors that create recovery
tasks or escalate to the board stay denied. Company-wide
`runtime:manage` stays denied.
**Steps to reproduce**
1. Assign an in-progress task to an agent.
2. Start a native run for that task with a `low_trust_review` execution
policy whose boundary root is the task.
3. Call `set_task_monitor` with a future `nextCheckAt` and notes.
4. See the 403 error with reason `deny_low_trust_boundary`.
**Paperclip version or commit**
`master` at
|
||
|
|
8a295f944d |
fix(runner): fail fast when an artifact storage read stalls (#22210)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use the `download_artifact` tool to copy a saved API artifact into their workspace > - The server reads the artifact bytes from storage before it installs the file > - This storage read has no deadline. A stalled storage response or stream keeps the tool call open > - The provider MCP client stops the call after about five minutes with the error "The operation timed out." > - This pull request puts a 60-second deadline on the storage read and returns a clear, retryable error > - The benefit is that the agent gets a fast, clear result and can retry, and does not wait for five minutes ## Linked Issues or Issue Description **What happened?** Some `download_artifact` calls stopped for four to five minutes. Then the agent got only "The operation timed out." In one case, a retry of the same 1 MB artifact completed in about 20 seconds. **Expected behavior** A stalled storage read must fail quickly. The error must tell the agent that no file was written and that it can retry the same download. **Steps to reproduce** 1. Give `PaperclipRunnerToolAuthority` a storage service whose `getObject` never resolves, or whose stream sends one chunk and then stops. 2. Call `download_artifact` for an artifact in that storage. 3. The call does not settle. Nothing aborts the storage request. **Root cause** `#readApiFile` called `storage.getObject(...)` without a `signal` and read the stream with no deadline. Other storage reads in the runner, for example chat attachment reads, already have a deadline. ## What Changed - `server/src/services/native-runtime/paperclip-runner-tool-authority.ts`: the artifact branch of `#readApiFile` now uses a 60-second `AbortSignal.timeout`. It sends the signal to `storage.getObject`. It destroys the stream when the deadline expires. Every storage stream gets an error listener before it can be destroyed, so a late reply cannot raise an uncaught `AbortError`. It returns HTTP 504 with code `artifact_storage_timeout` and the message "Artifact storage read did not finish within 60 seconds. No file was written; retry the same download." - A new optional binding field, `storageReadTimeoutMs`, sets the deadline for tests. The default is 60 seconds. The production callers do not change. - `server/src/services/native-runtime/artifact-transfer.integration.test.ts`: a new regression test covers a storage response that never arrives and a stream that stops after one chunk. ## Verification - Failing before the first fix: `cd server && npx vitest run src/services/native-runtime/artifact-transfer.integration.test.ts -t "stalls"` failed with `Error: Test timed out in 15000ms.` The `download_artifact` call never settled. - Failing before the late-reply fix: the same test now also covers a storage reply that arrives after the deadline, with a real `createReadStream` that gets the aborted signal. It failed with an uncaught `AbortError` (`ABORT_ERR`). - Passing after both fixes: the same command passes. Each stalled call fails in less than 5 seconds with the retryable error. All storage signals are aborted. No uncaught error occurs. The workspace has no files. - `npx vitest run src/services/native-runtime/artifact-transfer.integration.test.ts src/services/native-runtime/paperclip-runner-tool-authority.test.ts`: 49 passed. - `npx vitest run src/services/native-runtime/runner-api-artifacts.integration.test.ts src/services/native-runtime/runner-api-artifacts.test.ts src/services/native-runtime/issue-document-batch.integration.test.ts`: 38 passed (first revision). - `pnpm run typecheck` (server): passed. - `pnpm verify` selects a very large set of files for this module. I did not run the full set locally. CI covers it. ## Risks - Low risk. The deadline applies only to the artifact read from storage, and artifacts are at most 10 MiB. A healthy read takes seconds. - A very slow but working storage provider that needs more than 60 seconds for 10 MiB now gets a retryable error. - The error occurs before any file write, so no partial file stays in the workspace. - This change does not fix a slow storage provider. It makes the failure fast and clear. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in a sandboxed workspace. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7f9c115cc2 |
fix(github): let a queued push start the next review turn instead of a blocker continuation (#22223)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A GitHub review bot reviews each PR in a task. Each push sends a new review request to that task as a chat message. > - A push that arrives during a review waits as a deferred chat wake. That wake starts a new turn. The turn binds its tools to the newest pushed commit. > - #21959 and #22140 continue a run that reports a blocker in a fresh session. That continuation keeps the source binding of the blocked run. It also holds the task, so the waiting push request does not run. > - On staging, a push during a review made the publication fail with "requested for an older pull request head". The agent reported a blocker. Paperclip then continued twice with the old binding. Each continuation finished the review and could not publish it. > - A second failure mode: a complete re-review that listed only the files changed since the prior review got a short rejection. The rejection named no files, and a small model stopped instead of correcting the submission. > - This pull request lets a queued chat request take the next turn instead of a blocker continuation. It also names the missing files in the coverage rejection. > - The benefit is that reviews publish for the newest push without a person, and agents can correct incomplete coverage on the next call. ## Linked Issues or Issue Description **What happened?** On staging, automatic GitHub reviews stopped publishing. The bots posted "Publication is currently blocked" and "Paperclip must authorize the review for <newest commit>". The run history showed this sequence: a review run started for the first commit. Two pushes arrived during the run. Steering into the active run was rejected for both pushes, so they waited as deferred chat wakes. The run reported a blocker. Paperclip then started two `native_environment_blocker` continuations. Each continuation had no message of its own, so its GitHub tools resolved to the first commit's request and refused the current head. The waiting push requests did not run until a later push. **What did you expect?** A push request that waits behind a run starts the next turn, bound to that push. The agent then reviews and publishes for the current head. **Steps to reproduce** 1. Start an automatic review on a PR. 2. Push a new commit while the review runs, so the review request waits as a deferred chat wake. 3. Let the run report a blocker (for example after the stale-head publication refusal). 4. See that a fresh-session retry starts with the old binding and the push request stays deferred. Refs #21959, #22140, #22066 ## What Changed - `status-arbiter.ts`: new rule before the blocker continuation lanes. When a blocked run has a queued chat request for the same task and agent, the decision is `preserve` with only `release_checkout` (reason `native_blocker_superseded_by_queued_request`). The normal release drain then starts the queued request. A pause hold or a board-owned blocker keeps the existing behavior. - `native-runtime/queued-chat-request.ts`: new `hasQueuedChatRequest` query. It uses the same chat-wake test that keeps chat wakes out of first turns (`chatConversationId` or `externalChatProvider`). - `native-run-finalizer.ts`: supplies `hasQueuedChatRequest` to the arbiter, only for a blocked report. - `chat-github-review-policy.ts` / `chat-github-reviews.ts`: `githubCompleteCoverageGap` replaces the inline check. The rejection now names the missing files (up to 20), says that a complete review covers all changed files, and gives the two corrections (add them to `reviewedPaths`, or submit `complete: false` with `omittedPaths`). ## Verification - Red/green: `cd server && npx vitest run src/services/native-runtime/native-environment-blocker.integration.test.ts`. The new case failed before the fix with reason `native_environment_blocker` and one scheduled retry, as on staging. After the fix it passes: no retry, the task stays `in_progress`, and the chat wake stays deferred for the release drain. - Red/green: `npx vitest run src/services/native-runtime/status-arbiter.test.ts -t "queued chat request"`. - Red/green: `npx vitest run src/services/chat-github-review-policy.test.ts -t githubCompleteCoverageGap`. - `native-agent-blocker.integration.test.ts` and the full arbiter and review-policy suites pass. - `cd server && npx tsc --noEmit -p .` passes. - `pnpm verify` on the changed files (results in the CI checks below). ## Risks - Low to medium. The new rule changes only blocked reports that would otherwise auto-continue while a chat request waits behind the run. Without a waiting chat request, the behavior does not change. - The waiting request now starts the next turn instead of the continuation. If the blocker is real, that turn can report it again, and the existing lanes apply. - The finalizer does one more indexed read, only for blocked reports. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code with tool use: code search, editing, shell, test runs, and staging run inspection through the browser. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8285a81280 |
fix(native): retry a fresh launch whose provider pack could not be verified (#22106)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Before a remote native run starts its provider, the controller selects a provider pack and verifies it in the sandbox > - The verification command retries once. If both reads fail without a result (a timeout, a signal, or a transport error such as HTTP 502), the run fails with `runner_remote_provider_verification_unavailable` > - That code holds the run for a person in every case. The hold exists for reattachment, which does not spend a new provider attempt and could loop > - On a fresh launch the code is also not on the list of failures that prove no provider session started. So the run is classified as "ambiguous state", and the task waits behind a Board-owned Retry > - This pull request retries a fresh launch's unverified pack on the same run, within the provider attempt budget, and keeps the hold for every other attempt > - The benefit is that a short sandbox outage during startup no longer stops a chat or task until a person clicks Retry ## Linked Issues or Issue Description **What happened?** A chat message started a fresh native run on a new sandbox. The preinstalled provider pack did not match, so the controller uploaded the pack. Both reads that verified the uploaded pack failed with "Request failed with status code 502". No provider process started. The run failed with `runner_remote_provider_verification_unavailable`, the recovery mode was `ambiguous_state`, and the chat showed a Retry button. A manual Retry then succeeded. **Steps to reproduce** 1. Start a fresh remote native run. 2. Make both provider-pack verification reads fail without a result, for example with a 502 from the sandbox command API. **Expected behavior** The run retries its startup on the same run, with backoff, because no provider session started. **Actual behavior** The run is held for a person with "Inspect the original provider failure and explicitly resolve the ambiguous session state". ## What Changed - `native-session-executor.ts`: - `nativeProviderRecoveryEvidence` treats `runner_remote_provider_verification_unavailable` as definitely pre-session when the attempt is a fresh bootstrap (the caller passes `bootstrap`). As for every pre-session code, this applies only when the run has no checkpoint and no provider event. - `nativeSessionFailureDisposition` takes an optional `freshBootstrap` flag. With it, `runner_remote_provider_verification_unavailable` is retryable instead of permanent, so the existing same-run startup retry and its three-attempt budget apply. The executor passes the same fresh-bootstrap condition it already uses for `bootstrap`. - Reattachment, warm session recovery and retained transitions do not pass `bootstrap`, so they keep the hold. - `doc/native-workspace-finalization-recovery.md`: describe the fresh-launch retry. - Tests: the disposition retries attempts 1 and 2 of a fresh launch and stops at attempt 3, and the existing reattachment hold test is unchanged. An integration test shows `bootstrap_retry` for a fresh launch with no checkpoint, and `ambiguous_state` without `bootstrap`. ## Verification - `npx vitest run` on the new cases: pass. Both fail without the source change. - `npx vitest run` on the native session executor, controller recovery, remote provider pack, online source, status arbiter corpus, continuation and startup/retry suites: 2203 pass, 8 skipped. - `npx tsc --noEmit` passes for the server. ## Risks - Low. Only a fresh launch with no checkpoint and no provider event changes. A deterministic verification outage now costs up to two more startup attempts, with backoff, before the run is held. - A completed verification that fails (`runner_remote_provider_artifact_incompatible`) is unchanged. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
adb2bcf896 |
fix(connectors): activate plugin connectors with agent grants and no sign-in (#22201)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins can now add connector types (#22004) and serve their tools with the `plugin` transport (#22019) > - A plugin connector method can declare `grantKinds: ["agent"]` with `auth: "none"` or `"api_key"` > - Such a connector cannot reach its agent: the connect form cannot name the agent, connect creates no agent grant, and Paperclip Runner runs never list plugin-transport tools > - The product rule is that a plugin connector behaves exactly like any existing connector > - This pull request fixes the connect form, creates the agent grant at connect, and lets runner runs list plugin-transport tools > - The benefit is that an operator can activate a per-agent plugin connector, and only the chosen agent gets its tools ## Linked Issues or Issue Description Refs #22004 Refs #22019 **Subsystem affected** Connectors (connect wizard, tool access service), Paperclip Runner assigned-tool gateway. **Problem or motivation** A plugin connector with agent grants and no sign-in cannot be activated: 1. Connect form: it says "No sign-in needed, available to all agents". "Change" shows "No identity required" and "Which agent owns this GitHub account?", with no agent picker. Connect sends `grantKind: "agent"` without `subjectAgentId`, and the server rejects it with a validation error. The "Finish setup" resume form has the same problem. 2. Grant: connect with `subjectAgentId` saves a `per_agent` connection with zero grants. Finish adds the agent install, but there is still no grant. Only the OAuth callback writes agent grants, and these methods have no OAuth step. For `api_key` methods connect fails, because the health check finds no agent grant, and the key has nowhere to go. 3. Runner: a Paperclip Runner run builds its assigned-tool gateway from an allowlist of transports (`mcp_remote`, `local_stdio`, REST, Browser Use, GitHub bot, voice). `plugin` is not in it, so the run sees no plugin-transport tools, even when the install, profile and grant allow them. **Proposed solution** Create the agent grant at connect for curated methods without sign-in. Add `plugin` to the runner assignment and its digest. Show a one-agent picker with connector-neutral copy. **Expected behavior** Connecting alone grants no tools. Finish with agent A gives only agent A the tools, in gateway discovery and in a runner run. The connection has exactly one active agent grant, for A. ## What Changed - `server/src/services/tool-access.ts`: `connectGalleryApp` writes an active agent grant for `subjectAgentId` when a curated method with `auth` `none` or `api_key` uses `grantKind: "agent"`. The API key goes on that grant, not on the connection. A resume without `subjectAgentId` keeps the agent of the existing grant and keeps its saved key. A second agent on the same connection is a 409 conflict. The grant write is audited and joins the existing revive rollback. - Finish does not create agent grants. Finish carries no credential, and it can list many agents while a per-agent connection has one identity. A credential-less grant would also claim a connected OAuth identity that does not exist. The grant comes from connect (or the OAuth callback). Finish creates only the install. - `server/src/services/heartbeat/run-preparation.ts` and `native-runtime/runtime-context.ts`: include the `plugin` transport in the runner assignment and in its digest (the two lists must match). - `ui/src/features/connections/ConnectionSetupFlow.tsx`: for an agent grant, show the agent picker (exactly one agent), keep the install choice on "specific", open the "Change" panel and disable Connect until one agent is chosen, and send `subjectAgentId`. Copy is neutral ("Which agent uses this connection?"); the GitHub copy stays for GitHub. The summary no longer says "available to all agents" for an agent grant. Finish sends the chosen agent, never `all_agents`, for an agent grant. - `doc/connections/CONNECTOR-PLAYBOOK.md`: document agent grants for methods without sign-in. ## Verification Each new test failed before its fix and passes after it: - `server/src/__tests__/plugin-connector-agent-grant.test.ts` (embedded Postgres, real routes and gateway): - `auth: "none"` + agent grants: after connect, no agent has tools; after finish with agent A, A's discovery lists the tools and B's lists none; exactly one active agent grant, for A. (Red: zero grants.) - `auth: "api_key"` + agent grants: same, the key is on A's grant, and a tool call receives it. (Red: connect returned 422.) - A second agent on the same connection is a 409. (Red: 201.) - Resume of an API-key draft without the key keeps the key on the grant. - Paperclip Runner session: `buildPaperclipRuntimeMcpServers` + `createAssignedMcpTools` list the installed plugin tools for A, and search by the manifest tool name (`update_record`) finds the gateway tool. B gets none. (Red: no tools.) - Regression: an organization-grant method keeps one default organization grant, and both agents get the tools with `all_agents`. - `ui/src/pages/apps/AppsConnect.test.tsx`: for `none` and `api_key` agent-grant plugin methods, the form shows the picker, has no GitHub or "available to all agents" copy, blocks Connect until an agent is chosen, and sends `subjectAgentId`. (Red: "available to all agents" shown.) - Regression suites: `plugin-connectors-plugin-transport`, `plugin-connectors-circleback-parity`, `heartbeat-runtime-mcp-servers`, `run-preparation`, `tool-access-service`, `tool-access-oauth-protocol.integration`, `generic-mcp-connection` (OAuth, GitHub per-agent and organization flows) pass. UI: `src/features/connections`, `src/pages/apps`, `App.deep-link-routing` pass (66 files, 1103 tests). - `tsc --noEmit` passes for `server` and `ui` source. `pnpm check:module-boundaries` passes. ## Risks - Low. The grant change applies only to curated methods with agent grants and no OAuth. Organization, personal and OAuth flows keep their code paths. - The runner change makes plugin-transport tools visible to runner runs. Access still needs an install, a profile and a grant, and every call is re-authorized by the gateway. - Connections created before this fix have no agent grant. Connecting again (or "Finish setup") with the agent creates it. - No migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a7ea2a799a |
fix(ephemeral): save retained runtime skill bundles at startup and never strand a volume-free server on lost ones (#22190)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A volume-free server keeps no local volume. It must find every file it needs in durable storage. > - Before a volume-free server starts, a data gate refuses to start if records still point at files on a local disk. > - Old native runs can keep runtime skill bundles that a release before #19729 wrote only to its own disk. The gate reports these as `runtime_skill_assets_require_migration`. > - On a volume-free server those files are not on the disk, and no volume can be attached again. So the refusal repeats on every restart, and nothing can clear it. A staging stack crash-looped like this and stayed down. > - This pull request makes the server copy every retained bundle that is still on its disk into durable storage, and start over the bundles that are gone, the same way #20990 handles harness backups. > - The benefit is that a volume-free stack can always start again. No one has to repair it by hand. ## Linked Issues or Issue Description Refs #19675, #19729, #20990 **What happened?** A volume-free staging stack ran for twelve days on a release that kept runtime skill bundles only on its container disk. When that container stopped, every newer release crash-looped at boot with `Volume-free server data is not eligible: runtime_skill_assets_require_migration`. The old container was gone, so the bundles were gone. The deploy could not roll back, and a retry could not help. **Expected behavior** A volume-free server starts. It saves each retained bundle that is still on its disk to durable storage. It starts over a bundle that is not on its disk when the volume-free owner marker proves that no volume is mounted, and restart recovery owns those runs. **Steps to reproduce** 1. Retain a native run with an admitted runtime skill bundle that has no version ID and no durable preservation receipt. 2. Remove the bundle from the instance root. 3. Start the server with `PAPERCLIP_EPHEMERAL_SERVER=true` and the volume-free owner marker present. 4. Startup fails with `runtime_skill_assets_require_migration`, on every restart. ## What Changed - `runtime-skill-preservation.ts`: - The obligation scan is one generator. `hasLocalRuntimeSkillObligations` uses it. - New `preserveLocalRuntimeSkillObligations` copies each obligated run's bundles with `preserveRuntimeSkillAssets` and returns `{ bundles, unsaved }`. - A missing source is decided at its reader: `readLocalManifest`/`readSource` throw `RuntimeSkillSourceAbsent` (other instance root, or a missing directory, manifest or listed file). Only that error marks a run unsaved. Any other failure, including an `ENOENT` from storage or cache work, propagates. - `preserveRuntimeSkillAssets` no longer aborts on an invalid or non-durable existing receipt. It copies the local bytes again and replaces the receipt. - One durability helper (`preservedAssetRowDurable` for metadata, `preservedAssetDurable` with a metadata or object read) replaces the copies in the scan, staging, locking, preservation and rehydration. - `runtime-skill-preservation-reference.ts`: the scan matches only null pins, which is exactly what preservation copies. A non-null pin is a version pin and cannot become an obligation that nothing can settle. - `file-manifest-cache.ts`: new `inspectFileManifest` returns `verified`, `missing` or `changed` from the same walk. `verifyFileManifest` wraps it. - `ephemeral-server-contract.ts`: the gate saves the bundles still on the disk. If none stay unsaved, the blocker clears. Unsaved runs are waived only when the volume-free owner marker is present, with a warning. A save failure still refuses. The decision uses the copy pass result, not another scan. One exported blocker constant. - `ephemeral-migration-data.ts`: offline enrollment accepts the blocker and copies the bundles from the attached volume, in a phase after the package phases. Any unsaved run still fails the enrollment. - `index.ts`: comment update only. - Tests: `ephemeral-server-runtime-skill-startup.test.ts` (real scan, real copy, embedded Postgres) and an S3-gated case in `ephemeral-migration.test.ts`. ## Verification - `cd server && npx vitest run src/__tests__/ephemeral-server-runtime-skill-startup.test.ts` - Red on `origin/master`: 2 of 3 fail with the production error `Volume-free server data is not eligible: runtime_skill_assets_require_migration`. - Green with this change: all cases pass (lost bundle, saved bundle, no marker, mixed present and lost, one missing listed file, non-version pin, mixed pins, non-durable receipt recapture, storage failure, storage ENOENT, unsaved set). - `cd server && npx vitest run src/__tests__/ephemeral-server-startup-run-state.test.ts src/__tests__/runtime-skill-preservation.test.ts` pass (existing gate and preservation behavior unchanged). - `cd server && npx tsc --noEmit -p .` passes. - `ephemeral-migration.test.ts` needs `PAPERCLIP_TEST_S3_ENDPOINT`. It typechecks, but it was not run here. ## Risks - A run whose bundle is lost now starts under restart recovery instead of blocking the whole server. Recovery of that one run can fail. Before, the whole server stayed down, and the bundle was lost anyway. - Startup does more work only when unsaved legacy bundles exist. A current release saves bundles before admission (#19729), so new runs never add this work. - No schema change. No API change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1c556d5340 |
fix(native): never rest a task blocked on the agent's own blocker (#22140)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run ends with one semantic report. A `paperclip_block`
report names a blocker, its owner and the unblock action
> - The status arbiter binds a task-wide blocker. When the owner is not
a person, it binds the blocker to the same agent and wakes that agent
> - The dispatcher refuses those self-wakes
(`chat_control_recovery_proof_unresolved`), because only continuation
and retry effects are proof for a native wake. The task then rests
blocked with no owner, no wake and no Retry
> - #21959 continues a system blocker in a fresh session, but only for
the reason codes `environment_unavailable` and `tool_unavailable`. A
restart-chaos rerun showed the same dead end with the reason code
`tool_output_failure`
> - This pull request makes sure that a blocker that the agent reports
for itself always has a next owner: the automatic environment lane, one
bounded self-continuation, or a person with a Retry that works
> - The benefit is that no task rests blocked with nobody to act, and no
path wakes the agent in a loop
## Linked Issues or Issue Description
Refs #21959
**What happened?**
A native OpenCode run (DeepSeek) said that every tool call returned the
text "cryptic". It reported `paperclip_block` with owner `{kind:
system}`, scope `task_wide`, reason code `tool_output_failure`, and an
unblock action about the tool-output failure. The run succeeded with
disposition `blocked`. The arbiter bound the blocker to the same agent
(`task_wide_blocker_bound`). The `notify-owner` wake coalesced, and the
`blocker-owner` wake was skipped as
`chat_control_recovery_proof_unresolved`. The task rested blocked
(`needs_attention`, 0 blockers), with no Retry and no wake. An earlier
rerun showed the same shape with reason code `environment_unavailable`,
which #21959 fixed for those two codes only.
**Expected behavior**
A blocker that the agent reports for its own tools or environment is a
harness problem. Paperclip continues it automatically, with a bound. A
blocker that the agent says it owns gets one fresh session. A blocker
that needs a person goes to a person, with a clear prompt and a Retry
that starts the task again. Nothing is left with nobody.
**Steps to reproduce**
1. Run a native task. Make the agent call `paperclip_block` with owner
kind `system`, scope `task_wide` and a reason code other than
`environment_unavailable` or `tool_unavailable` (for example
`tool_output_failure`).
2. See the decision `task_wide_blocker_bound` with the agent as the
unblock owner.
3. See both owner wakes refused, and the task blocked with no Retry.
**Paperclip version or commit**
`master` at
|
||
|
|
43b2d8be22 |
feat(dashboard): add an experimental areas and responsibilities dashboard (#22069)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The dashboard is the first page a person sees in a company > - The classic dashboard shows agents, runs, costs and recent tasks > - It does not show what the person is responsible for, or if that work goes well > - People think in areas, such as Fundraising or Customers, and in the things they own in each area > - This pull request adds a dashboard built on areas and responsibilities, behind an experimental setting that is off by default > - The benefit is that a person sees what they own, how it is going, and what agents do for it, on one page ## Linked Issues or Issue Description No public issue exists for this work. The description follows the feature request template. **Subsystem affected** Dashboard, navigation, the task conversation, routines, and the instance experimental settings. **Problem or motivation** The classic dashboard counts agents and tasks. It does not say what a person owns or if it goes well. People who run a company with agents want to see their areas, the responsibilities in each area, the bar each one must meet, and the agents and routines that serve it. **Proposed solution** A person owns responsibilities. Each responsibility belongs to an area and has a bar, a number to reach. Projects, tasks and routines link to a responsibility, and the page shows its status from those links. An agent can interview the person and draft the dashboard. The person accepts or declines each change, and any change can be undone from a saved version. **Alternatives considered** Add the same data to the classic dashboard. That page is a fixed set of cards for company activity, so a second, person-centred page is clearer. The setting lets a person pick one of the two. **Roadmap alignment** This is an experimental feature behind `enableResponsibilities`. It is off by default and changes nothing for a person who leaves it off. ## What Changed - Add the setting "Areas & Responsibilities dashboard" (`enableResponsibilities`, off by default). On, Dashboard in the sidebar opens the new page, and the classic dashboard link is not shown in the sidebars or the command palette. Off, nothing changes. - Add tables and migrations 0492 to 0497 for areas, responsibilities, links to work, metrics, status reports, saved dashboard versions, and model-call records for summaries. - Add the API: areas, responsibilities, links, bars, metrics, status reports, summaries, runs, automation, and the dashboard conversation with versions and undo. Each route checks company access and writes to the activity log. - Add four task conversations with an agent: about a bar, about an automation, about a routine, and about the whole dashboard. The agent proposes a change. Nothing applies until a person accepts it. - Add standing checks: routines that an agent carries out on its heartbeat and reports. - Add the Dashboard page: search, counts, areas that open and close, a primary New area button, and a message box at the bottom of the page. Add the responsibility page with its bar, metrics, work and routines. - Add the Paperclip skill sections and runner API entries that let an agent use these routes. The runner skill inventory is regenerated (166 to 174 rows). - Keep the new runner API tests in their own file so `runner-api.test.ts` stays at 985 lines. - Onboarding is not changed. The onboarding changes that build on this work are a separate pull request. ## Verification - Type checks pass for `packages/db`, `packages/shared`, `server` (all layers) and `ui`. - `pnpm --filter @paperclipai/db check:migrations` passes. A new `drizzle-kit generate` run produces no further change. - Tests for every changed file pass: 26 server test files (3,009 tests), 116 UI test files (1,991 tests) and 28 shared test files (432 tests). - The runner capability inventory checks pass. - I started the app on a local database that already held earlier data. Master's new migrations and these migrations applied without error. With the setting on, the Dashboard and a responsibility page loaded with no console or server errors. The More menu and the command palette show one Dashboard. - I did not run `pnpm verify`, the full local suite, or the browser suites. Required CI covers them. - The agent conversations are tested with simulated agents. No real model has run them. ## Risks - Medium. The change is large, and the setting is off by default. With it off, the new routes refuse requests and the UI is unchanged. - Migrations only add tables and columns. Two replace a CHECK constraint, and one builds an index `CONCURRENTLY`. These two have the `online_compatible` override with a reason in the file, and the second one is listed in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`. - Master can add migrations before this merges. The migrations then need new numbers. - `responsibilities.ts` (1,663 lines), `responsibility-summaries.ts` (1,239 lines) and one Storybook fixture file (1,305 lines) are new files over 1,000 lines. They each keep one cohesive unit together: the service for one table family, the model-call path for summaries, and the fixtures that the stories share. No existing file crosses 1,000 lines. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude: Claude Fable 5.2 (`claude-fable-5-2`) wrote most of the change, with Claude Sonnet 5.5 (`claude-sonnet-5-5`) for the final steps. Both used tool use, code execution and browser checks. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
6902c3f193 |
fix(native): keep recovery workers off a run its live heartbeat is finishing (#22101)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A native run ends in two steps: the live heartbeat exports the workspace, then it commits the accepted result and records the run's exit code, log summary and usage > - Two recovery workers can also do these steps: the workspace finalization worker and the result commit worker > - The runner posts its accepted result before its process returns. That post wakes both workers at once > - The workspace finalization worker then takes the export lock while the heartbeat still waits for the runner. The heartbeat gets "busy", stops, and leaves the run to the workers > - On staging, 45 of 135 recent succeeded native runs had no exit code and no log size because of this > - This pull request makes the workers leave a run alone while its live heartbeat in the same process still runs > - The benefit is that the heartbeat finishes its own run again, with exit code, logs and usage. The workers still recover a run that the heartbeat leaves unfinished ## Linked Issues or Issue Description Refs #22043 **What happened?** About a third of succeeded native runs had a null `exit_code` and a null `log_bytes`. Their traces show a `task.workspace_finalization` span (only the workspace finalization worker makes it) and no `execution.native_finalize` or `completion.*` spans. The live heartbeat lost the workspace export lock to the worker, threw `NativeWorkspaceFinalizationBusyError` and stopped without completing the run. **Expected behavior** The live heartbeat exports the workspace, commits the result, and records the exit code, log summary and usage. The workers act only when the heartbeat cannot finish the run. **Steps to reproduce** 1. Start a native run that syncs its workspace. 2. Let the runner post its accepted result before its process returns. The result post wakes the workspace finalization worker at once. 3. The worker exports the workspace first. The heartbeat gets `acquired: false` and stops. The result commit worker then commits the run without the heartbeat's completion data. **Relevant logs or output** Staging run `cb1cfc23-79eb-4528-bcdc-dcf639430d31`: the worker restored the workspace in about 51s. The worker then committed the status decision and projected the run as succeeded. The heartbeat wrote no completion data. ## What Changed - Add `server/src/services/native-runtime/live-native-finalization.ts`. It keeps an in-process reservation for each run whose live heartbeat is still running. - The heartbeat (`runtime-execution.ts`) reserves a native run before it dispatches. It releases the reservation when its execution ends. The release wakes both worker queues for that run. - `withNativeWorkspaceFinalizationOwnership` returns `owner_active` to every caller except the run's own heartbeat while the run is reserved. The workspace finalization worker already waits for a release notification on `owner_active`. - The result commit worker skips a reserved run. The release notification wakes it again. ## Verification - New regression tests, red before the fix and green after: - `native-workspace-finalization-work.test.ts`: "leaves a result posted mid-run to the live heartbeat, then resumes after its release". Without the fix the worker exported the workspace while the heartbeat held the run. - `native-result-commit-work.test.ts`: "leaves a live heartbeat's run to it and commits only what the heartbeat leaves behind". Without the fix the worker committed the run while the heartbeat held it. - Both tests also prove that the workers still finish the run after the heartbeat releases it without finishing. - `pnpm verify` (affected tests) passes. Server typecheck passes. - After deploy: measure the share of succeeded native runs on staging with a null `exit_code`. It should go down toward 0%. ## Risks - The reservation is in memory. A worker in another server process (for example during a deploy overlap) can still race the heartbeat. That case behaves as before this change. - If the heartbeat stops in an unusual way, its `finally` block releases the reservation and wakes the workers. If the process crashes, the reservation goes away, and startup discovery recovers the run as before. - While a heartbeat is still running after the runner posted its result, the workers wait. Before, they could start the export at once. The heartbeat does the same work, so the result reaches the user at about the same time. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ea154ce55e |
Add bulk keyed issue document reads for native helpers (#22105)
## Thinking Path
> - Paperclip manages work for AI agents.
> - Native helpers read task documents through authorized API tools.
> - A scan of 146 cases needs one saved response and one download per
case.
> - The document service can read the same key across authorized tasks
with bounded queries.
> - This pull request adds a bounded bulk read with the existing task
access policy.
> - Helpers can transfer one batch and check current revisions before
fetching changed bodies.
## Linked Issues or Issue Description
**Subsystem affected**
Task document REST API, shared query contracts, and native API
discovery.
**Problem or motivation**
Per-task document reads make native sync scans cost hundreds of tool
calls. Task list timestamps do not prove that document content is
unchanged.
**Proposed solution**
Add `GET
/api/companies/{companyId}/issue-documents?key=notes&issueIds=...`. Read
up to 200 task UUIDs under current company and task permissions. Return
current bodies and revision metadata, or metadata alone with
`includeBody=false`.
**Alternatives considered**
Generic request batching would add tool execution and partial failure
rules. One bulk resource read removes the per-case relay cost with the
existing tools. Do not skip documents based only on task-row timestamps.
Keep managed skill files immutable; cleanup of copied skill test
fixtures belongs to those tests.
**Roadmap alignment**
This improves existing task documents and native API access. It adds no
new workflow or UI. The roadmap has no duplicate bulk document read
work. No duplicate open pull request was found.
## What Changed
- Add a bulk keyed document route and a company-scoped document reader.
Reuse the current batched task-read policy and completed-task read
receipts.
- Preserve request order and deduplicate IDs. Return `document: null`
for readable tasks without the key. Omit unreadable, hidden, foreign and
missing tasks without revealing which exist.
- Bound requests to 200 IDs and responses to 8 MiB of JSON bytes. Check
encoded body sizes in SQL before fetching bodies in the same read
snapshot. Build JSON within the byte budget. Oversized responses fail
with 413 and are never saved as partial artifacts.
- Admit at most two active reads per actor and eight per server
instance. Extra requests receive 429 with Retry-After. Keep the slot
until both the database work and response settle.
- Batch completed-handoff lookups and group child receipts by parent.
Extend the existing event batch writer for ordinary non-waking receipts
with atomic sequence allocation. Stored receipt shapes stay unchanged
for existing consumers.
- Expose revision-only reads without selecting document bodies. Document
native discovery, one save plus one download, and complete-inventory
checks.
- Add real HTTP, database and artifact tests for 146 open and completed
cases, 200 maximum-size bodies, metadata overhead, access failures,
query limits and JSON expansion. Cover admission limits, failure
cleanup, and concurrent event writers. Refresh the shared SDK source
pins.
## Verification
Final revision: `93dc8b029600211d79a050650f3dff82e5dad52f`.
- [CI run 44212220917, attempt
3](https://github.com/paperclipai/paperclip/actions/runs/44212220917)
passed. This includes typecheck, build, general tests, Runner checks,
serialized server suites, all e2e shards, and the final verify gate.
-
[Build](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134506305267),
[e2e](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134506309117),
and
[verify](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134510062263)
are green.
- [Linux CI job
134506309291](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134506309291)
passed all 16 bulk document integration tests. This includes the native
transfer of all 146 documents with the saved size and hash,
completed-case receipt batching, and size-only rejection of 200
maximum-size documents.
- [Greptile
review](https://github.com/paperclipai/paperclip/pull/22105#issuecomment-6965449032)
is 5/5 on the final revision. All review threads are resolved. Security
checks pass. GitHub reports CLEAN and MERGEABLE.
- Local server typecheck passed on macOS with Node 25.6.1. The focused
run below passed 177 tests; the Linux-only transfer test was skipped
locally and then passed in CI.
```sh
pnpm --filter @paperclipai/server typecheck
cd server
pnpm exec vitest run \
src/routes/issue-document-batch.test.ts \
src/services/native-runtime/issue-document-batch.integration.test.ts \
src/__tests__/documents-service.test.ts \
src/__tests__/openapi-routes.test.ts \
src/__tests__/heartbeat-run-event-atomic-append.test.ts \
src/__tests__/chat-completion-delivery.test.ts \
src/__tests__/native-child-task-read-routes.test.ts
```
- Shared typecheck and build passed. From `packages/shared/`, `pnpm exec
vitest run src/validators/issue.test.ts` passed 61 tests before the
rebase; the batch query contract did not change in the rebase.
- `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`
passed. Runtime and lockfile pins are unchanged.
- `pnpm verify --explain` returned exit 2 because the API docs and
shared contracts require broader CI. The successful CI run above
supplies that coverage.
Failure and recovery record:
- Both performance findings reproduced before the fixes. Scale tests now
verify a size-only query for 200 large documents and one handoff lookup
plus two receipt-write statements for 146 completed cases. Concurrent
writers retain contiguous unique sequences and decoded payloads.
- Initial local runs found missing shared exports and OpenAPI route
coverage. These are fixed. Fixtures now respect Linux-only installation,
thrown cross-company denials, and the ES2022 TypeScript target. Rebase
conflicts preserved the onboarding coverage from master.
- CI attempt 1 interrupted Build, e2e shard 7 and six general-test
shards with runner shutdown signals. Attempt 2 recovered the tests but
its [Build
job](https://github.com/paperclipai/paperclip/actions/runs/44212220917/job/134499680425)
also received a shutdown signal and exited 143. No compiler or test
assertion failure was found in those interrupted job logs. The reason
the runners stopped remains unknown. Attempt 3 recovered Build and its
dependent verify gate on the same revision. Earlier interruptions are
not counted as passes.
## Risks
- Additive API only. No migration or existing endpoint response change.
- Helpers must adopt the batch response to gain the tool-call reduction.
They must verify that every requested task is present before treating
the scan as complete. Unreadable tasks are intentionally omitted.
- Split large document batches when the route returns 413. Metadata
remains available without bodies. Concurrency limits are per server
instance and can refuse simultaneous reads with 429; callers retry after
an active read finishes.
- This does not change company-owned helper inventory expiry or cleanup
tests, installed skill permissions, or generic tool batching. It does
not deploy to a tenant.
## Model Used
OpenAI `gpt-6.1-sol` through Codex in T3 Code, with xhigh reasoning,
tool use, and code execution. Context window size was not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
|
bf55063567 |
fix(server): keep chat titles current and space verification retries (#22102)
## Thinking Path > - Paperclip manages AI agents and their work. > - Agent chats use persistent tasks, and remote runs verify their provider files before execution. > - A chat keeps the name that its agent had when the chat was created. > - A fast gateway failure can consume both verification attempts before the gateway recovers. > - This change updates generated chat titles on an agent rename and waits before a verification retry. > - Chats show the current name, and a short gateway fault has time to clear. ## Linked Issues or Issue Description **What happened?** Renaming an agent leaves existing generated chat titles unchanged. Remote provider verification also retries a transport failure immediately. Two fast HTTP 502 responses can fail startup before the model starts. **Expected behavior** Generated chat titles follow an agent rename. A read-only verification retry gives the remote gateway time to recover. Verification still rejects incompatible files and stops after two attempts. **Steps to reproduce** 1. Open a chat with an agent, then rename that agent. Inspect the task linked from a run: its title still has the old agent name. 2. Make the remote verification transport return HTTP 502 for one second. Start verification. Both immediate attempts fail before the transport recovers. Related: #22087 covers a provider connection failure during lease acquisition. This change covers a later provider-file verification request. ## What Changed - Update matching generated chat titles in the agent configuration transaction. Preserve custom chat titles, ordinary task titles, and other agents' chats. - Record each changed chat as `issue.updated` and publish its event after commit so open pages refresh. Roll back the rename and queued events if audit persistence fails. - Derive new chat titles from the current agent name under the existing company transaction lock. Both user-opened and proactive chats use this path, including a create that read the old name before a rename committed. - Acquire that company lock before proactive chat delivery takes chat or agent row locks, so delivery and renames use a consistent lock order. - Wait one second before the second remote verification attempt. Keep the existing attempt limit, transport timeouts, artifact checks, and failure codes. - Add route and startup-verification regressions. Document the retry delay. - Wait for the connection-review E2E fixture to save its Ask first permission before it creates a task or navigates away. ## Verification - Final acceptance on `3fb74ffaf6fa0ee2ecbe5dba3d01d252cb0c986f`: [full CI run](https://github.com/paperclipai/paperclip/actions/runs/44208144377) is green, all 61 reported checks are complete with no failures, Greptile is 5/5 on this exact commit, all review threads are resolved, and GitHub reports the branch clean and mergeable. - Red/green: the rename test retained the old titles before the fix. The gateway recovery test consumed both attempts before the one-second recovery point. Both pass with the changes. - The live-update and concurrent-create tests failed on the first PR revision: no title events were published, and a paused chat creation retained the old name. The fixes queue events for publication after commit and re-read the agent name under the company lock. Both reproductions now pass. - `pnpm exec vitest run server/src/__tests__/agent-configuration-save.test.ts server/src/__tests__/agent-chat-messages.test.ts server/src/__tests__/agent-conversations.test.ts server/src/services/native-runtime/remote-provider-pack.test.ts server/src/services/native-runtime/remote-provider-pack-verification.test.ts`: 191 passed on macOS, including the concurrent rename/message regression. - `pnpm --filter @paperclipai/server typecheck`: passed, including source and test layers. - Earlier CI on `de2a0a4` failed in the release-registry watchdog self-test ([job log](https://github.com/paperclipai/paperclip/actions/runs/44202511888/job/134460224014)). Its 1.5-second watchdog fired during `starting the file's tests`; the assertion expected `test "waits forever"`. Both reporter files are unchanged from the base and current head. `node --test scripts/ci/stalled-file-reporter.test.mjs` passed all 7 tests on macOS with Node v25.6.1. The current revision also passed the Linux Typecheck + Release Registry CI check; no reporter code was changed. - CI on `c83fcd9` failed the connection-review failure journey ([shard log](https://github.com/paperclipai/paperclip/actions/runs/44204043415/job/134465938135)). Navigation aborted the pending permission save, so the fixture tool ran without creating the expected approval card. A temporary two-second delay on that save reproduced the failure locally. With the response wait, the same delayed-save reproduction passed. The temporary delay is removed from the diff. - `node scripts/typecheck-test-projects.mjs --reuse-declarations tests/e2e`: passed using the server declarations from the same application source revision. - Before the rebase, the complete connection-review spec passed 4 journeys and skipped the controlled-restart journey under the default configuration. On `95fb2aa`, rebased onto `9032f55d83e`, the provider-failure journey, all 190 focused tests, server typecheck, and E2E test-project typecheck passed again. The dedicated controlled-restart suite was not run. - A concurrent rename/message regression reproduced PostgreSQL `40P01: deadlock detected` on `95fb2aa`. Delivery now takes the shared company lock before the chat and agent locks. The regression uses observed PostgreSQL blockers to stage the collision, and both operations commit with the fix. Concurrent creation also waits behind the same lock and retains the renamed title. - The application tree for `3fb74ff` passed all 191 focused tests and server typecheck. The subsequent rebase onto `96907f5518c` changed only `.github/CODEOWNERS` compared with that tested tree. The full CI run for this revision is now green. - CI jobs on `95fb2aa` were interrupted after their runners received shutdown signals: [server shard](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474311777), [workspace shard](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474311921), [typecheck/release registry](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474310521), and [serialized server shard](https://github.com/paperclipai/paperclip/actions/runs/44206614135/job/134474311690). The typecheck job is labelled failed, but its active step was cancelled. These interruptions did not supply passing test evidence. The fresh CI run on `3fb74ff` completed without failures. - `pnpm verify --explain`: exit 2. The changed recovery document requires broad CI coverage. Required CI supplies the full test and build coverage; no full local suite or build was run. - `git diff --check` and a Gitleaks scan of the diff passed. The diff was also checked for personal data and private identifiers. ## Risks - An inconclusive verification takes one extra second before its last attempt. A persistent outage still fails with the existing named error. No provider turn or artifact mutation is replayed by this retry. - Only chat titles that match the previous generated title change. Earlier stale titles and custom titles need an explicit correction. - Proactive chat delivery now shares the existing company writer lock with agent configuration. Concurrent writes in that company wait for the active transaction to finish. - No schema, session, checkpoint, launch identity, or pack identity changes. State saved by the previous release carries forward with the same verification checks and attempt limit. - Revert this commit to restore the previous behavior. Existing title updates are metadata and need no schema rollback. ## Model Used - OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning, with source inspection, browser investigation, and local code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
f05b90508a |
refactor(core): remove retired compatibility paths (#22055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core manages chat authorization, AI connection defaults, navigation, and the plugin host. > - Core still writes old default rows and checks an Agent Chat flag that always reads true. > - It also carries a desktop rail that cannot open and private aliases with canonical equivalents. > - This pull request removes those paths and moves their tests onto the paths Core uses. > - The result is fewer Core database operations and less code to maintain. ## Linked Issues or Issue Description **What existing behavior does this improve?** Core AI default writes, Agent Chat execution and delivery, private plugin host and Runner setup, and board navigation. **Subsystem affected** Cross-cutting. Core server, board UI, private Runner helpers, and a repository guard script. **Current behavior** The server writes provider-scoped and per-method AI defaults, then reads only the provider-scoped rows. It reads settings to decide whether to block Agent Chat, although every settings view enables it. The desktop sidebar renders expanded while still tracking requests to open and close a retired icon rail. The agent client retries conflicts that the server already resolves when there is exactly one active match. **Proposed behavior** Write provider-scoped defaults directly. Use the existing Agent Chat authorization and retry checks without the redundant settings reads. Render the current sidebar directly and let the API report true agent reference conflicts. **Reason and benefit** A personal non-custom connection save or explicit default selection sends one fewer SQL statement. It also avoids the provider-default write that the legacy trigger performed when the old statement inserted or updated a row. A successful task-to-chat send or receipt replay avoids two transaction settings SELECTs when the singleton row already exists. Conversation comment, wake release, and retry paths also avoid a transaction settings read when they previously entered the gate. Savings in other settings paths depend on cache hits. Core-to-plugin RPC and plugin-to-Launcher HTTP traffic stay the same. **Breaking changes** The public plugin SDK, host navigation, and organization-switcher fields retain their existing interfaces and behavior. The settings API still accepts and reports the Agent Chat compatibility key. This change needs no schema migration or rollout ordering. **Additional context** Refs #10610 for prior reporting on the retired sidebar rail. Refs #21756 for adjacent Coordinator plugin slots, which remain a separate change. ## What Changed - Stop writing per-method AI defaults. Preserve the provider-scope upsert and first-account behavior. Keep the existing tables and migration trigger. - Remove Agent Chat checks that read a constant true value. Keep conversation ownership, task scope, session, and request-retry checks. - Remove unused lock-owner fields and private plugin lifecycle, secret extraction, Gmail connector, PRP setup, heartbeat resolver, and ACPX aliases. - Use a single API request for agent references. Remove retired sidebar rail state, hooks, and branches across the active and production snapshot layouts. Preserve desktop resizing, mobile drawer controls, contextual navigation, and public plugin slot inputs. - Remove unused FeedCard and AgentDetail inputs and the test-only Git push matcher alias. - Keep the cleanup that erases historical plaintext chat retries from browser storage. Production source in the touched files is 106,800 → 105,633 nonblank lines, down 1,167. The production Git diff adds 251 physical lines and removes 1,480, down 1,229. Those figures exclude tests, fixtures, docs, and Storybook. The entire Git diff adds 482 physical lines and removes 2,325, down 1,843. ## Verification - Merged master at `b56f240f45` into the PR branch as `f73fea4009`. The five sidebar files retain the expanded navigation cleanup and master’s explicit session-replay masking contract. The merge also preserves master’s account-repair recovery guard and GitHub star-prompt behavior. - Focused UI Vitest run passed 361 tests across 11 files. This covers analytics privacy and replay masking, sidebar and layout, GitHub star prompt, and agent detail. - Focused server Vitest run passed 45 tests across three files. This covers restored configuration handling, conversation recovery, and Agent Chat authority and receipt replay. - `pnpm --filter @paperclipai/ui typecheck`, the Git push guard and its 14 tests, and `git diff --check origin/master...HEAD` passed. - `pnpm verify --base origin/master` classified the Git-push guard as a global CI input and returned exit 2. GitHub CI supplied the broader coverage. - GitHub CI and Runner image checks passed on `f73fea4009`. Both required checks, `ci / verify` and `ci / e2e`, passed. [CI workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261752) · [Runner image workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261413) - Standard Greptile reported 5/5 with no actionable findings or open review threads. [Greptile review](https://github.com/paperclipai/paperclip/pull/22055#issuecomment-6959078925) - The optional Superagent Security Scan finished neutral with an inconclusive workflow result and zero annotations. The final rollup had 61 successes, five skips, and one neutral, with no failed or pending checks. [PR checks](https://github.com/paperclipai/paperclip/pull/22055/checks) ## Risks Agent Chat authorization and retry are sensitive to small query changes. The final diff keeps the conversation agent field and all existing authorization and idempotency checks. The tests cover revoked defaults, legacy migration reads, bound-listener authority, and mobile and desktop navigation. Installations that still run software old enough to read the per-method AI table will no longer see new defaults in that table. Current Core reads the provider-scoped table. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with source inspection and shell-based tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
73dc33c620 |
fix(runner): keep a disk reserve so a full sandbox disk does not strand the runner (#22068)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents in a Paperclip-managed sandbox run under the Paperclip runner (`runnerd`), which keeps durable state on the sandbox disk > - An agent can fill that disk with its own work. In one case the agent installed a Rust toolchain and a second checkout with dependencies on a 10 GiB sandbox > - `runnerd` then could not write its state and exited. Each relaunch tried to upload its launch files to the full disk and failed, 83 times in 7 minutes > - The run ended as a generic provider failure that asked a person to inspect it, and Retry repeated the same failure > - This pull request keeps a small disk reserve that belongs to Paperclip. Recovery releases it to relaunch the runner, and stops at once with a named code when nothing is left to free > - The benefit is that the agent's turn can continue and see the full disk in its own tool output, instead of the run dying with it ## Linked Issues or Issue Description **What happened?** An agent in a managed Daytona sandbox filled the 10 GiB disk. `runnerd` exited with `failed to persist bounded diagnostic: No space left on device (os error 28)`. Runner process recovery then retried for the full 420000 ms reconnect grace. Every attempt failed with `.paperclip-upload-<id>.tar.gz: create: ... no space left on device`. The run failed as `native_provider_terminal_failed`, and the recovery asked a board user to verify that the provider stopped. **Steps to reproduce** 1. Run an agent in a managed sandbox and have it fill the disk (for example a Rust toolchain plus a full dependency install). 2. `runnerd` exits when it next writes its state. 3. Watch recovery retry the relaunch upload until the grace expires. **Expected behavior** Paperclip relaunches its runner on a disk the agent filled, so the turn continues and the agent sees the full disk in its tools. When the runner truly cannot start, recovery says so at once with a named cause instead of retrying for 7 minutes. **Environment** Managed Daytona sandbox (fleet snapshot, 4 CPU, 8 GiB memory, 10 GiB disk), `paperclip_runner` native runtime. ## What Changed - New `server/src/services/native-runtime/remote-runner-disk-reserve.ts` owns the reserve: hold/release scripts, the full-disk classifier, and `createRemoteRunnerDiskReserve`. A release waits for an in-flight hold, and no hold starts after a release. - Both remote runner launchers (agent runs and Dot) get a reserve path in managed sandboxes only: `<workspace>/.paperclip-runtime/paperclip-runner/disk-reserve`. Workspace saves and Git already exclude `.paperclip-runtime`. SSH hosts get no reserve. - After the first launch's identity is confirmed, the launcher holds 256 MiB in the background, only while 2 GiB more is free. It is off the startup path. - A relaunch releases the reserve before it stages. A launch that hits a full disk while staging releases the reserve and stages once more. - Release tries twice and reports `released`, `not_held` or `failed`. Only `not_held` ends the launch with `sandbox_disk_full`. A release that cannot run rethrows the original error, so recovery still retries. - Command-based staging (no `syncIn`) keeps a full disk recognizable with an `ENOSPC` code instead of a bare `runner_remote_*_staging_failed`. - `SANDBOX_DISK_FULL_FAILURE_CODE` and `isSandboxDiskFullFailure` live in the runner package. Runner process recovery checks the error code and stops at once instead of retrying for the whole grace. Other staging failures still retry. ## Verification - Red before the fix, green after: - `server/src/services/native-runtime/remote-runner-disk-reserve.test.ts`: release-and-restage, release before relaunch, `sandbox_disk_full` when release frees too little, no retry for a failure that is not a full disk, hold/release scripts in a real shell, no hold below the free-space floor. Before the fix, the launch rejected with the exact upload error from the incident. - `packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts` "fails at once when a replacement cannot be staged because the sandbox disk is full": before the fix it was still retrying after 4 s; after, it fails with `sandbox_disk_full` after one replacement. - Kept: "retries a replacement whose staging failed before any process launched" still passes, so a raw transient staging failure still retries. - Updated: "confirms no replacement process exists when staging fails before the launch RPC" now expects `sandbox_disk_full` with the upload error as cause. Exit is still confirmed and no launch RPC runs. - Real sandbox from the same fleet snapshot (Daytona, 10 GiB): hold created a 256 MiB reserve. The disk was filled to 100%. A 5 MB write then failed. Release returned `true`, and a 50 MB write then succeeded. A second release returned `false`. Hold on the now 98% full disk did not create a reserve. - `pnpm --filter @paperclipai/server exec tsc --noEmit` passes. `pnpm verify` result is in a comment below. - Review round: tests for a lost release RPC, hold/release ordering, command-based staging on a full disk, the hold after a dispatched first launch (not on relaunch), and no reserve on SSH. Each launcher-level test failed with its fix removed. - Focused suites on the review revision: transport 335/335, launcher 130/130 (filtered), launch assets 35/35, reserve 14/14. Server and runner typecheck pass. - Not run: a full product run in which an agent fills the disk and the relaunched `runnerd` resumes the turn end to end. ## Risks - Each managed sandbox holds 256 MiB more disk while it has more than 2 GiB free. That is 2.5% of a 10 GiB sandbox. - A relaunch for any reason releases the reserve for the rest of that run. The next run's launch takes it again. - If the agent keeps writing after the relaunch, the disk can fill again. The second time, nothing is left to release, and the run fails at once with `sandbox_disk_full` instead of after 7 minutes. - The run's error code is still `native_provider_terminal_failed`. The message now names `sandbox_disk_full`. A dedicated failure code and recovery text are a follow-up. - `runnerd` loses its own error message when it cannot persist its diagnostic. This PR does not change the Rust binary. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3c5b117cca |
fix(native): post the agent's reply when the result commit worker finishes the run (#22043)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A person sends a message on a task, an agent run starts, and the
agent answers with a task comment
> - A native run normally finishes in its live heartbeat, which commits
the result and posts the reply in one pass
> - The result commit worker also commits native results. It races the
live heartbeat as soon as the workspace save succeeds, and it often wins
> - That worker skipped reply presentation and left the reply to the
presentation worker, which often posted it many minutes later
> - The task showed its new status (for example "blocked") with no
reply, so the person thought the message did not run
> - This pull request makes the result commit worker post the reply in
the same pass
> - The benefit is that the reply appears when the run finishes,
whichever path finishes it
## Linked Issues or Issue Description
**What happened?**
A person commented on a task. A run started one second later, and the
agent wrote its answer in about one minute. The run finished and the
task changed to "blocked", but no reply appeared for 8.5 minutes. The
person checked the task and saw no answer.
**Expected behavior**
The agent's reply appears on the task when the run finishes.
**Steps to reproduce**
1. Run a native agent on a task. The agent reports a result (for example
`blocked`) with a final reply.
2. The live heartbeat saves the workspace. The result commit worker
commits the result before the heartbeat's own finalizer does.
3. The run is "succeeded" and the task status changes, but the task has
no reply.
4. The reply appears only when the presentation worker later picks up
the run.
**Evidence**
On one staging company, 135 recent succeeded runs had a reply comment:
| Path that finished the run | Runs | Reply delay after the run ended |
| --- | --- | --- |
| Live heartbeat (reply posted inline) | 90 | 1–16 s |
| Result commit worker (reply left to the presentation worker) | 45 | 0
s – 3,864 s; 29 of 45 took over 30 s |
All 45 slow-path runs have no live completion write (no exit code, no
log size). Traces confirm that the live `execution.native_finalize` and
`completion.*` steps did not run for these runs.
**Paperclip version or commit**
`master` at
|
||
|
|
904c01d1a3 |
feat(mcp): graduate Assistant connections (MCP) out of experimental (#22015)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Assistant connections (MCP) let an outside assistant, or another
Paperclip instance, sign in as a person and use the `/mcp/paperclip`
endpoint
> - The feature was behind the experimental `enablePublicMcp` setting.
The setting was off by default and in practice off on Cloud stacks
> - Instance-to-instance work (one person's agent files a task on a
colleague's instance) fails when the receiving instance has the setting
off
> - This pull request moves the feature out of Experimental. It is now
always on, and a stored or Cloud-managed "off" value is ignored
> - The benefit is that every authenticated instance with a public URL,
including Cloud workspaces, accepts assistant connections with no setup
step
## Linked Issues or Issue Description
**Subsystem affected**
Public MCP (`server/src/services/public-mcp/`), instance settings,
Connectors → Assistant Connection (MCP) setup page, Experimental
settings page.
**Problem or motivation**
A remote MCP connector on one instance cannot use another instance's
`/mcp/paperclip` unless an admin on the other instance finds and turns
on an experimental toggle. On Cloud, the toggle is off and is managed by
the fleet. Instance-to-instance delegation therefore does not work by
default.
**Proposed solution**
Remove the gate. Keep `enablePublicMcp` only as a compatibility key, the
same way `enableApps` was graduated: old stored rows, PATCH bodies and
Cloud managed configs still parse, but the effective value is always
`true`.
**Alternatives considered**
An instance-admin kill switch that defaults ON. I did not keep one. Each
person already controls access: a connection exists only after that
person approves OAuth consent for one company, and the person or an
admin can revoke it. Deployment policy (authenticated mode and a valid
public URL) still decides whether the endpoint exists. A second switch
would add a state that can silently break instance-to-instance work
again. A schema migration was also not needed: the value lives in the
`experimental` JSON column, and the stored key becomes inert.
**Roadmap alignment**
Requested by the maintainer to enable instance-to-instance
communication.
## What Changed
- `packages/shared`: `enablePublicMcp` is a deprecated compatibility
key. The schema default is `true`. The feature catalog entry is
"Assistant connections (compatibility)" with both defaults `true`.
- `server` instance settings: normalization always returns
`enablePublicMcp: true`. The Cloud managed overlay skips the key, so a
fleet value of `false` cannot disable it. Removed the key from the Cloud
staging experimental-key list. Removed the client-mirror policy step
that ran only when the toggle changed.
- `server` public MCP: personal discovery, OAuth, tools, setup and event
delivery no longer read instance settings (`isEnabled()` returns `true`
for the personal endpoint without a DB read). Dot, Grok and Muse agent
connections keep their own experimental settings.
- `server` Cloud paths: the client-mirror sync, the hosted event
authority, the ingress-handoff expiry worker and the idle sleep proof
treat the front door as enabled. The warm-standby `503
workspace_unclaimed` behavior is unchanged.
- `server` Dot/Grok gates: they no longer also require
`enablePublicMcp`. Error messages no longer tell people to enable
Assistant connections.
- `ui`: removed the Experimental toggle, the "Enable assistant
connections / Open Experimental settings" block on the Assistant
Connection page, and the "Enable Assistant connections (MCP)" hint in
Dot setup. Updated the Dot and Grok copy.
- Storybook: deleted the Experimental-setting story and removed the
"enable the experiment" step from the journey and presenter.
- Docs: `doc/public-mcp.md` ("Enable an instance", "Start from
Connections", `GET /api/mcp/setup`), `doc/openai-dot-runner.md`,
`doc/idle-grok-dispatch.md`, and the agent-access map.
- Tests: new red/green tests for "enabled with no setting stored" and "a
stored or managed off value is ignored". Removed the tests for the
"disabled" behavior, which no longer exists.
## Verification
- Red/green: with the old source and the new tests,
`instance-settings-readonly.test.ts`,
`instance-settings-service.test.ts` and `public-mcp.test.ts` fail (4
failures, and the public-MCP suite fails in `beforeAll` because
`isEnabled()` was `false` with no setting stored). With the new source
they pass.
- `server`: `public-mcp.test.ts`, instance-settings (readonly, service,
managed overlay), `managed-config`, `cloud-event-authority`,
`events-work`, `dot-runner`, `dot-oauth-onboarding` and
`agent-runtime-assignments`. 354 passed.
- Cloud client-mirror lifecycle
(`client-mirror-lifecycle.postgres.test.ts`, skipped in CI) passed
against a local Postgres 17 with
`PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL`. 6/6, plus `events-work` 28/28.
- Idle and sleep: `idle-incoming-wake` (service and route),
`idle-deadline-proof`, `idle-incoming-wake-capability`,
`idle-sleep-proof`, `client-mirror-ack`, `recovery-deadline-acceptance`
and `idle-completed-history` pass. The warm-standby test now also
asserts that discovery after the claim does no SQL read.
- `ui`: `InstanceExperimentalSettings`, `AssistantConnection`,
`NewAgentDialog`, `NewAgent` and `ExternalAgentInviteDialog.muse` pass.
267 passed.
- Typecheck: `packages/shared`, `server` and `ui` pass. `node
scripts/check-maps.mjs` passes.
- `pnpm verify`: see the CI checks on this PR.
- After the deploy: confirm
`/.well-known/oauth-protected-resource/mcp/paperclip` and `/mcp/setup`
return 200 on paperclip.staging without changing any setting.
## Risks
- Behavior change: instances that had the setting off now serve
discovery, OAuth and `/mcp/paperclip`. Security invariants are
unchanged: the endpoint is still mounted only in authenticated mode with
a valid public URL, OAuth still needs a real signed-in user's consent,
local implicit board authority cannot approve, and revocation works the
same way.
- Cloud sleep: on a managed stack, the MCP sleep proof blocks sleep
until the client mirror publishes the tenant front door to Cloud. The
client-mirror sync runs at boot and does this once. Stacks that already
had the flag on use this path today. A stack whose Cloud connector
cannot publish the mirror would stay awake. Watch fleet sleep after the
deploy.
- Personal public-MCP requests now read instance settings one time less.
- Fleet admin: the Cloud fleet catalog can still show and set
`enablePublicMcp`. The value is ignored. Removing it from the fleet
catalog is a separate change in the Cloud repo.
- No migration. The `0354` trigger still compares the stored key. That
causes at most one extra connector-runtime invalidation if an old client
writes the key.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
1d2b669e7a |
fix(ssh): allow CLI agents on volume-free servers (#21957)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An SSH environment runs an installed harness on a computer the operator controls. > - Volume-free Cloud servers require durable task files outside the controller. > - The current admission path accepts native sandbox workspaces and rejects existing SSH CLI agents before execution. > - The remote workspace owner already provides saved versions and a single-writer fence. > - This pull request gives that owner an SSH archive transport and admits the existing Claude and Codex CLI adapters under its authority. > - The benefit is SSH task execution without a controller volume or a new environment type. ## Linked Issues or Issue Description **What happened?** A task assigned to an existing SSH Claude Code agent fails on a volume-free Cloud server. It reports that execution requires native Paperclip Runner and remote workspace storage. The failure occurs before provider work starts, even when SSH and the configured AI account are valid. **Expected behavior** An SSH CLI agent can use the configured Paperclip AI account and an installed harness. Task files remain durable when the Cloud controller is replaced. The same SSH behavior applies to direct and Tailscale connections. **Steps to reproduce** 1. Configure an SSH environment on a volume-free server with external Postgres, stable encryption, and object storage. 2. Assign a Claude Code or Codex CLI agent to that environment. 3. Give it a task without a project, such as creating `hello_world.txt`. 4. Observe the workspace admission failure before execution. Related work: #21248 adds agent-selected SSH authentication. #20617 adds Tailscale network selection. #15120 and #17291 cover SSH process and lease cleanup. A search of open and closed PRs found no duplicate of this workspace admission fix. ## What Changed - Reuse remote workspace preparation, archives, saved versions, and single-writer admission for SSH targets. - Add an explicit file-sync capability to the existing SSH command runner. It streams archive files with bounded deadlines and private temporary files. Ordinary SSH runners keep their previous capabilities. - Run the existing CLI adapter in the remote workspace. Bind both the target and its SSH command specification to that directory. - Admit a direct SSH adapter on a volume-free server only when its durable workspace, environment, lease, and path agree. Local filesystem authority remains rejected. - Preserve the server-owned remote workspace receipt when selecting the legacy runtime. - Reuse the workspace finalization lock for a legacy run only when its company, workspace, writer, epoch, and lease receipt match. A claimed token can finish and clear itself after another task reserves a successfully saved shared workspace. Storage writes still require the current workspace authority. - Transfer recovered copies to SSH through the verified workspace archive format. Sandbox recovery retains its existing directory transfer. - Document Node.js, Git, explicit `engine=cli`, and uncertain-process limitations. No UI, Cloud control-plane configuration, schema, or deployment change is included. The remote workspace service owns this change. The shared SSH utility adds file transport because its existing authentication and stream cleanup must remain the only owners of SSH credentials. The heartbeat changes pass the storage authority through existing preparation, selection, and finalization boundaries. The finalization owner extension avoids a separate copy-back lock for SSH. Existing native callers keep their prior admission contract. The existing SSH utility and heartbeat test files are already large. The added code stays beside their existing stream lifecycle and shared fixtures. A larger extraction would widen this compatibility fix. ## Verification - Red-green tests reproduce SSH rejection and verify admission only with remote storage authority. - A real isolated OpenSSH fixture exercises archive capture, a replacement database client, a saved-workspace restore to another allocation, and a failed SSH transfer. Failed saves retain the files and block a competing writer. - A whole-heartbeat test uses the real SSH transport and Claude CLI adapter, a synthetic CLI, a selected Paperclip AI account, and fixture object storage. A follow-up resumes the same provider session and its retained history. It makes no provider API calls. The synthetic provider also supplies its task disposition update. - Existing Claude and Codex SSH adapter tests, workspace preparation, and native workspace owner recovery are included in focused checks. - Before the review fixes, clean-checkout verification passed 610 tests in 11 files. This includes the full workspace preparation, runtime selection, runtime execution, environment orchestrator, ephemeral-server, workspace transport, native owner recovery, SSH integration, SSH command, and Claude/Codex remote execution suites. The macOS SSH fixture was enabled with `PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1`; Vitest used `--maxWorkers=2`. - An initial run in the main worktree hit a skill-root lookup outside that checkout. The full selected set passes in the clean checkout. The final company-binding type correction changes no executable behavior; all 8 focused selection regressions pass after that correction. - At the initial head `dd77a4d3cf0`, server and adapter-utils typechecks passed, and the clean verification checkout matched its tree `3f90b0681b1004556d1a720ed45fc8d439b04e96`. - Review regressions first reproduced both a lost ownership token after shared-workspace handoff and a rejected recovered-directory transfer. With the fixes, 223 tests in 5 affected suites pass, including real SSH recovery and native owner-restart, save-concurrency, archive transport, and recovery-receipt checks. The handoff test also confirms that a stale owner cannot save again or claim ownership after a new writer takes over. Server typecheck passes after these fixes. The tested review-fix head is `c5182f5d9c56e738e683d9f26b226e9a820bba83`. - [The initial CI server shard](https://github.com/paperclipai/paperclip/actions/runs/44009944657/job/133794523181) failed on an assertion still expecting the old native-sandbox-only error. Its unsupported sandbox adapter remains rejected; the assertion now matches the supported-remote-environment error. After updating that assertion, all 115 tests in the workspace lifecycle suite pass. Together with the 5 affected suites, this gives 338 passing tests for the review fixes. - `pnpm verify` returns 2 and requests broad CI coverage for the shared/runtime change. The full local suite was not substituted for CI. Final-head [CI](https://github.com/paperclipai/paperclip/actions/runs/44012991355) is green on `c5182f5d9c56e738e683d9f26b226e9a820bba83`: 58 checks pass and 3 are skipped, with no failures or pending checks. Typecheck, build, end-to-end, and the aggregate verification gate all pass. Greptile is 5/5 on this head, security scan passes, and there are no unresolved review threads or merge conflicts. Review-fix checks: ```sh PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1 pnpm exec vitest run --maxWorkers=2 \ server/src/services/ssh-remote-workspace.integration.test.ts \ server/src/services/native-runtime/native-workspace-owner-restart-recovery.test.ts \ server/src/services/remote-workspace-transport.test.ts \ server/src/services/remote-workspace-recovery-receipt-fallback.test.ts \ server/src/services/remote-workspace-save-concurrency.test.ts pnpm exec vitest run --maxWorkers=2 server/src/services/remote-workspaces.integration.test.ts pnpm --filter @paperclipai/server typecheck ``` ## Risks This extends the remote workspace owner to another transport and allows qualified SSH CLI execution on volume-free servers. It changes remote file placement to a task directory under `.paperclip-workspaces`. It does not import an existing controller-local workspace. The installed harness uses its existing adapter. Set `engine=cli` explicitly; ACP archive staging does not support these in-place workspaces. This does not qualify native macOS provider packs or change an agent's engine or permission settings automatically. An unknown remote process after SSH loss remains held until it can be reconciled. This change does not add automatic reconciliation of that process. SSH computers are never stopped or deleted. Real provider acceptance and a hosted staging journey have not been run for this revision. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through the Codex harness, with xhigh reasoning effort and code tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
bc0a236b95 | feat(runner): accept a resume-compatible build for a relaunched released Runner (#21976) | ||
|
|
ca34aca777 |
fix(native): report a Runner caught mid-exit as not running (#21987)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A warm agent session keeps its Runner alive in a sandbox between
turns. The server closes an idle session after 5 minutes. A monitor loop
polls the Runner process with `REMOTE_RECOVERY_PROCESS_CHECK_SCRIPT`
until the process exits.
> - The check reads `/proc/<pid>/cmdline` and exits 48
(`process_runner_id_missing`) when the command line does not hold
`--runner-id`.
> - A Linux process that is exiting releases its memory before it
becomes a zombie. For that short time its command line reads as empty,
but the process still answers `kill -0` and its stat is not yet `Z`.
> - So a check that lands in that window reports exit 48, and the close
fails although the Runner is exiting as asked.
> - This pull request makes the check wait up to 1 second for a process
of the saved generation whose command line is empty. If the process
becomes a zombie or goes away, the check exits 45
(`process_not_running`).
> - The benefit is that an idle warm close no longer fails, and is no
longer logged as a failure, when it catches the Runner while it exits.
## Linked Issues or Issue Description
Refs #21853
**What happened?**
Restart-chaos rerun 11 A1 (master
|
||
|
|
ddb2b25055 |
fix(native): read "as the file reads" as a comparison, not a file request (#21983)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run reports its result with `paperclip_finish`, and the
server checks the cited evidence before it accepts a "done" result
> - The check uses `explicitlyRequestsFileOutput` to decide if the task
asked for a downloadable file
> - The check read "save the document ... exactly as the file reads" as
"save ... as the file", so it decided that the task asked for a file
> - Because of this, the server rejected each correct completion, and
the agent had to upload a scratch log with `register_deliverable` and
report again
> - Each extra report is one more model request, so the task takes
longer, and a slow provider can make it miss its deadline
> - This pull request makes the check treat "as the file reads / is /
shows ..." as a comparison, not as an output destination
> - The benefit is that the agent finishes on its first correct report,
without an unnecessary upload and a second model round trip
## Linked Issues or Issue Description
**What happened?**
A task asked the agent to run seven rounds that append to
`ledger-<id>.log`, then to "Save this issue's document with key
`ledger`: the full contents of `ledger-<id>.log` exactly as the file
reads". The agent saved the document and cited it. The server rejected
the completion with "The requested file has no accessible delivery
evidence" (or "Completion cites a workspace-only file"). The agent then
uploaded the scratch log as a deliverable and reported again. This
happened in 7 of 7 restart-chaos twin waves for this task (seed s7106)
on three master revisions. The detour (first report to accepted report)
took 9-23 s with fast providers and 107-316 s when the provider was
slow. In one wave the task missed its 40.6-minute budget.
**Expected behavior**
"Exactly as the file reads" describes the contents of the requested
document. It is not a request for a file. The first correct report that
cites the published document must be accepted.
**Steps to reproduce**
1. Call `explicitlyRequestsFileOutput("Save this issue's document with
key `ledger` (title \"Ledger\"): the full contents of
`ledger-c8d89f8d.log` exactly as the file reads, then every line the
board asked for.")`.
2. On master it returns `true`. It must return `false`.
**Paperclip version or commit**
master
|
||
|
|
22d3f9a03c |
fix(native): finish a failed warm session's cleanup by itself and continue the task (#21977)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runner runs keep a provider session per task. A follow-up run
can reuse the warm session of the run before it, in a remote sandbox.
> - When a warm follow-up fails, the previous session is closed. The
close saves a backup of the provider session first.
> - The close looked up the backup owner through the task's current tool
authority. The failed follow-up had already released that authority.
Every close attempt failed with `native_harness_storage_owner_missing`,
so the close could never finish.
> - After nine failed attempts, the task's admission gate became
`operator_required`. The gate retires only for a stopped local process
or for the exact run that owns the session. A remote sandbox session
owned by an earlier run never matched. A queued continuation and a Retry
stayed queued forever, and the task showed a Retry button that could not
work.
> - Separately, a remote transport failure could never produce the local
stop proof that the safe-replacement lane waits for. Nothing continued
the task until an unrelated wake arrived 15 minutes later.
> - This pull request makes the close save its backup under its own
owner, retires a cleanup gate when the task's sandbox is confirmed
terminated, and refuses an unprovable remote transport failure at once
so the task continues in a fresh isolated run.
> - The benefit is that recovery finishes by itself, and a Retry that is
shown starts a run.
## Linked Issues or Issue Description
**What happened?**
In a restart-chaos wave, a warm follow-up run on a remote sandbox failed
with `provider_transport_failed` (`PRP command run.attach timed out`).
The previous run's session close then failed on every attempt with
`native_harness_storage_owner_missing`. The cleanup gate became
`operator_required`. The continuation run was created 15 minutes later
and then stayed queued until the wave ended ("queued run kept queued
while the task's previous session finishes cleanup"). The task showed a
Retry banner, but a Retry would wait behind the same gate.
**Expected behavior**
A failed session close settles or is retired by itself with evidence.
The next run of the task starts in seconds. A Retry that is shown starts
a run.
**Steps to reproduce**
1. Run a native task on a remote sandbox with a warm session lifecycle.
2. Make the warm follow-up run fail during startup, after it has reused
the previous session.
3. Watch the previous session's close: every attempt fails with
`native_harness_storage_owner_missing`, and the queued next run of the
task never starts.
**Paperclip version or commit**
master at
|
||
|
|
7685b7a4fc |
fix(native): continue a run that reports a system environment blocker (#21959)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run ends with one semantic report: `paperclip_finish` or
`paperclip_block`
> - An agent can report a system environment blocker, for example when
it thinks its tool results are corrupted
> - Today that blocker binds to the same agent, its wake is dropped, and
the task stays blocked with no Retry
> - In a restart-chaos rerun, an OpenCode agent reported such a blocker,
then recovered in the same turn and did all the work, but its later
`paperclip_finish` was refused with an opaque conflict error
> - This pull request makes Paperclip continue a system environment
blocker automatically in a fresh session, and tells the agent clearly
that a turn's accepted blocker is final
> - The benefit is that a temporary environment problem no longer needs
a person to unstick the task
## Linked Issues or Issue Description
**What happened?**
A native OpenCode run reported `paperclip_block` with owner `{kind:
system, name: execution-environment}`, `reasonCode:
environment_unavailable` and the unblock action "Resume/retry the run".
In the same turn the agent recovered: it saved the task document and
posted the requested comment. Its later `paperclip_finish` failed with
"A different semantic result was already committed". The run succeeded
with disposition `blocked`. The status decision bound a task-wide
blocker to the same agent. Both owner wakes were skipped
(`chat_control_recovery_proof_unresolved`). The task stayed blocked with
no Retry.
**Expected behavior**
A system environment or tool blocker is a harness problem. Paperclip
must continue the task automatically, with a bound, in a fresh session.
A provider that tries to replace an accepted blocker must get clear
guidance, not an opaque conflict.
**Steps to reproduce**
1. Run a native task with an agent that calls `paperclip_block` with
owner kind `system` and reason code `environment_unavailable`.
2. In the same turn, call `paperclip_finish` with a done report.
3. See that the finish is refused with "A different semantic result was
already committed" and that the task stays blocked with no Retry.
**Paperclip version or commit**
`master` at
|