Commit Graph
6598 Commits
Author SHA1 Message Date
Devin FoleyandPaperclip bf53060904 fix(recovery): continue saved input after verified workspace copy-back (#19422)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native execution preserves accepted results and task files after a
failure.
> - A later user message can request a new turn after the old execution
stops.
> - File recovery can finish by copying files back or by retaining a
verified archive.
> - The retained-result gate accepted the archive proof but missed a
verified copy-back receipt.
> - This change accepts that existing receipt for terminal failures and
preserves execution authority and prior results.

## Linked Issues or Issue Description

**What happened?**
A task keeps a user message queued with `finalizing_saved_result` after
its old native coordinator reached `terminal_failure`. The accepted
result and a successful, exact `workspace_recovery` copy-back receipt
are saved. The gate only checks the alternate
`workspace_recovery_snapshot` receipt. An older failed result can also
keep a newer failed turn's saved input blocked.

**Expected behavior**
A new user message or deliberate task Retry can continue from verified
recovered files after the old execution stops. Preserve the old result
and unknown action outcomes. Keep an active finalizer or missing stop
evidence blocked.

**Steps to reproduce**
1. Retain an accepted result when a native run reaches terminal failure.
2. Complete host copy-back with the exact run, allocation, descriptor
and final file digest.
3. Save a new user message or request task Retry.
4. Observe that admission still reports `finalizing_saved_result`.

**Paperclip version or commit**
Reproduced against `4d49a3128613b282fb344272ed7ca15f98dea2ac`.

**Deployment mode**
Authenticated native execution with retained workspace recovery. Local
regressions use disposable PostgreSQL and the real heartbeat queue. They
start no model provider or sandbox.

Related work: #19355 covers saved input after Stop. #18267 concerns
manual reconciliation of duplicate legacy holds; it excludes workspace
and system recovery and does not fix this receipt gate. I searched open
PRs and issues before starting this change.

## What Changed

- Reuse the existing host-verified repair receipt when an accepted
result belongs to a coordinator in `terminal_failure`.
- Retain the separate checks for active controllers, environment
termination, live executions, questions, approvals and current user
authority.
- Add message, saved-message and task Retry regressions, plus concurrent
queue delivery and invalid-receipt cases.
- Document this continuation path. Add no schema, API or background
worker.

## Verification

- All 483 tests pass across explicit continuation, task Retry,
quarantined Retry and finalized recovery migration suites.
- The final focused fixture has 17 cases. All pass with the change. With
the original production gate restored, all four positive cases fail and
all 13 safety cases pass.
- Negative cases cover missing, unfinished and mismatched receipts,
another task or source, unrecovered files, a live finalizer, an active
controller, missing remote stop and active execution.
- The real heartbeat path delivers one saved message under concurrent
recovery calls. Later recovery calls create no second turn. The old
accepted result and recovery budget stay unchanged; unknown outcomes
stay unknown.
- The first delivery fixture omitted the newer failed source needed by
the recovery scanner. I corrected the fixture to preserve that history
and repeated the unchanged-code comparison. Earlier failure logs remain
in the verification record.
- `pnpm -r typecheck`: passed.
- `node scripts/check-module-boundaries.mjs` and `git diff --check`:
passed.
- `pnpm build`: passed.
- Full local `pnpm test:run`: running with a 20-minute bound; no
full-suite pass is claimed. All hosted CI checks passed. Greptile scored
the exact head `d04fb98892fd77853ad03e6b44cd612e805abc13` 5/5 with no
findings or review threads.
- Hosted delivery of the already saved input still requires a qualified
staging build containing this change.

## Risks

The new path requires the existing exact repair receipt and a terminal
failure. A coordinator still accepting or committing a result keeps the
previous hold. Stop, owner and live-run checks still run before
admission. The old result is neither applied nor replayed. Receipt
verification adds bounded indexed reads only on this retained-result
failure path. No migration or provider configuration changes are
required.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through Codex in T3 Code, with xhigh
reasoning. The context-window size is not exposed. Used code editing,
tool calls and local test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used, including the available runtime
details
- [x] I have checked ROADMAP.md and confirmed this is a scoped bug fix
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in this PR using the bug-report fields
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal task
id
- [x] I have run the affected tests locally and they pass
- [x] I have added regression tests and updated relevant documentation
- [x] I have considered and documented the risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open findings
- [x] I will address all review findings before requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1106.0-canary.8
2026-11-06 00:15:30 -08:00
Devin FoleyandPaperclip 11bc490cda fix(recovery): keep a stopped copy-back's files instead of holding the workspace (#19385)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A native run can have its result accepted before the copy-back of
its sandbox files finishes. If the copy-back fails, it retries.
> - A board Stop during those retries ends the run. It sets the run's
finalization to `terminal_failure` / `native_retry_cancelled`, and the
result stays accepted.
> - After that, nothing exports the files. The finalization worker skips
terminal rows, and cancelled-workspace preservation skipped any run with
an accepted result.
> - The task workspace stays dirty and bound to the stopped run. Every
later run on the task waits for the workspace, then fails setup with the
unfinished-writer error.
> - This pull request lets cancelled-workspace preservation also admit
an accepted result whose copy-back a Stop ended.
> - The benefit is that stopping a stuck copy-back saves its files and
frees the task, like stopping any other run.

## Linked Issues or Issue Description

Refs #16700, #17677, #18820, #19304 and #19305 (cancelled-workspace
preservation). This pull request adds the accepted-result case to that
path and keeps its other checks unchanged.

**What happened?**

On a hosted instance, a native run's result was accepted, but its
copy-back kept failing and retrying. A board operator pressed Stop to
unblock the task. The run became `cancelled`, its sandbox was stopped
and kept, and its finalization moved to `terminal_failure` /
`native_retry_cancelled` with the result still accepted.

The next run on the task was cancelled with "Shared execution workspace
is busy: run … ended, but its workspace save has not settled". About 24
automatic retries followed over an hour. The last one failed with "The
workspace still has an unfinished writer or save operation." After that
nothing ran: the task stayed `in_progress` with no run and no recovery
action. A Retry or a new comment would hit the same block.

The workspace stayed `dirty: true` with the stopped run as its writer,
and no `workspaceRestoreRecovery` was ever recorded. The reason:
`preserveCancelledNativeWorkspaces` requires `resultId IS NULL`, and
`registerCancelledWorkspacePreservationWork` and
`preserveNativeFailedWorkspace` both return early when `resultId` is
set.

**Expected behavior**

Stopping a run during its copy-back retries keeps its files, as stopping
any other run does. Paperclip's workspace recovery saves them and
releases the workspace. Nothing commits or replays the stopped result.

**Steps to reproduce**

1. Accept a native result for a run whose copy-back is in
`retryable_failure`.
2. Stop the run with the board Stop. The finalization becomes
`terminal_failure` / `native_retry_cancelled`.
3. Start another run on the task. It waits for the workspace and then
fails setup. No save of the stopped run's files is ever recorded.

The new database tests run steps 1 and 2 against PostgreSQL and check
that the save is recorded.

**Paperclip version or commit**

`master` at `9ceaec6f97e`.

**Deployment mode**

Any deployment that uses remote (sandbox) task workspaces. Seen on a
cloud-managed instance.

## What Changed

-
`server/src/services/native-runtime/native-failed-workspace-recovery.ts`:
cancelled-workspace preservation now also admits an accepted result when
the finalization is `terminal_failure` with `native_retry_cancelled`.
That is the state a Stop leaves after ending copy-back retries. All
three entry points use the same rule:
  - the startup scan `preserveCancelledNativeWorkspaces`;
  - the worker `registerCancelledWorkspacePreservationWork`;
- `preserveNativeFailedWorkspace`, only when it is called with
`requireCancelledStop`.

Preservation still requires everything it required before: a cancelled
run, an acknowledged Stop, no live finalization owner or controller
lease, a prepared workspace reference, and the exact retained source.
- An accepted result with any other failure code is still skipped; those
runs keep their own export repair path. Failed (not cancelled) runs are
unchanged.
- The accepted result stays recorded and uncommitted. Preservation does
not change the task's status or queue a run.
- `doc/execution-semantics.md`: describe the stopped copy-back case in
the paragraph on writers stopped mid-run.
- `native-cancelled-workspace-preservation.test.ts`: move the
accepted-result fixture into an `acceptResult` helper, and add a test
with three cases.

## Verification

- Failing first. The new test "preserves files of an accepted result
whose copy-back was stopped" has three cases: worker, startup scan, and
a sandbox that the Stop already stopped and kept, through the real
`prepareSandboxStopAndRetain` and `settleStopOnlyCleanup`. All three
fail on `master`, with no `workspaceRestoreFailure` recorded, and pass
with this change.
- Each case checks that:
- the save obligation and recovery action are recorded for the exact
lease;
  - `inspectWorkspaceRecoverySource` finds one source;
- the finalization keeps its `resultId`, `terminal_failure` and
`native_retry_cancelled`;
- the workspace stays held by the stopped writer until the save
publishes;
  - no run or wake is created, and the task status is unchanged.
- The existing "does not admit preservation for accepted-result" case
still passes. It uses an accepted result with a different failure code.
- `cd server && npx vitest run
src/services/native-runtime/native-cancelled-workspace-preservation.test.ts`:
44 passed.
- Related suites, 10 files and 854 passed:
- `native-failed-workspace-recovery`, `workspace-recovery`,
`heartbeat/run-control`
  - `native-cancellation-request` (unit and integration)
- `heartbeat-remote-workspace-busy`, `native-workspace-export-retry`,
`native-workspace-reconstruction`
  - `native-stop-accounting`, `task-workspace-binding`
- `cd server && npx tsc --noEmit -p tsconfig.json`: pass.
- Not run end to end on a live instance. Diagnosed from that instance's
run, finalization, lease, workspace and recovery-action rows.

## Risks

- On deploy, the startup scan and the worker will pick up existing
cancelled runs in this state and record their saves. That is the
intended repair, and it uses the same retention checks as other stopped
writers.
- A copy-back interrupted by a server restart can leave a
`remote_workspace_snapshot` operation marked `running` after its process
is gone. This change does not settle such rows. If one blocks the later
save, the save follows its existing retry and board hand-off.
- No schema migration.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use
(shell, file edits, test runs, read-only database queries). The runtime
does not expose the context window or reasoning configuration.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-06 00:03:41 -08:00
DottaandClaude Opus 5.5 39096a55a4 fix(runner): answer lease renewal outside the PRP frame chain and name lease loss (#19419)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs in a managed sandbox talk to the server over one
authenticated PRP connection. The runner (`paperclip-runnerd`) renews
its connection lease on that connection.
> - The server serializes every inbound PRP frame through one promise
chain per connection. Before this change, `lease_renew` waited in that
chain behind earlier events. An event whose business commit stalls (for
example the commit of a tool event) therefore also stalls the lease
reply. The runner then stops polling its provider (ACK backpressure) and
exits with "active connection lease expired" when the lease runs out.
> - When that happens, the server sees only a socket close. The run log
and the terminal error say "transport loss", not "lease expired".
> - A follow-up from #19407 asked for renewal to be handled outside the
chain and for lease loss to be named.
> - This pull request answers `lease_renew` (and `pong`) as soon as the
frame is decrypted. It keeps in-order decryption and the order of
state-changing frames. It also records why a lease was lost and reports
it as `runner_connection_lease_expired`.
> - The benefit is that a slow event commit can no longer starve lease
renewal, and a lease loss is visible in the run log and error code.

## Linked Issues or Issue Description

Refs #19407

**What happened?**
On staging, native runs with ACPX providers (Grok, Kimi, and Pi with
OpenRouter/DeepSeek) ran a long foreground shell command that printed
nothing (a 15–20 minute `sleep 60` loop). Each run went silent soon
after `tool.execution.started`. The command finished in the sandbox, but
the run never got the completion and stayed `running` until someone
cancelled it. It happened in 3 of 3 attempts. The same task on Codex
kept reporting activity and succeeded.

**What did you expect to happen?**
A quiet tool call keeps its runner connection and its lease. If the
connection or lease is lost, the run log and the error code say why.

**Steps to reproduce**
1. Assign a native ACPX agent a task that runs a foreground shell loop
with no output for longer than 2 minutes.
2. Observe that the run goes silent and does not finish after the
command ends.

**Root cause analysis**
- The ACPX path emits no PRP frames during a silent tool call. The
runner has no timer, and the agent sends no `tool_call_update`. Codex
keeps the connection busy, because its model polls the running command.
Silence is the only adapter-specific factor that I found. I found no
ACPX code path in runnerd that blocks the main loop: polls use a 1 ms
timeout, and synchronous sidecar requests run only for server commands.
- The staging build (`de1143448b`) did not include #19407. That build
had no server pings. With a 60-minute connection lease (the default for
`runner-prp-coordinator` and `DurablePrpCodexTransport`), the runner
renews only every 30 minutes. A fully silent connection that goes
through the sandbox preview proxy therefore carried no traffic at all. A
route that the proxy drops without a FIN then becomes half-open forever.
#19407 added 20-second pings and a 90-second termination, which closes
that window.
- The lease hypothesis from #19407 is real but secondary. The 60-second
lease applies only when the TTL is not set. Renewal waited behind any
slow event commit, and the server could not report lease loss. This pull
request fixes both.

## What Changed

- `DurablePrpControlPlane.attachWireConnection` decrypts each secure
frame when it arrives, in counter order. Frames that arrive before the
channel is secure, or behind such a frame, still decrypt in the chain.
- `lease_renew` and `pong` are handled at once, outside the
per-connection chain. Admission against persisted authority is
unchanged. Events, command results, and handoff frames keep their
serialized order.
- `#renewLease` persists the renewed expiry with `save()`, so an event
write that is already staged is absorbed into the same write. Before,
the renewal used `commit()`, which would invalidate that staged event
and close a healthy connection when renewal runs concurrently.
- New `connectionLeaseFailure` getter (exported type
`DurablePrpConnectionLeaseFailure`). It records
`connection_lease_expired` when an authenticated connection is refused
or closes with an expired lease. A new authenticated connection clears
it. A credential revoked by a warm handoff is not reported, because a
replacement runner owns the run.
- `connectRunnerPrpIngress` reads that reason after a close. It logs
`state=reconnecting failure=runner_connection_lease_expired` in the run
log. If reconnect grace runs out, it fails with that code instead of
`runner_ingress_unavailable` or `runner_ingress_auth_failed`, and keeps
the reconnect error as the cause.
- `packages/paperclip-runner/docs/durable-recovery.md` documents the new
behavior.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/control-plane` (220 passed).
- New: "answers lease renewal while an earlier event commit is still in
flight". It was red before the fix (no `lease_renewed` while
`onCommittedEvent` waits), and it is green now. It also checks that the
event later commits and ACKs, and that the renewed expiry survives.
- New: "keeps a staged tool-progress write when a lease renewal lands
mid-write". It holds a file-backed `tool.execution.progressed` write at
`fsync`. It is red with the old `commit()` renewal (the event is dropped
and the connection closes), and green with `save()`.
- New: "reports an expired connection lease instead of closing
silently". It was red before the fix and is green now.
- `pnpm --filter @paperclipai/server exec vitest run
src/realtime/runner-prp-outbound.test.ts
src/realtime/runner-prp-ws.test.ts
src/services/native-runtime/runner-prp-coordinator.test.ts` (all
passed).
- New: "names an expired runner connection lease in the run log and
terminal error when reconnect is refused/unauthenticated". It was red
before the fix and is green now.
- `src/live/runnerd-codex-transport.test.ts` has the same result with
and without this change with a locally built `paperclip-runnerd`: the
same 48 tests fail locally on both, for environment reasons, and the
other 207 pass.

## Risks

- Lease renewal now runs concurrently with an in-flight event handler.
The handler builds its state candidate at commit time, and `save()`
absorbs a staged event write. A new regression test covers the
file-backed race. External checkpoint persistence commits events
synchronously, and the renewal reply still waits for the checkpoint
flush.
- Renewal still stays silent while a warm handoff or a terminal
lifecycle command is pending. The runner treats any other control frame
inside those fences as fatal, so this pull request does not change that
behavior.
- No protocol, schema, or contract-digest change. The runner is
unchanged.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs) and a read-only
exploration subagent on the same model.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 02:01:01 -06:00
DottaandPaperclip 4d49a31286 test(testops): pin reviewed four-case recovery source (#19418)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The protected TestOps workflow evaluates a fixed source commit.
> - The last five-case attempt retained two passes and three failures.
> - Reviewed fixture changes address observed admission and reporting
gaps.
> - Product denial guidance and closed diagnostics remain candidate
overlays.
> - This PR updates only the immutable source pin and its public
metadata snapshot.
> - A bounded four-case measurement can test this candidate without
merging catalog or core code.

## Linked Issues or Issue Description

Refs #19325, #19416, #19395.

**What happened?**
The current pin targets the prior five-case source. The next reviewed
source changes question fidelity context, Personal menu observation,
Company secure-save acknowledgment admission, and Team partial-install
reporting. The last attempt's original failures remain unchanged.

**Expected behavior**
The workflow must admit the exact reviewed source and matching
descriptors. It must preserve the selected model, native and human
bounds, clocks, credentials, and zero automatic retries.

**Steps to reproduce**
Compare the current manifest and descriptor snapshot with source
`58449c4ecd7fd374f21f175411ef3249f55c18a3`. Run the local manifest,
adapter, reconciliation, and CLI admission controls listed below.

**Paperclip version or commit**
Controller base `90b8190c33e8f8eae44cae1f57721251fe1e1343`; candidate
source `58449c4ecd7fd374f21f175411ef3249f55c18a3`.

**Deployment mode**
Protected AWS Product E2E through the existing TestOps workflow.

## What Changed

- Pin the reviewed immutable source and exact descriptor snapshot.
- Update four existing oracle-version leaves and Company/Personal
source-file provenance.
- Append current conditions while preserving the complete historical
conditions prefix.
- Retain all 244 case identities, YAML requests and outcomes, counts,
model, clocks, runtime conditions and policy gates.
- Change one expected target literal in the manifest test. No workflow,
lock, runtime, SDK or credential changes.

## Verification

- 40 local controls pass:
`node --test
tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs
tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs
tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs
tests/runner-e2e/chat-completion-testops-cli.test.mjs`
- Parsed all 244 rows. Only the four oracle versions and two source-file
arrays differ. Runtime conditions are identical. The candidate snapshot
matches committed source bytes.
- The source qualification includes focused positive/negative controls,
full runner E2E types, and unchanged SDK506 registry
`efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`.
Counts from overlapping component sets are not summed.
- All 67 controller dependency graph inputs match the accepted
frozen-install proof. No install, SDK build or runtime build was
repeated.
- The existing selector admits exactly cases 53, 578, 580 and 610 in the
native lane with cap 2. No workflow or settings change is needed. This
local admission control did not submit a run.
- Full repository build/typecheck/test commands were not repeated for
this metadata-only change. Normal PR CI and fresh review remain
required.

## Risks

- The candidate has no new native PASS. Launch 3244 retains two original
PASSES and three original FAILURES. Older failed and missing-original
results remain unchanged.
- Case 53 changes factual setup context to identify the existing visible
form and adds stricter request fidelity. Its YAML query and original
clock are fixed. This is not a latency fix.
- Personal v14 treats the inline code path as an imprecise hint. Actual
current-company UI links must establish navigation before private entry.
Save, owner isolation, consumer delivery and final confirmation remain
separate required proofs.
- Team v8 fixes truthful partial-report admission. It does not qualify
end-to-end runtime migration.
- Company v15 accepts completed-entry acknowledgment questions. Actual
current-human save, payload, source, continuation and governed-approval
proofs remain mandatory. The corrected 403 handoff guidance is a
recovery hypothesis. Closed diagnostics do not prove a native repair.
- Pinned OpenCode model-free tests preserve the tested finish types in
both directions. They do not prove real provider behavior or engine wait
authority.
- The next measurement selects only 53/578/580/610 once, with native cap
2, zero automatic retries and original bounds. A later uniform ten
remains separately unrun. The unchanged 585 pass is carried-forward
evidence.
- Core #19325 and #19416 must remain open and unmerged. This PR does not
merge the held full catalog or change its 196/244 gate. A maintainer
will perform any normal merge and actual measurement after required
checks.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, code execution and
repository tools. The host did not report a context-window size. No
model inference was used for local fixture controls.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
nightly/v2026.1106.0-nightly.0 canary/v2026.1106.0-canary.7
2026-11-06 01:16:57 -06:00
DottaandPaperclip c1d567ff6a fix(workspaces): isolate archived nested Git reconstruction (#19409)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Durable workspace import must preserve archived task files and Git
history.
> - A saved nested checkout can be a linked worktree whose branch
belongs to its old absolute path.
> - Archived import reconstructs that checkout in a new scratch
directory.
> - Ordinary linked recovery rejects the old branch at that new path. It
can also write into a cache used by other import workers.
> - This pull request reconstructs archived nested checkouts as private
standalone repositories.
> - Each import preserves saved files and extra history without changing
shared worktree registrations or branches.

## Linked Issues or Issue Description

**What happened?**
Archived workspace import fails when a saved nested linked checkout has
a branch owned by the old path. The new scratch path fails the
linked-worktree ownership check. The existing cache lock is
process-local, so separate import processes also share mutable Git state
without a common lock.

**Expected behavior**
Archived import must create an independent portable copy of the saved
checkout. It must retain the saved commit, branch name, staged edits,
working files, ignored recovery files, settings, and extra saved
commits. It must keep the source cache unchanged and enforce source
authorization.

**Steps to reproduce**
1. Create a managed nested linked checkout with an unpushed commit and
staged and unstaged edits.
2. Save an encrypted restore point, including ignored files and an extra
history tip.
3. Restore its saved contents under a different absolute workspace path.
4. Before this repair, the new path cannot adopt the branch owned by the
old path.

**Paperclip version or commit**
The real Git regression fails before the repair at `fc1f5f3052`. The
final branch includes the merged interrupted-upload and nested-source
authorization fix from #19396.

**Deployment mode**
Archived remote workspace migration with real Git repositories,
disposable PostgreSQL, and encrypted object storage. Local integration
tests use the filesystem storage provider.

Builds on merged #19402 and #19396. Related #19374 concerns interrupted
clone staging; it does not implement private archived reconstruction. No
duplicate open repair was found.

## What Changed

- Enable private repository reconstruction only for archived migration.
Ordinary linked recovery keeps its path ownership guard.
- Add an object-only source to the managed checkout owner. A verified
cache supplies only the saved commit and authenticated bundle
prerequisites through scoped fetches. The private copy includes no
unrelated task refs or objects, hard links, object alternates, or
current cache files.
- Fetch missing authenticated bundle prerequisites before importing a
saved bundle in both private reconstruction and ordinary linked
recovery. A pruned cache can use the approved remote; unavailable
history still blocks restore.
- Keep every authenticated restore point's extra commit under a private
ref in root and nested private copies so portable archive capture and
garbage collection retain it. Ordinary recovery keeps its existing
history and expiration rules.
- Share the verified, bounded ignore-file copy between root and nested
reconstruction. Sources remain unchanged.
- Retain source authorization before and after reconstruction and the
final source-row locks from #19396.
- Test concurrent private copies, source rejection, foreign task refs
and objects, ordinary linked prerequisite recovery, a real migration,
and cold restore after removing all original repositories. Test hard
links, oversized excludes, unsafe info paths, and replacement during a
descriptor read.

## Verification

- Final affected run: 292 tests pass across ten suites. These cover real
Git repositories, encrypted restore points, disposable PostgreSQL
migration, portable archives, the restore-point ledger, source
rejection, and concurrent copies.
- Full repository typecheck and build pass after the scoped-object
repair. Final server typecheck and full build also pass after
restricting preserved refs to archive mode.
- RED then GREEN: a linked checkout at a different path; a pruned bundle
prerequisite still available from the approved remote; foreign task refs
and objects entering the archive; root extra history disappearing after
cold restore; and archive-only refs blocking ordinary standalone
expiration.
- Two concurrent private copies retain the saved head, branch, staged
and unstaged edits, ignored recovery bytes, settings, and modes. Source
refs, index, worktree registrations, and working files remain unchanged.
- Delete the original checkout, cache, remote, and private copies after
portable capture. Cold restore retains the saved files and extra history
after garbage collection. No alternates or unrelated task objects
remain.
- Ordinary linked recovery retains its path ownership check and recovers
pruned prerequisites. New and existing ordinary standalone destinations
have no archive-only refs and remain eligible for expiration after
normal object cleanup.
- Unsafe exclude tests reject hard links, oversized files, nonregular
paths, info-directory substitution, and replacement during the
descriptor read. They preserve the target and replacement source bytes.
- Preserve the failed attempts: an initial migration fixture changed its
home after constructing the source path. Later expiration fixture
attempts used a filtered current-head tip, omitted the local-source
callback, or used cleanup that generated Git server-info files or left
packed unreachable objects. Correct those setup assumptions before the
actual regressions fail. Final affected suites pass.
- Two earlier local full-suite attempts were interrupted before editing
valid review findings. Neither is reported as a full-suite PASS.
Final-head full CI passes; review is 5/5 with no open findings.

## Risks

Archived imports use additional private disk space for independent Git
objects. Missing, changed, or unsafe sources still block import. The
repair does not weaken the default linked-worktree ownership check,
change stored formats, or update live provider state. The shared
checkout addition is optional and has no effect for existing callers
that do not supply an object source.

## Model Used

OpenAI GPT-6 in Codex, with code editing, tool use, and test execution.
The exact runtime model identifier and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1106.0-canary.6
2026-11-06 00:56:06 -06:00
DottaandClaude Opus 5.5 90b8190c33 fix(native-runtime): detect half-open runner ingress and project Stop for live runs (#19407)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs in a managed sandbox talk to the server through a
provider-ingress WebSocket. The server opens this socket to the runner
(`paperclip-runnerd`) through the sandbox preview proxy.
> - A staging run ran a 20-minute foreground shell command that printed
nothing. After the first second, no run events arrived. The run stayed
"Working" for over 35 minutes. No watchdog acted, and the run projection
said `canStop: false`.
> - The server only detected a lost runner connection from a `close` or
`error` event. It never sent pings. A half-open route, for example a
proxy that drops the connection without a FIN, never emits `close`. The
run then waits forever and the 7-minute reconnect grace never starts.
> - A long silent tool call is legitimate. The fix must tell "quiet but
alive" apart from "nothing alive". A protocol ping does that: runnerd
answers every ping with a pong while its provider waits on the tool.
> - This pull request pings authenticated runner connections and
terminates the socket after 90 seconds of total silence. The existing
reconnect grace and transport-failure path then recover the run. It also
makes the execution projection report `canStop: true` for every queued
or running run.
> - The benefit is that a run with a dead runner connection now ends in
a bounded time instead of hanging. API readers also see the Stop that
the board already offers.

## Linked Issues or Issue Description

**What happened?**
A native-runner run (managed sandbox, OpenCode with an OpenRouter model)
ran a foreground shell command for about 20 minutes with no output. The
last run event came at about T+1s. The command finished on disk, but the
run never got another event and stayed `running` / "Working" for over 35
minutes. Its projection had `canStop: false` and `permittedActions:
["inspect_run"]`. A board cancel through the API worked.

**Expected behavior**
The server detects a dead runner connection and recovers or fails the
run in a bounded time. A runner that is alive but quiet stays attached.
The projection reports that a live run can be stopped.

**Steps to reproduce**
1. Start a native run in a managed sandbox.
2. Make the agent run a long foreground command that prints nothing.
3. Let the preview route drop the WebSocket without a close frame.
4. Before this fix, the run stays "Working" forever.

## What Changed

- `server/src/realtime/runner-prp-outbound.ts`: after PRP
authentication, `WsJsonWireConnection.monitorLiveness` sends a WebSocket
ping every 20 s. Any inbound frame or pong counts as liveness. After 90
s with nothing inbound, it terminates the socket. That emits an ordinary
`close`, so the PRP authority and the reconnect loop use the existing
7-minute grace and then fail the transport. If the server's own event
loop stalls, the window restarts instead of blaming the runner. The 90 s
bound is longer than runnerd's own 60 s connection lease, which already
requires runnerd to read the socket.
- `server/src/services/execution-projection.ts`: `canStop` is now `true`
for every `queued` or `running` run. The old failed-native-retry rule is
unchanged.
- `server/src/services/execution-projection.ts`: a queued run now shows
"Waiting for previous run to stop" (cause `previous_run_settling`) when
a predecessor that admission checks (the task execution owner, the
retried run, or the agent's latest settled run on the task) still has
pending cleanup by `hasPendingRunCleanup`. Before, a turn queued after a
Stop showed only "Queued" with no reason while the stopped run's
controller still waited on the dead runner. That in-process wait
(`claimQueuedRun` settling-owner gate) also made Retry a no-op. The
liveness bound above makes that controller settle. Its existing cleanup
then calls `startNextQueuedRunForAgent`, which starts the queued turn.
- `server/src/services/run-cancellation.ts` and
`server/src/modules/run-dispatch/adapters/postgres.ts`: a user-stopped
native run whose runner was unreachable can later settle as `failed`
(`native_provider_terminal_failed`) when its transport fails closed. The
dispatch gate's message deferral required a `cancelled` source and
compared the message with `finishedAt`. The user's follow-up message was
therefore marked `skipped`, and the task asked for a second message. The
new `recordedUserStop` helper honors the recorded user Stop on such a
failed source, so the gate saves the message for the deferred-receipt
worker.
- `packages/shared/src/types/execution-projection.ts`: documents the
`canStop` meaning.
- `doc/execution-semantics.md`: documents the ingress liveness bound and
the new `canStop` meaning.

## Verification

- Red then green: `runner-prp-outbound.test.ts` "detects a half-open
authenticated ingress and fails the run within a bounded window". On the
old code it failed because no ping was sent and the socket was never
terminated. It now passes and the active-run `failure` rejects with
`runner_ingress_unavailable`.
- Safety: "keeps a quiet but live runner attached through a long silent
tool call" checks that a runner answering pings stays attached for 20
simulated minutes with no termination and no failure.
- Red then green: `execution-projection.test.ts` "offers Stop for every
live run, including a long-silent native turn". On the old code it
failed with `canStop: false`.
- Red then green: `execution-projection-recovery.test.ts` "names the
wait when a stopped predecessor still holds the task's runner" (real
Postgres). On the old code the successor projected a plain "Queued". It
also checks that the wait clears on lease release and ignores another
agent's run.
- Red then green: `workspace-recovery.test.ts` "continues a message sent
after Stop when the stopped run later failed closed". On the old code it
failed 3 of 3 runs with the receipt `skipped`. It now passes, and the
full file passes twice (392 tests). The test now pins the message after
the recorded Stop. Before, the fixture could write both in the same
millisecond, which made the sibling variants flaky.
- Passed: run-cancellation, explicit-native-continuation, run-dispatch
and run-control suites (622 tests).
- Passed: `npx vitest run src/realtime/
src/services/execution-projection*.test.ts
src/__tests__/heartbeat-run-privacy-routes.test.ts`, the
native-runner-execution-driver tests, and the plugin-runner-task tests.
- Passed: `pnpm --filter @paperclipai/server typecheck`.
- I read runnerd (`runner-core/src/durable/transport.rs`) and confirmed
it answers `Ping` with `Pong` in its non-blocking read loop.
- Not run: a live managed-sandbox reproduction. Staging and Daytona were
not reachable from this environment. `plugin-runner-probe.test.ts` needs
a local cargo build of runnerd, and it did not start in this worktree.

## Risks

- A false termination is possible if a runner fails to answer pings for
90 s while it is still alive. runnerd already loses its own connection
lease after 60 s without reading, so this bound does not add a stricter
requirement. A terminated live runner reconnects within the existing
grace.
- If the preview proxy drops WebSocket control frames, ping alone does
not prove liveness. Inbound PRP frames (runnerd's lease renewals, about
every 30 s) still count, so a live runner stays attached.
- `canStop` now means "the board can Stop now" for live runs. The UI
already shows Stop for running and queued runs without reading this
hint, so the UI does not change.
- Out of scope: the inbound runner WebSocket server
(`runner-prp-ws.ts`), and a lost OpenCode SSE event while the runner is
alive. A stopped run whose controller hangs for a reason other than a
dead transport still holds the queued successor. The projection now
names that wait, but this PR does not force-release it, because the
controller has no termination evidence. A queued reason for the agent
concurrency cap is also not covered. The wait label calls the admission
predicate (`hasPendingRunCleanup`) for up to three predecessor runs per
queued run. This adds a few reads to projections of queued runs only.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
file edits) in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 00:52:01 -06:00
Dotta 92452524f4 fix(server): recover authenticated empty retention checkpoints (#19417)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Ephemeral server enrollment must preserve existing task files.
> - Archived task directories can depend on encrypted retention
checkpoints.
> - A missing object blocks migration even when its authenticated
contents were exactly empty.
> - Metadata counters alone cannot prove that no saved files existed.
> - This change verifies the exact candidate against the original
encrypted key envelope before normal migration publication.
> - The benefit is lossless migration of this bounded case without
resetting task history or weakening ordinary restore.

## Linked Issues or Issue Description

**What happened?**
Migration stops at an archived empty directory when its retained
encrypted object is missing. The original encrypted key envelope still
authenticates the payload hash, byte length, company, workspace,
checkpoint ID, and generation.

**Expected behavior**
Administrative migration can reconstruct those exact original bytes when
storage confirms absence and every authenticated check passes. It must
keep missing nonempty, expired, corrupt, or ambiguous sources blocked.

**Steps to reproduce**
1. Capture and encrypt an empty task directory.
2. Retain its reference and encrypted key envelope.
3. Remove the old directory and storage object.
4. Run the normal remote workspace migration.

**Paperclip version or commit**
The regression fails before this change at `205ffcb606`. Related
preservation work is in #19396, #19402, and #19409. No duplicate open
fix was found.

**Deployment mode**
Administrative migration to external workspace storage. Tests use real
encryption, real filesystem storage, and disposable PostgreSQL.

## What Changed

- Let the encryption owner verify a candidate against the existing
scoped encrypted key envelope. Normal ciphertext validation keeps its
previous order and checks.
- Share the task-directory encoder with an exact empty payload
constructor.
- Add a default-off loader option for confirmed missing empty
checkpoints. Only the administrative migration caller enables it.
- Keep the original checkpoint reference and object unchanged. Publish
the recovered bytes through the existing snapshot and source-fence
owners.
- Reject forged counters, wrong identities, missing nonempty data,
present corruption, unreadable present objects, unknown absence, and
expiry.

## Verification

- Final affected run: all 238 tests pass across five crypto, snapshot,
retention, and migration suites. Full repository typecheck and build
pass; server typecheck and full build also pass after the cancellation
repair.
- RED then GREEN: exact missing-empty recovery and cancellation of a
stalled HEAD through the real storage-service boundary. Cancellation
retains the caller's original abort reason.
- Disposable PostgreSQL migration publishes a cold-restorable empty
archive. Archived status, task status, task binding, original reference,
and the absent original source stay unchanged. Replay keeps one
canonical snapshot.
- Negative cases reject forged empty counters, an authenticated
directory or file, every foreign identity field, expiry, present
corruption, unreadable present objects, unknown absence, and missing
nonempty data. Ordinary restore keeps its previous failure behavior.
- Initial verification retained one new fixture-path failure after 145
passes, and a typecheck rejected the fixture's widened schema literal.
Correct the fixture to use the returned staging path and its explicit
reference type. The final complete affected run passes.
- The new HEAD request runs only after a failed read with the
administrative option enabled. It uses the existing cancellation
contract and adds no polling or background work.
- Final-head full CI passes; review is 5/5 with no open findings. This
PR does not claim deployed enrollment or a live migration result.

## Risks

Recovery requires the original valid encrypted key envelope and an exact
format match. A format change or unavailable key continues to block
migration. Existing ciphertext hash, size, authentication, and ordinary
restore behavior remain enforced. This change has no schema,
provider-session, API, or UI contract changes.

## Model Used

OpenAI GPT-6 in Codex, with code editing, tool use, and test execution.
The exact runtime model identifier and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub references)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-06 00:50:28 -06:00
DottaandClaude Opus 5.5 a4bb059f59 fix(heartbeat): retry transient sandbox provider lease refusals and explain the block (#19412)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A task runs in an execution environment. A managed sandbox
environment gets a lease from a sandbox provider (for example Daytona)
before the agent starts.
> - When the provider answered sandbox create with a temporary error
(HTTP 502), the run failed as the generic `setup_failed`.
> - `setup_failed` is the fail-closed code for unknown setup errors. It
gets no automatic retry, so the task went to `blocked` at once. The task
showed only "Couldn't finish this attempt".
> - A user had to find and press Retry. The Retry then worked, which
shows that the failure was temporary.
> - This pull request classifies temporary provider refusals during
lease acquisition from structured fields. It retries them within the
existing bounded bootstrap budget, and it explains the block when the
retries are used up.
> - The benefit is that a short provider outage no longer stops a task.
When the outage continues, the task shows the cause and a Retry that
works.

## Linked Issues or Issue Description

**What happened?**
A task was assigned to an agent that uses a managed Daytona sandbox
environment. Sandbox creation returned `HTTP 502`, and the cleanup of
the provisional sandbox could not be confirmed. The run failed with
`setup_failed` and the error `Failed to acquire lease for environment
"Paperclip Sandbox" (sandbox): Daytona sandbox creation failed (HTTP
502); cleanup could not be confirmed for paperclip-create-…`. Three
seconds later the task was `blocked`. No automatic retry was scheduled.
The task showed no specific reason. A manual Retry started a new run
that completed normally.

**Expected behavior**
A temporary provider error (HTTP 5xx, 429, timeout) before the agent
starts must retry automatically with bounded backoff. The task must show
why it waits. If the retries are used up, the task must show the cause,
a next action, and a Retry that works on the first click.

**Steps to reproduce**
1. Configure an agent with a sandbox environment.
2. Make the provider return HTTP 502 on sandbox create.
3. Assign a task to the agent.
4. See that the run fails with `setup_failed` and the task goes to
`blocked` with no retry.

**Paperclip version or commit**
`master` at `612da1e8e0`.

## What Changed

- Plugin SDK: an `environmentAcquireLease` error now carries a closed
diagnostic envelope (HTTP status or a closed errno code) across the
worker RPC. It takes the status of the create call only, not of the
compensating cleanup lookup. The existing creation-cleanup ownership
envelope is unchanged. New module: `environment-acquire-error.ts`. The
resume diagnostic helpers are shared.
- Server: new `EnvironmentProviderUnavailableError` and code
`environment_provider_unavailable`. The environment run orchestrator
sets it as the cause of `lease_acquire_failed` only when structured
evidence shows a transient status (408, 425, 429, 500, 502, 503, 504, or
a closed connection errno). It never parses message text. Saved-sandbox
resume and stopped-worker refusals keep their own codes.
- Setup failure (`failRunSetup`): a fresh, pre-dispatch run records
`environment_provider_unavailable` with a `providerFailure` (phase
`startup`, retryability `transient`). The run uses the bounded bootstrap
retry (30 s, then 60 s). An accepted interaction continuation keeps its
own continuation retry, as for `plugin_worker_unavailable`.
- Recovery: the new code is registered with the other bootstrap codes.
It is not sent to the generic continuation retry. It does not create an
execution-reconciliation hold, because no provider work started. On
exhaustion the stranded notice has the title "Sandbox provider was
temporarily unavailable" and a specific next action.
- UI: `ProviderFailureNotice` labels the `sandbox` provider as "Sandbox
provider". The task notice shows the provider failure, "Paperclip will
retry automatically" while a retry is scheduled, and Retry after
exhaustion.
- Docs: `doc/execution-semantics.md` describes the policy and the
sandbox ownership rule.

## Verification

Red before the fix, green after the fix:

- `server/src/__tests__/heartbeat-process-recovery.test.ts` › "transient
sandbox provider refusal during lease acquisition":
- "retries automatically and runs exactly one successful successor
without user action": the real heartbeat. The lease runtime returns a
Daytona 502 once, then delegates. Before the fix: one `failed` run and
no successor. After the fix: one `scheduled_retry` successor with no
hold and a `retry_scheduled` presentation. After promotion, exactly one
successor succeeds.
- "explains the block with a working Retry once automatic retries are
used up": three refusals. The task is `blocked` with a "Sandbox provider
was temporarily unavailable" notice. The presentation is `{ state:
"failed", canRetry: true, providerFailure: { httpStatus: 502 } }`, and
no execution-reconciliation hold exists. One `requestTaskRetry` is
accepted, and its run succeeds.
- `server/src/services/heartbeat/run-completion.test.ts` › "routes a …
sandbox provider lease refusal through bounded bootstrap recovery": the
real orchestrator wrapping. Only the fresh, structured 502 is retried. A
message-only 502, a 400, a run after provider dispatch, and a retained
native run stay `setup_failed`.
- `packages/plugins/sdk/tests/environment-acquire-error.test.ts`: the
real worker RPC. The test checks the create status (not the 404 cleanup
lookup), the errno, no provider text, the unchanged cleanup envelope,
and that forged envelopes are rejected.

Commands:

```sh
pnpm --filter @paperclipai/plugin-sdk exec vitest run tests/environment-acquire-error.test.ts tests/environment-creation-cleanup.test.ts tests/environment-resume-error.test.ts
cd server && npx vitest run src/__tests__/heartbeat-process-recovery.test.ts src/services/recovery/ src/services/heartbeat/run-completion.test.ts src/services/heartbeat/retries.test.ts src/services/legacy-execution-recovery.test.ts src/services/task-interruption-presentation.test.ts src/services/task-retry.test.ts src/__tests__/environment-runtime.test.ts src/__tests__/heartbeat-retry-scheduling.test.ts
cd packages/plugins/sandbox-providers/daytona && npx vitest run
pnpm --filter @paperclipai/plugin-sdk typecheck && pnpm --filter @paperclipai/server typecheck
```

I did not run a live Daytona 502 on staging. Re-check on staging after
deploy (see Risks).

## Risks

- Sandbox leaks or double creation: each retry is a new run with a new
durable creation record and a new provider name
(`paperclip-create-<operationId>`). The failed attempt stays owned by
its own record, and the existing creation reconciler settles it. A retry
never reuses or abandons it. While a provisional sandbox waits for
reconciliation, up to two more sandboxes can be created for the same
task. This is the same as a manual Retry today.
- Plugins built with an older SDK do not send the envelope. Their
refusals stay `setup_failed`, which is the current behavior.
- An in-process error with `status`/`statusCode` 500–504 or a connection
errno during lease acquisition is now retried twice. The budget is
bounded and shared with the other bootstrap codes.
- An RPC timeout on the host side is not classified by this change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code as an agent with tool use (shell, file edits, test runs).
Extended reasoning was on.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1106.0-canary.5
2026-11-06 00:08:44 -06:00
Devin FoleyandClaude Opus 5.5 558a5143ed fix(workspaces): reject damaged Git entries before workspace capture (#19160)
Workspace capture now checks that each .git entry is the root of its own
repository before it reads any Git state. A structurally damaged entry
(an empty .git directory, a gitfile for a removed worktree, an invalid
HEAD, or missing objects/refs) stops capture with the closed reason
remote_workspace_archive_invalid_git_repository instead of a raw
"Command failed: git ..." error, and capture no longer saves an enclosing
repository's state as the workspace's own. Other Git failures (dubious
ownership, a full disk, a timeout, a failing Git program) keep their
existing diagnostics. Metadata reads are bounded and never block on named
pipes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:06:05 -08:00
Nicky Leach 205ffcb606 Verify live Claude and Codex ACPX eligibility before parking (#19295) 2026-11-05 21:37:34 -08:00
DottaandPaperclip b918ee165d test(evals): pin coherent chat feature follow-ups (#19411)
## Thinking Path

> - Paperclip manages agent work and its permission boundaries.
> - The chat catalog runs feature checks from an immutable source.
> - The last full-ten attempt had five reported passes, four failures
and one missing original.
> - Local evidence identified narrow assertion, setup and
evidence-retention repairs.
> - This PR updates the trusted source pin and its exact descriptor
snapshot.
> - The next five-case measurement can test the affected features under
the same model, clocks and native cap.

## Linked Issues or Issue Description

Refs: #19395, #19405, #19093, #19084, #19112.

The previous source remains preserved. This PR selects reviewed source
`f5692c57dafaf9afbef479c25fcd277815932516` on
`codex/chat-full-ten-coherent-qualified-f5692c57-20261106`, tree
`b9e302d2ea7b605a60125dec8c81672eac7a3c57`. The full catalog and all
core PRs remain unmerged. Core #19405 is included only as a source
overlay.

## What Changed

- Update the source target, exact descriptor snapshot and the matching
pin checks. Preserve historical conditions, frozen source anchors and
the prior query clarification.
- Record Company ACK handling and Company/Personal negation corrections,
plus the skill-update preview negation correction. A denied or unapplied
action still fails when the agent claims it completed.
- Disclose the Team setup change: a disposable catalog package selects
the approved OpenCode runtime and same-company account. Other file bytes
and non-runtime sidecar fields stay unchanged. Original and effective
package identities and hashes are recorded. A real PostgreSQL install
without caller overrides proves the saved runtime. This gives no
end-to-end runtime-migration or override-handling credit.
- Include defensive failed-turn evidence retirement and generic,
schema-valid human-input examples. The old case 53 missing-original
ENOENT cause is not diagnosed. The examples are a usability hypothesis,
not a schema repair or native-success claim.

All 244 YAML definition fields (query, assumed context and expected
outcome) remain byte-identical versus source 936c. The Team derived
runtime fixture and task setup context change is disclosed above.
Native/human bounds, clocks, DeepSeek 1017, credentials, SDK506/efa,
launcher, private interfaces, native cap two and zero automatic retries
remain unchanged. Only descriptor `sourceFiles` and oracle versions
change. The 196/244 gate stays unchanged.

## Verification

Pin-local checks:

```sh
node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-completion-testops-contract.test.mjs
```

- All 49 pin/admission controls pass locally. They join all 244
identities, historical query overrides, native bounds and the exact
242-row descriptor snapshot.
- Source qualification: 447 controls across 32 suites, full E2E
TypeScript checks and normal contract generators pass. The separate 21
core contract controls are carried-forward component evidence; they are
not added to 447. Local source checks used Node 26.4.0. AWS runtime
settings are unchanged.
- The selected source descriptor SHA256 is
`24fe88af167724c6fb120d3984296aab47c993a65782158e03d80a936fbd94ed`.
- The actual source launcher is byte-identical to 936c: SHA256
`2b9a6953ee51f0ea0dbbd475b0491d9a6c39a1d6835eeaa912e0dbb7cf987724`, Git
blob `c88df15f509b2ac8638af826647a97d45f16bd1f`.
- All 506 registered SDK inputs and the ten trusted helper bytes are
unchanged. Five required source-helper joins pass. No SDK renewal or
native build is part of this PR.
- The current controller's 67 dependency inputs match the qualified
frozen reporting graph. No reporting-lock, package, permission or
workflow change is included.
- Exact head `b833036fb477b55b05cbdf1f4feb2e2c5387b420` is green: CI run
42166216108 attempt 2 succeeded, Windows succeeded, and both current
reviews are 5/5 with zero unresolved threads. The check rollup has 56
successes, three configured skips and one neutral result. New
representative cases 53, 578, 580, 585 and 610 are unrun. A later
uniform-ten regression is separate.
- CI attempt 1 remains retained: one cloud-auth browser fixture read a
trailing health response after context teardown began, after all
task-recovery assertions had passed. One authorized rerun of only that
failed job and its dependent checks succeeded on the unchanged head.
This does not fix or qualify fixture reliability; no eval or model run
was retried.

The previous full-ten outcome remains five reported passes, four
failures and one missing-original/reporting gap. Case 585 reached only
its first preview before the false rejection; its update phase is
unqualified. Case 610's old noncompletion and runtime-override journey
remain unresolved. All original failures and evidence remain retained.
No old result is regraded or replaced.

## Risks

The local repairs may not produce a native pass. The new Team setup
tests installation with an already selected runtime package; it does not
qualify runtime migration. Failed-turn retirement has local regression
coverage but does not establish the cause of the old case 53 reporting
gap. The core guidance overlay remains unmerged and its native benefit
is unproved. Normal protected merge and a new authenticated measurement
are still required.

## Model Used

OpenAI `gpt-6.1-sol` through Codex. It used code inspection, local test
execution and GitHub tooling. The context window size is not exposed. No
model inference or provider call was made for this PR's local checks.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 23:32:34 -06:00
DottaandPaperclip 66d3fa97a9 fix(workspaces): preserve ignored files in archived Git imports (#19402)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Durable workspace migration must preserve archived files and saved
recovery data.
> - An archived Git workspace can contain ignored recovery receipts and
bundles in its encrypted restore point.
> - The importer reconstructs a private repository from a verified
source before it applies that restore point.
> - Reconstruction omitted the source repository's local ignore rules.
The restored files then changed classification, and the exact
restore-point check failed.
> - This pull request copies the verified local ignore rules into the
private repository before restoration.
> - The source stays unchanged, and the original restore-point integrity
checks still apply.

## Linked Issues or Issue Description

**What happened?**
An archived workspace with saved files under an ignored recovery
directory failed migration with `restored_contents_mismatch`. The task
commit and staged index matched. The ignored files were restored, but
the private repository treated them as untracked work because its
`info/exclude` rules were missing.

**Expected behavior**
Reconstruction from an existing verified source must preserve its local
ignore policy. Saved ignored files must survive migration and a cold
object-store restore with their original classification. Import must not
follow a symlink to an unrelated ignore file or edit the source
repository.

**Steps to reproduce**
1. Exclude a recovery directory in the managed source repository's
`info/exclude` file.
2. Save staged edits, unstaged edits, an ignored recovery receipt, and
an ignored binary bundle in an encrypted restore point.
3. Archive and remove the task worktree.
4. Import the archived workspace. Before this change, restore-point
validation fails because the reconstructed repository lacks the local
exclusion rule.

**Paperclip version or commit**
Reproduced on master at `879140cbbd`.

**Deployment mode**
Cloud workspace migration with a retained verified Git source,
PostgreSQL, and encrypted durable object storage. The tests use real
disposable PostgreSQL, Git repositories, and encrypted filesystem-backed
object storage.

Related migration work: #18826, #19138, and #19396. No duplicate open
fix was found.

## What Changed

- Copy the verified source's `info/exclude` file into the private
reconstructed repository before applying its restore point.
- Bound the read to one MiB. Reject symlinked, non-regular, hard-linked,
or changing source files.
- Keep the original source bytes unchanged.
- Add regressions that check staged and unstaged edits, ignored recovery
files, cold restore, and rejection of a symlinked ignore file.
- Document the reconstruction behavior.

## Verification

- Before the repair, the saved-ignored-files regression fails with
`restored_contents_mismatch`.
- After the repair, both focused cases pass.
- At rebased head `81a0b1c041`, all 206 tests across migration, snapshot
storage, archive, and migration concurrency suites pass. The merged
interrupted-upload and nested-source approval regressions remain intact.
- Server package typecheck passes.
- Full repository typecheck and build pass at the rebased head. The full
local test suite is running. The prior local full-test run was
interrupted for rebase, and is not a passing result.
- Initial CI hit a 15-second timeout in unchanged
`activity-routes.test.ts`; the exact case passed in a focused local run.
The browser shard lost its self-hosted runner, as confirmed by its check
annotation; GitHub did not retain its full log. Those failures remain
recorded. An unchanged retry was superseded by the required conflict
rebase. At exact head `81a0b1c041`, all 53 CI checks and both required
aggregate gates pass; Greptile is 5/5. The full local suite remains in
progress, so no local full-suite PASS is claimed.

A second automated review noted the existing nested-checkout
counterpart. This PR fixes root reconstruction. The nested
linked-checkout repair and its saved-ignored-files/cold-restore
regression are implemented in #19409, which is required before the
migration rollout uses nested imports. This PR leaves that sibling
recovery path unchanged. The two identical scope warnings are addressed
through that linked repair.

## Risks

The copy applies only when an archived Git restore-point source is still
present and verified. It changes only the private reconstruction
directory. Missing sources retain the existing restoration path. Unsafe
or unstable local ignore files block import and retain the original
migration obligation. No database migration or saved format changes are
included.

## Model Used

OpenAI GPT-6 in Codex, with code editing, tool use, and test execution.
The exact runtime model identifier and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1106.0-canary.4
2026-11-05 23:11:28 -06:00
DottaandClaude Opus 5.5 612da1e8e0 test(server): drain throttled follow-up in busy recovered-source execution-control test (#19403)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server runs durable execution-control workers, which include the
workspace-recovery sweep
> - `execution-control-work.test.ts` has a test for a Retry that commits
during a busy recovered-source pass
> - This test fails intermittently in PR runs that do not touch it, so
it blocks unrelated PRs
> - The test assumed that one `settle()` pass always ends idle, but the
worker can correctly owe a throttled follow-up pass
> - This pull request makes the test drive fake time through that
throttle window while it waits for the sweep to finish
> - The benefit is a deterministic test that still proves every source
is visited and the worker becomes quiet

## Linked Issues or Issue Description

**What happened?**
`execution-control durable work > visits a newly committed earlier Retry
during a busy recovered-source pass, then becomes quiet` failed in 2 of
the last 250 failed `PR` workflow runs (jobs 127272079294 and
127206639630). Both failures were identical: `AssertionError: expected 1
to be +0` at `vi.waitFor` in `settle()`
(`execution-control-work.test.ts:146`). The test took about 6s, which is
the 5s `waitFor` limit plus overhead.

**Expected behavior**
The test passes on every run. Its result does not depend on runner
speed.

**Steps to reproduce**
1. Add a 300ms real-time delay in `registerExecutionControlWork` after
`await duty.run()` and before `await duty.next()`. This simulates slow
database I/O on a loaded runner.
2. Run `pnpm exec vitest run
src/__tests__/execution-control-work.test.ts -t "busy recovered-source
pass"` in `server/`.
3. The test fails 3 of 3 times with the same CI error.

**Root cause**
- `registerExecutionControlWork` lets a duty make one full pass per
`EXECUTION_RECONCILIATION_INTERVAL_MS` (15s), measured from the start of
the previous full pass.
- The released recovery jobs call `worker.wake()` when they finish. If a
wake arrives while the full pass is still reading its next deadline,
`hasPending()` stays true. The coordinator then keeps its idle-tracked
intent until the next permitted pass, up to 15s of fake time later.
- `settle()` advances fake time by 15s and then waits 5s in real time
for `idleWorkSnapshot().active === 0`. The full pass starts inside that
15s advance, so the follow-up is due after the window ends. The wait
then expires.
- On a fast machine the jobs usually finish after the deadline read, so
the test passes. On a slow runner the order changes, so the test fails.
- This is correct product behavior: the worker owes the follow-up pass.
The defect is in the test's timing assumption.

## What Changed

- In the busy recovered-source test, replace the bounded `settle()` loop
with a `vi.waitFor` that advances fake time by
`EXECUTION_RECONCILIATION_INTERVAL_MS` on each attempt. It waits until
every source was visited and no idle-tracked work is active.
- The test still calls `quiet(c)` after this, so it still asserts no
further scheduling and no database calls during the next 90s.
- The shared `settle()` helper is not changed, because `reschedules an
earlier deadline…` asserts an exact `nextWakeAt()` after `settle()`.

## Verification

- Red: with the 300ms delay injected (not committed), the test failed
3/3 times with the CI assertion.
- Green: with the same injection at 300ms (3 runs) and at 1000ms (1
run), all four recovered-source Retry tests pass.
- Without injection, the full `execution-control-work.test.ts` passed
5/5 times with 12 CPU-burner processes running.
- `pnpm exec vitest run src/__tests__/execution-control-work.test.ts` in
`server/`.

## Risks

- Low risk. This is a test-only change. The test still requires every
source to be visited, the worker to become idle, and then 90s of quiet.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, GitHub CLI). Default context
window. No extended thinking mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:58:33 -06:00
DottaandClaude Opus 5.5 223687bbb9 test(server): size heavy chat-channel tests to their workload to stop timeout cascades (#19404)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Chat channels connect GitHub, Slack, and other providers to tasks,
and `chat-channels.integration.test.ts` covers them against a real
embedded Postgres
> - Two parameterized tests do much more database work than the other
tests in the file, but they use the global 15s `testTimeout`
> - On a slow CI runner they time out. Vitest then keeps the timed-out
test body running, and that body breaks later, unrelated tests in the
same file
> - This failure blocks PRs that do not touch chat code
> - This pull request gives the two heavy tests a time budget that fits
their workload
> - The benefit is that a slow runner no longer turns one heavy test
into a cascade of 2–7 failures

## Linked Issues or Issue Description

**What happened?**
In run 42098299218 (job 127495756270, shard `General tests (chat
(3/3))`), 7 chat tests failed in two Vitest processes on a PR that
changed only unrelated test files:
- Process 1: `reactivates denied retry notices when authority is
restored during bounded pass` timed out at 15000ms. Then `… during
requester merge` failed with `A function to advance timers was called
but the timers APIs are not mocked`.
- Process 2: `preserves one current GitHub Retry across backlog_51` and
`… consumed_backlog_52` timed out at 15000ms. About 110 tests later,
three pre-run FIFO tests failed: `advances pre-run FIFO selection past a
suppressed page with microsecond timestamps (manual)` (`expected +0 to
be 1`), `retires an already-visible pre-run FIFO notice once without
publishing scheduler failures` (`expected [] to have a length of 1`),
and `retries a contended pre-run FIFO receipt before I/O and observes
cancellation after the lock releases` (`expected undefined to match
object { state: 'retry' }`).

**Expected behavior**
These tests pass on a slower runner. A timeout in one test does not
change the result of a different test.

**Steps to reproduce**
1. On `master`, in `server/`, run:
`pnpm exec vitest run src/__tests__/chat-channels.integration.test.ts -t
"GitHub Retry across (backlog_51|consumed_backlog_52)|advances pre-run
FIFO selection past a suppressed page|retires an already-visible pre-run
FIFO notice once|retries a contended pre-run FIFO receipt before I/O"
--testTimeout=1200`
2. The lower timeout simulates a slow runner. The two backlog tests time
out, and then the same three FIFO tests fail with the same assertions as
in CI. The `scoped` FIFO case passes, as it did in CI.
3. Run the same command without `--testTimeout`. All 6 tests pass.

**Root cause**
- The workload is large. `backlog_51` and `consumed_backlog_52` deliver
51–52 real inbound GitHub messages through the full delivery path.
`bounded pass` inserts and authorizes 100 rejected retry notices.
- CI timing data supports this. On passing runs these tests take
5.4–7.7s (backlog) and 3.6–5.3s (bounded pass), and lighter tests in the
same file take 0.5–1s. On the failing runner, the lighter tests were
2–3x slower (for example, `idle` took 1856ms instead of 500–680ms). This
put the heavy tests over 15s.
- When Vitest times out a test, it does not stop the test body. The body
keeps running while the next tests start:
- The unfinished backlog tests left 51+ `deferred_issue_execution`
inbound wakeups. The pre-run FIFO tests use unscoped, bounded passes
(`enqueueInboundWakeupPublications`, `processPendingPublications`). The
leftover rows filled those pages, so the new test's row was not reached.
- The unfinished `bounded pass` body later ran its `finally` block,
which calls `vi.useRealTimers()`. This disabled the fake timers that
`requester merge` had just installed.

## What Changed

- `preserves one current GitHub Retry across %s`: set an explicit 60s
timeout.
- `reactivates denied retry notices when authority is restored during
%s`: set an explicit 60s timeout. Increase the `bounded pass` wait for
its held 100-row reconciliation from 10s to 30s, so that the inner wait
does not become the new limit.
- The same file already uses explicit 20s, 25s, 30s, 40s, 60s, and 90s
budgets for other heavy tests. No product code changed.

## Verification

- Red: the reproduction command above fails on `master` with the CI
cascade. A tighter `--testTimeout=1600` on `reactivates denied retry
notices` also shows that later modes fail after `bounded pass` times
out.
- Green: the same reproduction command with this change (still
`--testTimeout=1200` for all other tests) passes 10/10 selected tests,
including the three FIFO tests.
- `pnpm exec vitest run src/__tests__/chat-channels.integration.test.ts
-t "GitHub Retry across|reactivates denied retry notices|pre-run FIFO"`:
46 passed.

## Risks

- Low risk. This is a test-only change. A real hang in these two tests
now fails after 60s instead of 15s. This is well inside the job limit.
- The cascade can still occur if one of these tests needs more than 60s.
That would mean a runner about 10x slower than a normal one.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, GitHub CLI). Default context
window. No extended thinking mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:40:13 -06:00
DottaandClaude Opus 5.5 ec94e43c4a test(server): budget the 130 MB isolated re-home test for its data volume (#19400)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Isolated repository re-home moves a task workspace onto the shared
project clone, so the private clone can be deleted
> - One re-home test moves two 65 MiB files on purpose, to prove that
large changes still move
> - On CI this test times out at the 15000 ms default in pull requests
that do not touch re-home
> - The test does a fixed, large amount of CPU work, and a loaded runner
cannot finish it in 15 seconds
> - This pull request gives that one test a 60000 ms budget that matches
its data volume
> - The benefit is a stable server shard, with no change to what the
test proves

## Linked Issues or Issue Description

**What happened?**

A survey of failed `PR` workflow runs found intermittent failures of one
test in pull requests that do not touch re-home:

```
FAIL src/__tests__/isolated-repository-rehome.test.ts > isolated repository source re-home > re-homes a workspace whose changes exceed the default restore point limits
Error: Test timed out in 15000ms.
```

Examples: jobs 127465954304, 127499682176, and 127538360101 (shard
`general-server-without-chat` 7/12). The survey counted 4 such failures.
The other 27 tests in the file passed in each of these runs.

**Expected behavior**

The test passes on every run, and it still proves that a 65 MiB
untracked file and a 65 MiB saved ignored file move with an exact
snapshot match.

**Steps to reproduce**

Run the unchanged test on master while the CPU is oversubscribed:

```sh
cd server
node load.mjs 72 pnpm exec vitest run src/__tests__/isolated-repository-rehome.test.ts -t 'exceed the default' --reporter=verbose
```

`load.mjs` starts 72 busy-loop Node processes (4 per core on an 18-core
machine) and runs the command. The test fails with `Test timed out in
15000ms`.

**Root cause**

The file sizes are part of the test contract. #17410 added the test to
prove two fixes that only show above 64 MiB: `applyRestorePoint` streams
`git diff --binary` into `git apply`, and the ignored-file read sizes
its buffer from the tree. So the files cannot get smaller.

Each full pass over the workspace writes both files into a Git object
store, so it hashes and deflates 130 MB. One test run makes about eight
passes: the test's `before` snapshot, the re-home `before` snapshot, the
capture, the import into the shared clone, the apply, the two
verification snapshots, and the test's final snapshot. That is about 1
GB of SHA-1 and zlib work, plus the byte comparison.

I timed the phases with the re-home hooks on an idle machine: snapshot
0.6 s each, capture and import 1.9 s, worktree add and apply 0.9 s,
verification 1.0 s. With 2 busy processes per core the same phases took
1.3 s, 3.9 s, 1.3 s, and 2.1 s. The work is CPU-bound and scales with
load. No phase waits on a timer, a lock, or a background task. The 15000
ms default is the cause.

## What Changed

- Give `re-homes a workspace whose changes exceed the default restore
point limits` an explicit 60000 ms test budget.
- No other test changes. No production changes. No retries, no skips.

## Verification

Red, on master, 72 busy-loop processes:

```
× ... re-homes a workspace whose changes exceed the default restore point limits
  → Test timed out in 15000ms.
```

Green, with this change, same load:

```
✓ ... re-homes a workspace whose changes exceed the default restore point limits 30414ms
```

So the test needs about 30 s at 4 processes per core, which is half of
the new budget. A hung re-home still fails inside 60 s.

Full file, idle: 28/28 tests pass.

```sh
cd server
pnpm exec vitest run src/__tests__/isolated-repository-rehome.test.ts
```

## Risks

- Low risk. One test budget changes.
- A real slowdown in the re-home path of up to 4 times would no longer
fail this test. Other tests in the file keep the 15000 ms default.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:31:00 -06:00
DottaandClaude Opus 5.5 9668e73b4b fix(native): continue a turn automatically when reattachment loses its runner (#19399)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs in managed sandboxes recover through reattachment: after
a server restart, and for a same-run retry when a turn ends without a
semantic result
> - Reattachment fails closed when the original runner or sandbox is
gone (`native_remote_recovery_lease_mismatch`,
`runner_remote_process_identity_unavailable`)
> - Saved-work recovery then saves the files, but the task stays blocked
with "Saved work recovered. Send a message to continue." The recovered
hold offers no Retry
> - A restart, deploy or maintenance window must not need a user message
to continue work
> - This pull request requests the task Retry automatically, once, when
it is safe, and offers Retry on the recovered hold when it is not
> - The benefit is that interrupted work continues after infrastructure
events, and a user who must act gets a Retry that works on first use

## Linked Issues or Issue Description

**What happened?**
A native task in a managed sandbox was interrupted. In one case a server
restart claimed a queued user-comment turn for reattachment. In another
case a turn completed without a semantic result, and the same-run retry
claimed the run for reattachment with no restart. In both cases the
original runner or sandbox lease was gone. The run failed with
`native_remote_recovery_lease_mismatch` or
`runner_remote_process_identity_unavailable`. Recovery saved the work.
The task stayed `blocked` with "Saved work recovered. Send a message to
continue." The execution blocker had `canRetry: false`. A manual `POST
/api/issues/:id/retry` started a run that succeeded.

**Expected behavior**
After an infrastructure interruption, the task continues automatically
when the interrupted turn's outcome is known. Otherwise the task offers
a Retry that works on first use.

**Steps to reproduce**
1. Start a native task in a managed sandbox from a user comment.
2. Lose the sandbox lease or the remote runner process (for example,
restart the server during a maintenance window).
3. Let reattachment fail and saved-work recovery complete.
4. See that the task waits for a new message and offers no Retry on the
execution blocker.

**Root cause**
Reattachment (`reattach_remote_runner`) correctly refuses to replace a
missing runner or allocation. Nothing then continued the user's request.
The automatic continuation rule only allows a new user turn after
reconciliation. `getExecutionBlocker` hid Retry whenever a workspace
restore hold existed, also after the files were verifiably recovered.
The same-run retry path also re-enters the restart-recovery owner, so
its event said "after server restart" with no restart.

## What Changed

- New `server/src/services/heartbeat/restart-interrupted-turn.ts`: after
a reattachment claim executes, if the run failed because its runner or
allocation was lost, request the same task Retry that `POST
/issues/:id/retry` creates. It runs once per run (idempotency key
`restart-interrupted-turn:<runId>`), under the authority of the user who
sent the message, assigned the task, or clicked Retry. It requires that
the run recorded no provider turn or that its last provider turn is
settled. System wakes, in-flight turns, and a Retry that this hook
already requested do not continue automatically.
- `heartbeat/recovery.ts`: call the hook after each reattachment claim.
Label same-run retry reattachment as "Reattaching to the existing remote
runner to resume this run", and add `trigger: "same_run_retry"` to the
event payload.
- `execution-blocker.ts`: a failed run whose saved work is verifiably
recovered now offers Retry (`canRetry: true`) with "Saved work
recovered. Retry or send a message to continue." An intentional Stop
keeps "send a message" only.
- `task-retry.ts`: optional `initiatedBy: "restart_recovery"` records
the activity as actor `system`/`restart-recovery` with
`onBehalfOfUserId`.
- `doc/execution-semantics.md`: document the rule.

## Verification

- New regressions in `server/src/services/workspace-recovery.test.ts`
use the real recovery, Retry and dispatch services on embedded Postgres:
- `continues a user turn once ... after $title and recovery saved the
work` (3 cases: restart lease loss before the turn, same-run retry with
the runner gone after a settled turn, same-run retry with the lease gone
after a settled turn). Each asserts one queued successor bound to the
Retry and the user's authority after recovery, idempotent hook calls,
system activity attribution, and no second automatic attempt for the
automatic Retry's own failure.
- `does not continue automatically when $title, and offers a Retry that
works on first use` (in-flight turn, system wake, unrelated error). Each
asserts no successor, `canRetry: true` on the blocker and the
interruption presentation, and one successor after one Retry click.
- Red before the fix: all 6 tests fail (no successor, blocker `canRetry:
false`). Green after the fix.
- `npx vitest run src/services/workspace-recovery.test.ts
src/services/task-retry.test.ts
src/services/task-interruption-presentation.test.ts
src/services/execution-blocker-read.test.ts
src/services/heartbeat/recovery.test.ts
src/services/explicit-native-continuation.test.ts
src/services/quarantined-task-retry.test.ts
src/__tests__/workspace-restore-repair-route.test.ts
src/services/native-runtime/native-failed-workspace-recovery.test.ts`:
1098 passed.
- Not verified on staging. After the next deploy, check a task
interrupted by a restart: it should continue without a message, or show
Retry.

## Risks

- The automatic Retry uses the original user's authority. Admission
rechecks that user's current permission, stop proof, saved-work
recovery, approvals, pauses, budget and ownership. A refused request
leaves the visible Retry.
- A settled turn that ended without a result is continued with a fresh
Retry turn from saved work. A turn whose outcome is unknown (in flight)
is not.
- The hook runs after the claim's execution returns. If the server stops
between the failure and the request, the task shows Retry instead.
- Not in scope: same-run disposition recovery still cannot start a new
runner in the existing sandbox after the original remote runner exits.
This change continues the task with one fresh Retry turn instead. If
that Retry turn also ends without a result and loses its runner, the
task shows Retry.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Related: #19355

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:29:46 -06:00
DottaandClaude Opus 5.5 e7fb554647 test(server): load setup-token route modules during suite setup (#19398)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The setup-token login route lets an owner sign in to Claude for a
sandbox environment
> - The route suite `server/src/routes/setup-token-route.test.ts` fails
on CI in pull requests that do not touch it
> - The first test in the suite times out at 15000 ms, and the other 29
tests pass
> - The first test pays the one-time import cost of the agent route
graph inside its own test budget
> - This pull request moves that import to suite setup, which has a
30000 ms hook budget
> - The benefit is a stable server shard with no change to what the
tests prove

## Linked Issues or Issue Description

**What happened?**

A survey of failed `PR` workflow runs found intermittent failures of one
test in pull requests that do not touch this area:

```
FAIL src/routes/setup-token-route.test.ts > company-and-environment setup-token route — full path > drives start, status, prompt, code, and completion with no agent id
Error: Test timed out in 15000ms.
```

Examples: jobs 127423956746 and 127457298585 (shard
`general-server-without-chat` 12/12). The survey counted 5 such
failures.

**Expected behavior**

The full-path test passes on every run. The test budget measures the
route behavior, not the module import.

**Steps to reproduce**

Run the unchanged suite on master while all CPU cores are busy:

```sh
cd server
node load.mjs 24 pnpm exec vitest run src/routes/setup-token-route.test.ts --reporter=verbose
```

`load.mjs` starts 24 busy-loop Node processes and runs the command. The
first test fails at 15005 ms.

**Root cause**

`createApp()` called `vi.importActual("../routes/agents.js")` and four
other modules on each call. The first call loads and transforms the full
agent route graph. I timed it: `createApp()` took 11670 ms of the 11698
ms test on an idle machine. The HTTP steps took less than 30 ms. On a
loaded CI runner the import alone crossed the 15000 ms test budget.
Later tests reused the module cache, so only the first test failed.

## What Changed

- Add `loadRouteModules()`, which imports the route and logger modules.
- Call it once in the suite `beforeAll`, after `vi.resetModules()` and
the mock registration. The hook budget is 30000 ms in
`server/vitest.config.ts`.
- `createApp()` now reuses the loaded modules. Each test still builds a
fresh Express app, a fresh transport, and a fresh log capture.
- This follows the pattern in
`server/src/__tests__/agent-skills-routes.test.ts` (#16396).
- No timeout changes, no retries, no skips.

## Verification

Red, on master, with 24 busy-loop processes on an 18-core machine:

```
× ... drives start, status, prompt, code, and completion with no agent id 15005ms
```

Also red on an idle machine on 1 of 3 runs (15006 ms; the other runs
took 11790 ms and 13748 ms).

Green, with this change:

- Idle, 3 runs: 30/30 tests pass. The first test takes 42 to 46 ms.
- With 24 busy-loop processes: 30/30 tests pass. The first test takes 40
ms. The file takes 19.4 s in total.

```sh
cd server
pnpm exec vitest run src/routes/setup-token-route.test.ts --reporter=verbose
```

`tsc --noEmit -p server` reports no errors in this file.

## Risks

- Low risk. The change touches one test file only.
- The import cost now counts against the 30000 ms hook budget. Locally
it is 12 to 15 s idle. The mocks register before the import, as before,
so each test sees the same mocked services.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:23:35 -06:00
DottaandPaperclip da34bcd48d fix(workspaces): resume archived imports from their saved upload (#19396)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Durable workspaces let a replacement web server restore saved task
files.
> - Legacy workspace imports keep a journal and an encrypted upload
receipt across interruption.
> - An archived Git import can rebuild a missing source from its saved
task commit and edits.
> - A fresh clone can also contain newer remote refs. Those refs change
the archive manifest even when the task files are unchanged.
> - This pull request recovers the exact authenticated upload for an
interrupted archived Git import.
> - The original journal, source binding, task binding, and execution
checks still control publication.

## Linked Issues or Issue Description

**What happened?**
An archived Git workspace import stopped after its upload persisted. A
retry rebuilt the source from the remote repository. New remote refs
changed the archive manifest, so the original upload receipt rejected
the retry with `remote_workspace_snapshot_operation_content_changed`.

**Expected behavior**
The retry must recover its original saved upload and complete the same
import. It must keep the saved task commit, staged edits, unstaged
edits, lifecycle state, and task binding. A changed source or task
binding must still block publication.

**Steps to reproduce**
1. Archive a Git task with staged and unstaged edits, save its
restoration recipe, and remove its local repository.
2. Start the workspace import and lose the upload reply after object
persistence, or interrupt its independent restore after the upload is
acknowledged.
3. Advance the remote default branch and add an unrelated branch.
4. Retry the original import. Before this change, both interruption
cases fail with the same content-change error.

**Paperclip version or commit**
Reproduced on master at `d8b0eae68772a3bd2bb4a8163b5ce95764b97cff`.

**Deployment mode**
Cloud workspace migration with external PostgreSQL and durable encrypted
object storage. The regression tests use real disposable PostgreSQL, an
HTTP Git remote, and encrypted filesystem-backed object storage.

Related merged work: #18826 and #19138. No duplicate open fix was found.

## What Changed

- Add scoped recovery of an acknowledged snapshot or a pending upload
whose object persisted. Authenticate the archive before returning it.
Recovery alone does not publish the snapshot.
- Resume archived Git imports from those exact saved bytes. Keep the
original operation and all publication checks.
- Lock and recheck each saved nested source’s company, project, and
repository identity in the final publication transaction.
- Keep mutable local sources on the existing capture and
content-comparison path.
- Add regressions for both interruption points, unrelated remote
changes, source revocation during recovery, and local edits after
interruption.
- Document the retry behavior.

## Verification

- Before the repair: both new archived retry regressions fail with
`remote_workspace_snapshot_operation_content_changed`; the
mutable-source control passes.
- After the repair: all 204 tests in the migration, snapshot storage,
archive, and archive environment suites pass. The suites use real
PostgreSQL and encrypted storage.
- Server package typecheck and full repository typecheck pass on
`bffebbab60edbdf50b42fc37a9e58beb01ceb94d`.
- An added source-revocation test initially expected a later expiry
error. The existing publication fingerprint rejected the change first.
The test now requires that earlier
`remote_workspace_migration_publication_changed` error and still checks
that authority was not published.
- The review finding was reproduced before the safety repair: changing a
saved nested repository URL after upload still allowed publication. The
new regression requires rejection, and unchanged approved nested sources
still restore.
- Full repository typecheck and build pass on the final head. The full
local test command stopped at one Slack manager authorization fixture
with `socket hang up`; the exact case then passed in isolation on the
same head, and all three CI chat shards passed. The original local
failure is retained.
- The first CI server shard run had 4,724 passes and one runtime-process
readiness failure in an unchanged fixture. The exact fixture passed in
isolation. One unchanged rerun of that shard passed. The first failure
remains recorded in the workflow history.
- The earlier candidate’s full local test run was stopped after the
confirmed review finding; that interrupted run is not reported as a
pass.
- All CI gates, including `ci / verify` and `ci / e2e`, pass on
`bffebbab60edbdf50b42fc37a9e58beb01ceb94d`. Greptile is 5/5 on that
head, with no remaining findings or unresolved review threads.

## Risks

Recovery applies to archived Git sources, whose saved recipe and
lifecycle checks still run before publication. A source change during
recovery must keep the snapshot unpublished. Missing or corrupt saved
objects still fail authentication or retain the existing import
obligation. Mutable local sources still reject changed contents. This
change adds no database migration and changes no saved archive format.

## Model Used

OpenAI GPT-6 in Codex, with code editing, tool use, and test execution.
The exact runtime model identifier and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 22:19:20 -06:00
DottaandClaude Opus 5.5 5c185db77c fix(tree-control): resume work a subtree pause interrupted (#19394)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Task tree control lets the board pause a task tree and then resume
it.
> - A pause stops the runs that are active in the tree. The release must
then continue the work that was pending.
> - A child task whose run was active at pause time did not start again
after release. The child became blocked and asked for a user message.
> - A pause is an operator hold, not a failure. It must not strand work,
and it must not require reconciliation for a run that never started.
> - This pull request re-dispatches interrupted work one time on
release, and it settles a legacy Stop that never entered the adapter.
> - The benefit is that pause and resume keep a task tree moving without
manual recovery, and real safety gates stay in force.

## Linked Issues or Issue Description

**What happened?**

A parent task delegated two children. One child's run was active (in
host workspace preparation, before the adapter started) when the board
paused the parent's tree. The pause cancelled that run. After the board
released the hold, no new run started for that child. The child became
`blocked` with the execution blocker cause
`legacy_execution_requires_reconciliation`. Later the next action became
"Send a message to continue." A sibling child that had no active run at
pause time resumed normally.

**Expected behavior**

A run that a subtree pause cancels is an operator hold. On release, the
pending work starts again one time, automatically. It does not go
through execution reconciliation when the run never entered its adapter.
It does not wait for a user message.

**Steps to reproduce**

1. Create a parent task and a child task that is assigned to an agent.
2. Wake the child. While its run is in host preparation (or queued),
call `POST /api/issues/:parentId/tree-holds` with `{ "mode": "pause" }`.
3. Call `POST /api/issues/:parentId/tree-holds/:holdId/release` without
`wakeAgents`.
4. Before this change, no new run starts, and `getExecutionBlocker`
reports `legacy_execution_requires_reconciliation` for the child.

**Root cause**

- (a) The pause records each interrupted run in the hold member snapshot
(`activeRunId`). Nothing used that record on release.
- (b) Release woke tasks only when `metadata.wakeAgents === true`.
Without it, release relied on stranded-issue recovery. That recovery
stands down when the latest run was cancelled by an operator
(`isOperatorCancelledRun`). So the interrupted child had no owner.
- (c) The pause Stop of a legacy run in preparation is written directly
as `cancelled`. Legacy terminalization classifies a cancelled run with
no bootstrap evidence as an unknown outcome. It records
`legacy_execution_requires_reconciliation`, although the gated dispatch
never entered the adapter.
- Native runs: the verified native Stop does not create the
reconciliation hold, but gap (b) applies to native runs too. The release
change is runtime-independent.

## What Changed

- `issueTreeControlService.findOwedPauseResume` decides if a task is
owed one resume. The task's latest run must be an operator Stop that a
pause hold member snapshot names, recorded while that hold was active.
The task must still be executable by the same agent, and no pause may
still cover it. The obligation belongs to the run, so overlapping pauses
keep it until the last covering release.
- New `services/issue-tree-resume.ts` re-dispatches owed work with wake
reason `issue_tree_resumed` (`holdId` and `interruptedRunId` in the
context). A compare-and-set claim (`treeHoldResume`) on the interrupted
run makes the resume happen at most once. A failed wake returns the
claim.
- Pause release and `mode: "resume"` call it for every member of the
released hold, also without `wakeAgents`. When `wakeAgents` is true, the
existing loop wakes every executable member.
- A task that still has an execution blocker is not replayed. The
release writes `issue.tree_resume_wake_skipped`, the blocker stays
visible, and the obligation stays unclaimed.
- `releaseHold` updates only an `active` hold. A repeated release gets
`409`.
- `settleCancelledLegacyPreparation`: when the executor that owned a
cancelled legacy run proves that dispatch never entered the adapter, it
records bootstrap evidence and resolves that run's
`legacy-execution:<runId>` hold as `false_positive`. It also corrects an
automatic no-replay disposition and restores a `blocked` status only
when that disposition made the change. Then it re-dispatches owed work,
for a release that happened while the Stop was unwinding. It requires a
requested Stop with no existing recovery receipt, and no process,
session, native finalization, unreleased lease, or execution-start
event.
- `settleUnrecoverableExecutions` waits while a legacy executor still
holds its controller lease, and records a `legacyCancellationBlock`
receipt when it blocks a task for a cancelled legacy run.
- Docs: `doc/execution-semantics.md` and `doc/composer-stop.md` describe
the resume rule.

## Verification

- New regression
`server/src/__tests__/issue-tree-pause-resume-run.test.ts` uses embedded
Postgres, the real heartbeat, and the real tree-control routes:
- A run paused during preparation: nothing runs during the hold, there
is no blocker, and release creates exactly one `issue_tree_resumed` run
that succeeds. A second release returns `409` and creates no run.
- Release before the stopped preparation finishes unwinding (with the
periodic disposition run in between): exactly one resumed run after the
executor settles.
- An automatic no-replay disposition that ran first: the blocker clears,
the status returns to `in_progress`, and release resumes once.
- Overlapping child and parent pauses: no resume while the parent pause
remains, then exactly one resume after it is released.
- A queued run cancelled by the pause: release creates exactly one
resumed run.
  - A task reassigned to a user during the hold: no resume.
- A mid-turn legacy Stop with unknown outcome: no replay, the
reconciliation blocker remains, and `issue.tree_resume_wake_skipped` is
logged.
- Red before each fix: the first 4 cases failed on master; the 3 review
cases failed on the first PR commit. All 7 pass now.
- Also passed locally: `issue-tree-control-routes`,
`issue-tree-control-service`, `heartbeat-workspace-busy`,
`heartbeat-undispatched-execution-wait`, `execution-control-work`,
`legacy-workspace-restore-recovery`, `legacy-continuation-authority`,
`task-retry`, `legacy-execution-recovery`, `agent-conversations`,
`heartbeat-retry-scheduling`, `satisfied-connection-intents`.
- Not verified here: a live staging pause/resume. Re-check on staging
after deploy.

## Risks

- Behavior change: release now starts the work that the pause
interrupted, also without `wakeAgents`. Before, that work stayed idle.
Other idle tasks still wake only with `wakeAgents`.
- The settlement trusts the executor's adapter-entry flag. The flag is
set only inside the atomic dispatch gate, which refuses a cancelled run.
The settlement also requires no process, session, lease, or execution
event.
- A native run whose Stop has not finalized at release time is not a
target. Stranded recovery and explicit wake still apply.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:08:01 -06:00
DottaandClaude Opus 5.5 6ca7b151f7 feat: compact an agent's context from the task Properties panel (#19378)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A task's Properties panel shows the assignee agent's context window
(paperclipai/paperclip#19273)
> - The board can see that the context is nearly full, but it cannot act
on it
> - A full context makes the agent compact at an unplanned time, often
in the middle of work
> - This pull request adds "Compact now" to the Context hover card
> - The benefit is that the board can compact the agent's context
between runs, at a time it chooses

## Linked Issues or Issue Description

Refs paperclipai/paperclip#19273

**Problem**
The Context row shows how full the agent's context window is. The board
cannot ask the agent to compact it.

**Proposed solution**
A "Compact now" button in the Context hover card. It runs the harness's
own compaction: Codex `thread/compact/start`, and Claude's `/compact`
command through ACP.

**Alternatives considered**
A separate PRP `session.compact` command. Both harnesses run compaction
as a turn, so a compaction turn keeps the existing durable turn binding,
interruption, and recovery. A separate command needs a second turn
lifecycle.

## What Changed

- UI: the Context hover card shows Compact now with three states:
available, Compacting…, and unavailable with a reason (agent mid-run,
harness unsupported, task closed). The Storybook story shows each state.
- Server: `POST /api/issues/:id/context/compact` (board only,
`decideIssueAccess` `issue:mutate`, activity log
`issue.context_compaction_requested`). It queues a short run marked by
`contextSnapshot.contextCompactionRequestId`. `GET
/issues/:id/context-usage` returns the `compaction` state.
- Server: a compaction run does no task work. The native status arbiter
preserves the task status. The lifecycle skips handoff, liveness
continuation, review-path disposition and the issue comment, as it does
for goal-control runs.
- Runner TS: `startTurn({ compact: true })`. The Codex harness session
sends `thread/compact/start`. The runnerd transport maps it to PRP
`turn.start` with `compact: true` (Codex) or `text: "/compact"`
(Claude/ACPX). The native runtime option `sessionCompaction` requires a
resumed session and settles with a synthesized yielded result.
- Runner Rust: `turn.start` accepts `compact: true` for Codex.
`CodexProvider::start_compaction_turn` sends `thread/compact/start` and
binds the turn from the `turn/started` that follows the empty response.
- Fix (from #19273): Codex emitted `context.compacted` on both the start
and the end of one compaction item. It now emits it once, and the count
is distinct by `compactionId`.
- Fix (from #19273): the conversation event view dropped
`context.compacted`, so a finished run never showed the divider. The
view now keeps compactions, plus context readings for runs that
compacted. A finished run's compactions show as thread dividers.
- Fix: the Codex harness stamps before/after sizes on the compaction
event. Codex reports the compacted size before the compaction item ends,
so the old neighbour-based guess showed the wrong size.

## Verification

- `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, `node
scripts/storybook-catalog.mjs`: pass.
- New tests: `server/src/__tests__/context-compaction.test.ts`,
`status-arbiter.test.ts`, `issue-context-usage.test.ts` (duplicate
compaction counts once), `ui/.../context-usage-row.test.tsx`,
`native-session-runtime.test.ts` (context compaction), Codex driver and
runnerd transport tests, Rust
`codex_compaction_turn_binds_the_turn_reported_after_its_empty_response`.
Rust `codex_provider` suite: 115 passed.
- `pnpm test:run` locally: one failure in
`chat-channels.integration.test.ts`. It reads
`.claude/skills/slack/SKILL.md` from the main checkout, which is not
present in this worktree. It is not related to this change.
- Live, local worktree instance, Codex agent on the Paperclip runner:
clicked Compact now. The run succeeded. Context went from 35k to 9.4k
tokens, and later 9.4k to 9.2k. The task stayed In Review. The thread
shows "Context compacted · 9.4k → 9.2k tokens" and the meter dropped.
- Live Claude proof: pending. The local Claude agent needs an Anthropic
API key. claude-agent-acp handles `/compact` as an echo-less turn, and
the runner already maps its "Compacting completed." notice to
`context.compacted`.

## Risks

- A compaction run resumes the agent's session. If the session cannot
resume, the run fails with `native_context_compaction_unavailable` and
the task is unchanged.
- Compact now is not available while the agent runs on the task, because
compaction interrupts the active turn.
- The conversation view now returns context readings for runs that
compacted. Runs that never compacted return the same events as before.
- An old runner binary ignores `compact` and rejects the `turn.start`
because it has no text. It does not start an ordinary turn.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:07:12 -06:00
DottaandClaude Opus 5.5 240d46e2ec fix(native-runtime): join the deferred observer seen when a duplicate append arrives (#19397)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runtime writes runner events through
`PaperclipControlPlanePort.appendEvent`.
> - A terminal event can defer its post-commit observer. A duplicate
append of the same event must join that observer. It must not run
`onDuplicateEvent`.
> - The duplicate looked up the observer only after its own event
transaction finished. If the original observer settled during that
transaction, the observer left the map. Then the duplicate ran
`onDuplicateEvent`.
> - This made the test "joins the exact deferred observer on duplicate
append..." fail on slow CI (job 127633060467, and once on Windows).
> - This pull request captures the pending observer when the duplicate
arrives. The join decision no longer depends on database latency.
> - The benefit is a deterministic join and a stable test.

## Linked Issues or Issue Description

**What happened?**
The server test `paperclip-control-plane-port.test.ts > joins the exact
deferred observer on duplicate append and replays it after a new port
opens` failed intermittently with `Error: must join the original owned
observation`. The test gives the duplicate append 30 ms and then
releases the original observer. On a slow runner, the duplicate's event
transaction took longer than 30 ms. The original observer settled and
its `.then` handler removed the map entry. The duplicate then found no
observer after its transaction and ran `onDuplicateEvent`. (The CI
failure took 90 ms.)

**Expected behavior**
A duplicate append that arrives while the original deferred observer is
pending joins that observer. Database latency does not change this
result.

**Steps to reproduce**
1. Add a 60 ms sleep inside the `appendEvent` transaction when the
receipt disposition is `duplicate`.
2. Run `pnpm vitest run
src/services/native-runtime/paperclip-control-plane-port.test.ts -t
"joins the exact deferred"` from `server/`.
3. The test fails with `must join the original owned observation`. This
is the same error and stack as CI.

**Paperclip version or commit**
`master` at d8b0eae687.

## What Changed

- `appendEvent` now reads the pending deferred observer before it opens
the event transaction. After the transaction, it uses that observer. If
the duplicate did not see an observer on arrival, it reads the map again
after the transaction, as before.
- A new regression test holds a `FOR UPDATE` lock on the run row from a
second transaction. This forces the duplicate append to wait in
persistence while the original observer settles. The test then checks
that the duplicate joined and that `onDuplicateEvent` did not run. No
sleeps or product hooks are necessary.

## Verification

- Red: without the product change, the new test fails (`expected 1 to be
+0`: `onDuplicateEvent` ran once).
- Green: with the change, `cd server && npx vitest run
src/services/native-runtime/paperclip-control-plane-port.test.ts` passes
95/95. The two "joins the" tests pass on 5 consecutive runs.
- Injected 60 ms delay on the duplicate transaction: the original test
fails without the fix and passes with the fix. The delay was local only
and is not in this PR.
- `pnpm --filter @paperclipai/server typecheck` passes.

## Risks

- Low risk. The change affects only a duplicate append that arrives
while a deferred observer for the same event is pending. Such a
duplicate now always joins that observer. Before this change, it joined
only when its transaction finished first. If the original observer
rejects, the joined duplicate gets the original rejection, as a fast
duplicate did before. Committed appends and duplicates with no pending
observer behave as before.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (shell, file edits, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1106.0-canary.3
2026-11-05 21:51:47 -06:00
Nicky Leach 879140cbbd Retain ACPX read events before idle sidecar replacement (#19290) 2026-11-05 19:34:59 -08:00
DottaandClaude Opus 5.5 958f9aa9f3 feat(issues): limit agent-created task trees to parent and child (#19312)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents split work into tasks. A task can have a parent task.
> - Today agents can nest tasks to any depth: parent, child, grandchild,
and deeper.
> - Deep agent-made trees are hard to follow, and an agent that works on
a child task makes grandchildren by default.
> - This pull request limits agent-created trees to two levels: a
top-level task and its children.
> - A request for a grandchild does not fail. The server puts the new
task beside the requester and tells the agent.
> - Board users keep full control. They can still nest to any depth.
Existing deep trees stay valid.
> - The pull request also records which task created each new task, so
the audit trail stays complete when the parent changes.

## Linked Issues or Issue Description

No public issue exists. This is the feature description. Related: #16853
(budgets that include descendant work) also reasons about task trees;
this change does not alter its traversal.

**Subsystem affected**

Issues (tasks): create, child create and move in the issue service and
routes. Agent tools, chat suggestions and creation provenance use these
paths.

**Problem or motivation**

Agents create subtasks as deep as they want. An agent that runs on a
child task gets that child as the default parent, so its follow-ups
become grandchildren. The trees become deep and hard to read. The parent
link is also the only record of which task created a follow-up, so that
record is lost when the parent changes.

**Proposed solution**

Limit agent-created trees to a parent and its children. When an agent
asks for a grandchild, put the new task under the requester's parent and
return `parentAdjusted`. Do not reject the request. Keep board users
unrestricted. Record the originating task in a new server-set
`origin_issue_id` column. Keep `origin_run_id` as the originating run.

**Alternatives considered**

Reject grandchild requests with an error. We did not do this, because
agents then fail mid-task. Add a database constraint. We did not do
this, because existing deep trees must stay valid and nesting can return
later.

## What Changed

- Add `MAX_AGENT_TASK_HIERARCHY_DEPTH` (2) and the
`IssueParentAdjustment` type in `packages/shared`.
- Add `resolveNewTaskParent` and `planIssueParentChange` in
`server/src/services/issues.ts`. `create`, `createChild` and `update`
use the agent limit by default (`parentDepthPolicy: "flatten"`).
- The issue create, `/children` and PATCH routes pass `allow_nested` for
board users. For agents, the routes check authorization against the
parent that the server actually writes.
- The requester keeps its behavior when the server moves a task beside
it: `blockParentUntilDone` blocks the requester, and the new task
inherits the requester's workspace and privacy.
- The server never moves a task above a low-trust boundary root that
contains the requester.
- These subsystems keep any depth, because they find their work by
parent: watchdog reviews, monitor recovery, email-channel placement, and
routines that a person configured.
- An agent move of a task that has children skips only the parent change
and returns `reason: "has_children"`.
- Add `issues.origin_issue_id` (nullable, `on delete set null`, indexed)
in migration `0434`. Only the server sets it. The `createdFrom` detail
field, the `createdFromIssueId` filter and task-creation provenance read
it first.
- Chat prompts now let a chat agent create one task, a task with
children, or separate tasks. The conversation is never a parent.
- The runner `create_task` tool accepts an optional `parentTaskId`. A
chat agent can create a top-level task and then put subtasks under it.
The run must be able to read and change the parent. A conversation is
never a parent. The depth limit still applies. Project tools map the
field to `parentId` through the issue route. The protocol manifest,
semantic contracts and eval fixture are regenerated.
- Watchdog runs that have an active watchdog keep their requested parent
on all create paths, because their access is limited to the watched
subtree.
- Fix a bug that the test drive found: the chat (MCP) `create_task`
ignored the requested `status`, so a `backlog` task became `todo` and
started to run. It now uses the same rule as the native tool.
- Update `skills/paperclip`, `doc/SPEC-implementation.md`, and add the
plan `doc/plans/2026-11-05-two-level-task-hierarchy.md`.

## Verification

- `cd server && npx vitest run
src/__tests__/agent-task-hierarchy-depth.test.ts
src/__tests__/agent-task-hierarchy-depth-routes.test.ts`: 19 new tests
for service and route behavior, all pass.
- Related suites (64 files: issues service, thread interactions,
privacy, watchdogs, provenance, runner tool authority, decisions,
routines, project tools): 1208 tests pass. Two tests changed because of
the intended behavior and pass now. Two `email-channels.integration`
tests fail locally because the test cannot find a bundled skill file
from this worktree path. This change does not touch that code.
- `packages/db`: migration safety, snapshot drift and journal tests
pass.
- Typecheck passes for `server`, `packages/shared`, `packages/db`, `cli`
and `ui` (local `ui` errors come only from generated Storybook catalog
files that are absent).
- Local test drive on an isolated instance with real Claude agents: 29
of 29 use cases pass, including an agent on a child task (follow-ups
placed beside it, with the notice) and a chat that creates a top-level
task with three subtasks.
- I did not run the full `pnpm test:run` or `pnpm build` locally. CI
runs them.

## Risks

- Behavior change for agents: some follow-ups that were grandchildren
are now siblings. The parent of the requester is woken when all its
children finish, which now includes these tasks.
- The runner `create_task` input contract adds one optional field, so
the protocol manifest and the pack contract change. The tool output does
not change: it shows the actual `parentId`, and the activity log records
`parentAdjusted`.
- Migration `0434` adds a nullable column, a foreign key and an index.
It does not backfill rows.
- `IssueCreationSource.run` is now nullable.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 21:22:33 -06:00
Dotta d8b0eae687 test(evals): pin completed timeline command account for full-ten measurement (#19395)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The chat catalog needs fixed source and explicit conditions for each
measurement.
> - The latest timeline run delivered a complete account within its
original deadline.
> - Its assertion missed ordinary exitCode notation and a command with
an omitted subject inside a named worker bullet.
> - The reviewed source corrects those bounded textual checks while
retaining independent history and command proof.
> - This pull request pins that immutable source for one full-ten
measurement.
> - Original failed grades and unresolved Company timing remain visible.

## Linked Issues or Issue Description

Refs: #19392, #19086, #19388.

This follows the prior source pin. The latest timeline response
correctly described requirements, a real failed command, saved pause,
human reassignment, repair and successful verification. Its original
grade still failed one narrative assertion. This pin carries the
reviewed correction without merging the full catalog or its production
overlays.

## What Changed

- Pin source `936c3bbe0e83ca15616254f23d9023b2065fc5fb`, tree
`e5c64d67935497164b098e31e995f87ba7875d06`.
- Advance only the timeline oracle from v11 to v12 in the candidate
snapshot and manifest.
- Retain all 244 current queries, contexts, expected outcomes, model,
credentials, clocks and human/native accounting.
- Record one planned full-ten bounded measurement at native cap 2 and
zero automatic retries.
- Change only the manifest, its target assertion and the exact candidate
descriptor snapshot.

## Verification

- I verified the immutable source branch readback before I published
this pin.
- The new narrative control fails on the parent. The fixed source passes
300 timeline, native-proof and declaration controls, including all 242
lazy declarations with zero fixture callbacks.
- The response from result 5094 is accepted by the local diagnostic. The
original grade is retained as FAILED; this is not a new native PASS.
- Fresh full runner E2E types and the SDK guard over 506 inputs pass in
the qualified source graph. An earlier isolated worktree type attempt
had dependency-alias errors; the clean source qualification supersedes
that setup gap.
- Controller: `node --test
tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs
tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs
tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs` — 40
controls pass.
- The snapshot equals the committed source descriptor bytes. All 244
rows compare equal to the prior pin except the timeline oracle version.
- Source descriptor SHA256:
`19185aa5298c93d159c3d8547661fd6ef22185e74be1c18b8497bae2ef8051c7`.
- Controller manifest SHA256:
`822494b170139a8204ff36b33891e2f4960ac0aba6800e9856f2d25dd24de7c6`.
- Launcher, protected evidence wire and the accepted 506 SDK inputs
remain unchanged. The source SDK registry stays
`efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`.

- Exact pin head `31f47fcdfb600e6b15b35ed50c2a447d03681f06` has 56
successful checks and three configured skips. Both protected checks
pass. Greptile is 5/5 for this head with zero open threads. Paperclip
Review completed neutral at 5/5. GitHub reports CLEAN and MERGEABLE.
These checks do not qualify the unrun native experiment.

## Risks

- Original timeline result 5094 remains FAILED. All three native runs
succeeded and the account was complete in 248,876 ms, but the original
narrative check failed. Local calibration does not rewrite that grade.
- Original Company result 5093 ended without first-CEO completion. Its
model/runtime timing cause remains unresolved. This source makes no
Company runtime or latency fix claim.
- The full-ten measurement is unrun. It is a bounded experiment, not
proof of deterministic timing, reliability or an all-ten pass.
- Company retains two base human inputs plus at most one independently
verified UI approval, minimum two and maximum three native runs, and
480,000 ms. Timeline retains three native runs, one human input and
720,000 ms. No condition is waived.
- Secret journeys still require independent semantic safety review.
Metadata shape alone does not establish that a proposed name is
non-sensitive.
- The interaction-accept and API-search overlays remain separate,
reviewed, unmerged core PRs. This pin does not merge them or the full
catalog. The 196/244 admission gate stays fixed.

## Model Used

OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, repository
tools and code execution. The host does not expose an exact
context-window size. No model or paid provider was called for local
controls.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-05 21:13:17 -06:00
f8cb4957dc fix(ui): default Tasks to All tasks with a neutral sort header (#19393)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Tasks page helps people inspect work across their company.
> - A new viewer currently starts in Mine and can miss other company
tasks.
> - The task table also shows an active sort header before the viewer
selects a sort.
> - This pull request starts new viewers in All tasks and keeps the
default sort header neutral.
> - People see company work in newest-update order and keep their saved
view choices.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The initial task view and the task table's default sort state.

**Subsystem affected**

ui/ — React board UI.

**Current behavior**

Tasks opens Mine when the viewer has no saved view. The table highlights
Last updated even when the viewer has not selected a column sort.

**Proposed behavior**

Tasks opens All tasks when no saved view exists. The table keeps
newest-updated tasks first without an active header or arrow. A column
selection or an explicit sort URL activates the selected header.

**Reason and benefit**

New viewers can inspect company work immediately. The header
distinguishes default ordering from an explicit sort choice.

**Breaking changes**

The first-visit view changes from Mine to All tasks. Saved views and
explicit view URLs keep priority. Existing Inbox and Needs attention
preferences retain their meaning. The change does not alter task access
or database contracts.

**Additional context**

Refs #19363. This change builds on the grouped task menu from that
merged pull request. It uses the existing menu and design tokens.

## What Changed

- Default the Coordinator and combined task surfaces to All tasks.
- Preserve saved personal views and migrate previous picker choices.
- Keep implicit Last updated descending order separate from active
header state.
- Test default ordering, explicit sorting, reset behavior, view
selection, and preference migration.
- Update the product and implementation docs.

## Verification

- Passed 118 focused tests across task routing, table sorting, views,
and collection preferences.
- Passed `pnpm check:token-gates`, `pnpm -r typecheck`, and `pnpm
build`.
- Full `pnpm test:run` is running. Its result will be updated here.
- Greptile passed the current head with 5/5 and no findings or open
review threads.
- Browser check: All tasks uses the grouped My work and Organization
menu. Last updated has no active highlight by default. Selecting the
header adds the sort URL and active direction. Removing the explicit
sort restores the neutral header.
- Manual check: Open Tasks with no saved task view. Confirm All tasks
opens. Select Mine, leave Tasks, and return. Confirm Mine remains
selected.

## Risks

- The new default shows more accessible company tasks on the first
visit. Saved view choices remain intact.
- Task tables share the sort-state change. Explicit sort links and
column clicks retain active styling. Default row order stays
newest-updated first.
- No schema, API, permission, or execution changes.

## Model Used

OpenAI Codex, GPT-6 family. This session does not expose the exact model
ID or context-window size. Used reasoning, code editing, terminal
commands, and browser verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1106.0-canary.2
2026-11-05 18:44:59 -08:00
Nicky Leach 5ab7b78f2b Runner: retain the provider across hosted park and reclaim (#19140) 2026-11-05 18:40:44 -08:00
DottaandPaperclip 7c04adb2d6 test(evals): pin company environment and API search repairs (#19392)
## Thinking Path

> - Paperclip manages AI agents and their governed actions.
> - The TestOps controller runs an immutable product-eval source.
> - The last company-secret attempt rejected a safe metadata question
before private entry.
> - A governed approval also had a redundant server notification and
wake path.
> - Natural-language API search could hide the resource needed for a
timeline answer.
> - This pull request pins the reviewed source and its exact descriptor
snapshot.
> - New native evidence must still prove the affected journeys.

## Linked Issues or Issue Description

Refs #19381, #19093, #19387, #19388. This is the next controller pin for
those reviewed changes. The core pull requests remain unmerged.

## What Changed

- Pin source `2c99c248e839f41b520fc2d7e3702169ac6cb0a2` and its exact
descriptor snapshot.
- Advance company oracle v12 to v13. Admit bounded environment-name
questions without a required authored pattern or exact prompt template.
Preserve independent human-answer validation.
- Include the reviewed interaction-accept notification and API-search
ranking overlays. Record their route and catalog dependencies in the
company and timeline provenance.
- Retain the existing cap of two, zero retries, DeepSeek1017 model, and
all 244 current YAML definitions. Company keeps 480000ms, native2..3,
base human2 plus one independently proved approval. Timeline
keeps720000ms, native3/human1.

## Verification

- Source:396 controls in9 suites,2046 API catalog controls,145 route
controls, full runner TypeScript check, and unchanged SDK506 input guard
passed. Fresh-master API PR has2059 controls; its other baseline tests
are absent from this catalog source.
- Controller:133 contract, API, adapter, reconcile, streaming and
manifest controls passed. Existing report-lock preparation returned
`already-reviewed`.
- All current YAML definitions and launcher bytes remain unchanged from
the prior source. No model or native eval ran for this pin.
- The original company5059 and timeline5060 remain failed. Company
private entry, approval and final binding remain unproved on this
successor.
- Exact-head normal checks are terminal:56 success,3 intentional skips,
and one neutral Paperclip review. Greptile and Paperclip both report5/5
with no findings; review threads are empty. The next TestOps/AWS run
follows the protected controller merge.

## Risks

The new source is not a native feature pass. Its effect on model
completion is unproved. The original480000ms company budget may still be
too short for three native turns. No budget change or count waiver is
included. The full catalog remains outside master and its existing
admission gate remains unchanged. The controller reuses byte-identical
SDK506 evidence; no compiler or registry renewal is claimed. Actual
post-merge helper imports still need qualification against the merged
controller lock. The CI virtual merge includes a newer upstream lock
than the local frozen dependency proof.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with repository tools and code
execution. The host did not report its context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is5/5 with no open findings
- [x] I will address reviewer comments before requesting merge

Co-authored-by: Paperclip <paperclip@paperclip.ing>
2026-11-05 20:05:16 -06:00
Devin FoleyandClaude Opus 5.5 4033f7f17f fix(recovery): let a task Retry run keep its bounded transient retry (#19384)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A user can click Retry on a blocked task. Paperclip then admits one
new run that continues in the saved workspace
> - That run can fail before the provider starts, for example when the
download of the saved workspace snapshot gets a transient `ECONNRESET`
> - The scheduler classifies this failure as transient and tries to
schedule the normal bounded retry
> - The automatic-retry admission accepts only continuations that a user
comment started. It refuses a Retry continuation (`commentId: null`,
`taskRetryRequestId` set) with `continuation_user_authorization_missing`
> - So the task does not retry. Paperclip escalates the restore failure,
and the task becomes blocked again after one network reset
> - This pull request lets a plain task Retry renew its authority for
the bounded transient retry, with its own receipt for each new run
> - The benefit is that a transient setup failure after Retry uses the
normal retry budget and does not block the task

## Linked Issues or Issue Description

**What happened?**
A run started by the task Retry button failed during setup:
`remote_workspace_restore_failed: step=snapshot_download;
storage=transport_closed code=ECONNRESET`. The run event said "The
automatic retry could not revalidate the original user continuation."
(retry reason `transient_failure`, attempt 1 of 2). No retry was
scheduled, and the task became blocked.

**Expected behavior**
A transient setup failure of a Retry-admitted run gets the same bounded
automatic retry as a run that a user comment started.

**Root cause**
`admitExplicitContinuationRetry` in
`server/src/services/explicit-native-continuation.ts` required
`explicitUserContinuation.commentId` to be a GUID. A Retry continuation
has `commentId: null`. Also, dispatch in
`server/src/services/execution-continuation.ts` bound a Retry receipt to
exactly the run that it admitted, so a successor run could not prove its
authority.

Refs #19207 (it named remote restore failures and left a board recovery
action)
Refs #19340 (it retries quarantined tasks in their saved workspace)
Refs #19326 (same bounded retry path for an out-of-date Runner)

## What Changed

- `admitExplicitContinuationRetry` now accepts two kinds of authority.
The comment path is unchanged and moved into
`readCommentRetryAuthority`. The new `readTaskRetryRetryAuthority`
accepts a plain task Retry only. The Retry request must be coalesced,
from a user, with the same generation and actor, strategy `continue`,
and no chat, queued-message, canonical-intent or fresh-workspace
readiness. Interrupt and failed-run retries stay one-run.
- The successor run gets its own `explicit_user_continuation_retry`
receipt with the same `taskRetryRequestId` and actor and
`automaticRetry.sourceRunId` set to the parent. The Retry request stays
bound to the run that it admitted.
- Dispatch (`buildExecutionContinuation`) accepts a Retry successor only
from its own receipt, its exact `retryOfRunId` parent, the current
`executionRunId` and the unchanged Retry intent. The direct path for the
admitted run is unchanged.
- Tests in `server/src/services/task-retry.test.ts`: one regression for
the full retry chain, and eight refusal cases.

## Verification

- Red: before the fix, the new test failed with `{ outcome:
"not_scheduled", errorCode: "continuation_user_authorization_missing"
}`. This is the same refusal as the incident.
- Green: `pnpm --filter @paperclipai/server exec vitest run
src/services/task-retry.test.ts` gives 58/58.
- The regression test proves these points: the first retry is scheduled
with `{ previousRunId: parent, commentId: null, taskRetryRequestId }`;
the Retry request stays bound to the admitted run; dispatch accepts the
successor; a repeated schedule call adopts the same successor without a
second receipt; a second transient failure retries from its immediate
parent; the budget then stops with `retry_exhausted`; dispatch refuses
the successor when its own receipt is removed.
- The refusal cases prove that no successor and no receipt are created
after: wrong actor, unadmitted request, wrong generation, request bound
to another run, chat Retry, fresh-workspace readiness, reassignment,
superseding work.
- Nearby suites pass (601/601): `execution-continuation`,
`legacy-explicit-continuation-recovery`, `explicit-native-continuation`,
`execution-continuation-provenance`, `quarantined-task-retry`,
`issue-queued-comments-routes`, `task-retry`.
- I did not run the layered server typecheck locally because of a disk
limit in the sandbox. CI must confirm it.

## Risks

- Behavior change: a plain task Retry run that fails transiently now
retries automatically within the existing bounded budget (2 attempts).
The scheduler still decides which failures are transient. The same rule
already applies to runs that a user comment started.
- Chat, queued-message and fresh-workspace Retries, failed-run retries
and Interrupt keep their one-run contracts.
- No schema or migration change.
- Documentation gap: `doc/execution-semantics.md` (line 1210) says
"One-run Retry and Interrupt intents retain their separate admission
contracts; they are not renewable user-message authority." That sentence
must change to describe this rule. The author's publication path cannot
carry the full 320 KB file, so that one-sentence edit is still pending.
The proposed text: "A plain task Retry that continues in the saved
workspace renews the same way: the successor's receipt names the same
Retry request and actor, while the Retry request itself stays bound to
the run it admitted. Dispatch accepts that successor only from its own
receipt, its exact retry parent, and the unchanged admitted Retry
request, so a transient failure before or during that run, such as a
reset saved-workspace download, uses the normal bounded retry budget
instead of blocking the task. Chat, queued-message and fresh-workspace
Retries, failed-run retries, and Interrupt intents retain their separate
one-run admission contracts; they are not renewable authority."

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code as a Paperclip agent, with tool use and code execution (shell,
Vitest on embedded PostgreSQL).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (the
`doc/execution-semantics.md` sentence above is pending)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:54:00 -08:00
Nicky Leach 96516a8622 Keep ACPX output while Core reconnects (#19365) 2026-11-05 17:44:00 -08:00
github-actions[bot]andlockfile-bot 675f7283dd chore(lockfile): refresh pnpm-lock.yaml (#19389)
## Thinking Path

> - Paperclip uses a dependency lock for reproducible builds.
> - Dependency changes on master need matching lock entries.
> - Paid eval reports also pin a reviewed offline correction.
> - Reports must install before paid model work starts.
> - This automated PR keeps the lock and report proof together.

## Linked Issues or Issue Description

**What happened?**

Dependency manifests on master changed after the last committed lock
refresh.

**Expected behavior**

Frozen installs must match those manifests. Eval reports must retain
matching offline lock proof.

**Steps to reproduce**

Run a frozen install before and after this lock refresh.

## What Changed

- Refresh the dependency lock from the manifests on master.
- Include the matching offline report correction and hashes.

## Verification

- Pinned pnpm resolution completed with lifecycle scripts disabled.
- The report proof refresh accepted the reviewed committed baseline.
- The workflow rejected changes outside the lock and report proof files.

## Risks

The lockfile bot merges this PR without CI or review when every added
package version is at least 3 days old on npm. Otherwise the PR waits
for a maintainer, and the daily refresh tries again. The bot merges only
the exact commit this workflow generated from manifests already reviewed
on master. Unknown report lock baselines remain blocked. This update
changes no permissions or paid enable gates.

## Model Used

None — generated by pnpm and the deterministic report-lock updater.

## Checklist

- [x] Thinking path and problem description included
- [x] Generated lock and matching report proof included
- [x] Release-age gate decides between bot merge and maintainer review

Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>
2026-11-06 01:42:27 +00:00
c9c4949c86 build(deps): bump zod from 4.4.3 to 4.6.6 (#17364)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The shared validators use zod to check API input, and the Runner
contracts check some of the same values again
> - zod below 4.6.6 has a high-severity memory issue
(GHSA-mrhx-c7xc-m3w5), so this bump is necessary
> - zod 4.6 also changes string `min`, `max`, and `length`: they now
count Unicode code points, not UTF-16 units
> - The Runner question parser and the goal readers still use JavaScript
string length, so emoji text can pass zod and then fail later
> - This pull request bumps zod and keeps the string-length bound on
those schemas
> - The benefit is that we get the security fix, and these inputs still
fail at validation

## Linked Issues or Issue Description

Refs GHSA-mrhx-c7xc-m3w5.

**What happened?**
With zod 4.6, three CI tests failed on this bump. Two showed an emoji
body that is over the agent commentary ceiling and that zod now accepts.
The third is the scheduler timeout described under Verification.

**Expected behavior**
A dependency bump does not widen an API limit. Text over the limit gets
a validation error.

**Impact**
Emoji text between the two bounds passed validation and failed later:
- A question set is created, and then each answer gets a 422. The
question stays pending and nobody can answer it. A schema and parser run
confirmed the mismatch. This was not run over HTTP.
- The Muse route returns a 500 because `parsePaperclipQuestionSet`
throws an error that is not an `HttpError`. This comes from reading the
code. It was not run.
- A Runner goal normalizes to null and replaces the stored goal, or it
is marked blocked. This comes from reading the code. It was not run.

## What Changed

- Bump zod to `^4.6.6`. The lockfile resolves 4.6.7. (Dependabot)
- Add `maxStringLength` to `packages/shared/src/validators/text.ts`. It
checks zod `max` and JavaScript string length.
- Use `maxStringLength` for the question-set text fields, the Runner
goal objective, and the agent commentary body.
- Add regression tests for the question-set bound and the goal objective
bound.
- Dependabot recreated the branch on current `master` (`@dependabot
recreate`) to resolve `pnpm-lock.yaml` conflicts. This happened twice.
The lockfile is Dependabot's. Each time, the two fix commits were
applied again without changes.

## Verification

- Compared zod 4.4.3, 4.6.5, 4.6.6, and 4.6.7 directly. The change to
`max` is in 4.6.5 or earlier, and 4.4.3 does not have it.
- Passed locally: `packages/shared` tests `agent-commentary`,
`validators/issue`, and `runner-goal`, plus `tsc --noEmit` in
`packages/shared`.
- Passed locally on the first fix commit: server
`agent-commentary.integration`, `agent-commentary-helper`, and
`openapi-routes`. The later commit changes these schemas again. CI
passed these tests at `b5e9dc7352`.
- Local testing was focused. GitHub CI is the full regression check. CI
passed at `b5e9dc7352`, including `ci / verify` and `ci / e2e`.
- Greptile reviewed `b5e9dc7352`. The score is 5/5, with no inline
comments.
- After the recreate, the fix commits applied cleanly as `bad62c3ba5`
and `3294552e65` on Dependabot's `46789d1487`. Their diff is the same as
the diff Greptile reviewed. The lockfile still resolves zod 4.6.7. The
focused `packages/shared` tests (`validators/issue`, `runner-goal`) and
`tsc --noEmit` in `packages/shared` passed again locally. A second
`pnpm-lock.yaml` conflict came before CI finished at `3294552e65`.
- Second recreate: Dependabot ignored the first `@dependabot recreate`
(no reaction and no update job after 20 minutes). It acted on the second
command and pushed `83c328c15c`. The fix commits applied cleanly as
`dd3e87d93e` and `95b450048f`, with the same diff. The lockfile still
resolves zod 4.6.7. The same focused tests and `tsc --noEmit` passed
again locally. Greptile first scored `95b450048f` 4/5 with one P2: the
description did not explain the source changes. Greptile read
Dependabot's replacement description, which was live during that review.
The description was restored and the thread resolved. The new review
scored `95b450048f` 5/5 with no findings. CI failed at `95b450048f` in
two shards. Neither failure comes from the zod change:
- `server (9/12)`: `server/scripts/reporting-lock-policy.test.mjs` fails
with "Reporting lock changed; review the dependency correction before
install". #19283 added this test today. It requires the committed
`pnpm-lock.yaml` hash to match `tests/runner-e2e/report-lock.json`.
Every pull request that changes the lockfile fails it until the hashes
are renewed, for example #17172 and #17911.
- `workspaces-a (1/2)`:
`cli/src/__tests__/company-import-export-e2e.test.ts` timed out waiting
for its child server. The same test failed the same way on two unrelated
branches in the same hour (runs 42115721271 and 42109846042). A rerun of
this shard at `95b450048f` passed. The rerun of `server (9/12)` failed
again on the reporting-lock test.
- #19356 fixed the reporting-lock gate for Dependabot lockfiles and
merged. A re-run reuses the old merge commit, so the empty commit
`482d76686e` starts a run against current `master`. `pnpm-lock.yaml` and
the reporting-lock proof on `master` did not change since this branch's
base.
- CI at `482d76686e`: `ci / policy` passed, and `server (9/12)` skipped
the reporting-lock test as #19356 intends. `e2e shard (6/8)` failed once
in `cloud-auth.spec.ts` ("apiResponse.json: Response has been disposed",
a Playwright race in a route handler). The same test passed at
`95b450048f` with the same code. A rerun passed. All checks pass,
including `ci / verify` and `ci / e2e`.
- Earlier failure: `issue-monitor-scheduler.test.ts` ("lets the board
check a scheduled monitor on an in_progress task") timed out in CI at
`d60b5f56d8`. It passed 5 of 5 local runs with the first fix. The cause
is unknown. The test waits 3 seconds for heartbeat runs to settle, so CI
load is possible, but this is not proven. The test passed in the CI run
at `b5e9dc7352`. This shows recovery. It does not explain the earlier
timeout.

## Risks

- All other zod string bounds now count code points. Emoji text can be
up to 2N UTF-16 units and 4N UTF-8 bytes (it was N units and at most 3N
bytes). This pull request does not change those bounds.
- An audit of those bounds found no other concrete failure. It checked
index limits, byte limits, `.length` guards, provider limits, and
security `min` checks. It used code search and sampling, so it is not
exhaustive.
- Not verified: Slack `topic` and `purpose` use `.max(250)`, and we did
not confirm how Slack counts that limit.
- `.min(N)` is now stricter for emoji text. A single emoji no longer
passes `.min(2)`.
- `@dependabot rebase` or `@dependabot recreate` on this branch will
remove the manual commits. This happened twice to resolve lockfile
conflicts, and the commits were applied again each time.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution, in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused tests only; see
Verification)
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
documentation change needed)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green (at `482d76686e`)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(5/5 at `95b450048f`)
- [x] I will address all Greptile and reviewer comments before
requesting merge

---

Bumps [zod](https://github.com/colinhacks/zod) from 4.4.3 to 4.6.6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/colinhacks/zod/releases">zod's
releases</a>.</em></p>
<blockquote>
<h2>v4.6.6</h2>
<blockquote>
<p>Six malicious commits pushed to <code>main</code> on September 21
with a compromised git credential were reverted the same day in <a
href="https://redirect.github.com/colinhacks/zod/issues/6620">#6620</a>
and have since been removed from the history of <code>main</code>. None
were ever published to npm. The npm provenance for 4.6.6 names 284243fb,
the pre-rewrite copy of 9c219833 with identical contents.</p>
</blockquote>
<h2>Commits:</h2>
<ul>
<li>0cd3af5e9f50ad11bbcd4407a46a21b8848ba574 docs: load the en locale
directly in Zod Mini</li>
<li>9d5e60f162997b764b40d45aa02481888db50dea docs: import locales from
zod/locales, not z.locales</li>
<li>34a626f307871ecbb0123947559328dc52ed5e3e docs: tighten the z.locales
tree-shaking note</li>
<li>74304ce5b5796b5edb9a88ceb8e170b6ea3e82df release: stage npm versions
instead of publishing them (<a
href="https://redirect.github.com/colinhacks/zod/issues/6622">#6622</a>)</li>
<li>b2ef983e0711ff6517d6020b021071e2e5653efa docs: add Code for Japan to
bronze sponsors</li>
<li>8fb2faf0584828a6a027d8279db36f6f52784462 docs: crop the Code for
Japan logo and list it first in bronze</li>
<li>bacad518c41610d654658172d325874bf0ee6290 docs: square the Code for
Japan logo to match the other bronze tiles</li>
<li>c42596bd1c87e941d7241d10e012c5c196adcc10 release: hold the run on
the release-approval environment until the staged versions are approved
(<a
href="https://redirect.github.com/colinhacks/zod/issues/6638">#6638</a>)</li>
<li>a2ba4a67986a3e6b2b131be30084a2f460d9ea67 fix(types): declare record
output and input as members, not base type arguments (<a
href="https://redirect.github.com/colinhacks/zod/issues/6645">#6645</a>)</li>
<li>a0ebf970f2d752e5f58527cd2942ff48be4f46bd fix(v4): check array, set
and map size bounds before parsing elements (<a
href="https://redirect.github.com/colinhacks/zod/issues/6652">#6652</a>)</li>
<li>529f78dd9431b07f49647fee3aa0819b3f8b66b7 4.6.6</li>
<li>9c219833c3a3764fa1f1ea9c5ecec8254a63c0b1 ci: stage at verbose so a
failed OIDC token exchange says why</li>
</ul>
<h2>v4.6.5</h2>
<h2>Commits:</h2>
<ul>
<li>d2b135cfb7a3582b9eb515756b9166bcb9521f4a docs: add the 4.6.x patch
highlights to the 4.6 post</li>
<li>f1448f7cee00df9fe1e9ad84a000aa1828cc8bc1 docs: fold the 4.6.x patch
highlights into the 4.6 post's own sections</li>
<li>de65a5cb39ed22a507fac935788f718fa88d104f docs: lead the properties
section with the check and add a Zod Mini tab (<a
href="https://redirect.github.com/colinhacks/zod/issues/6598">#6598</a>)</li>
<li>56222cd1532c07bcb91b67df529cab4c0a215330 feat(instanceof): key the
.properties() shape off the instance type (<a
href="https://redirect.github.com/colinhacks/zod/issues/6600">#6600</a>)</li>
<li>ca0229a404818290e6cdcfefcd7eb2d04bcbb543 Revert &quot;feat: add
z.currencyCode() over a vendored ISO 4217 list, refreshed weekly by CI
(<a
href="https://redirect.github.com/colinhacks/zod/issues/6595">#6595</a>)&quot;</li>
<li>cc4cd4ee9c52fcaa10964e48cc144541e41a5ed9 Revert &quot;Revert
&quot;feat: add z.currencyCode() over a vendored ISO 4217 list,
refreshed weekly by CI (<a
href="https://redirect.github.com/colinhacks/zod/issues/6595">#6595</a>)&quot;&quot;</li>
<li>0f3f5ee3ca56c7574bf849e54f79e9a6e02562ee 4.6.5</li>
<li>59bbc03e10c636b9eb3c393dfeb552819774ec21 chore: re-pin the
integration peers to the workspace zod after the 4.6.5 bump</li>
</ul>
<h2>v4.6.4</h2>
<p>A patch on top of <a
href="https://github.com/colinhacks/zod/releases/tag/v4.6.3">4.6.3</a>.</p>
<ul>
<li><a
href="https://github.com/colinhacks/zod/commit/d6bc1e30"><code>d6bc1e30</code></a>
feat: add <code>z.currencyCode()</code> over a vendored ISO 4217 list,
refreshed weekly by CI (<a
href="https://redirect.github.com/colinhacks/zod/pull/6595">#6595</a>)</li>
<li><a
href="https://github.com/colinhacks/zod/commit/ad32d751"><code>ad32d751</code></a>
perf: <code>z.url()</code> rejects an invalid URL with
<code>URL.canParse()</code> instead of a throwing constructor, about 50x
faster; fewer allocations on the validation path (<a
href="https://redirect.github.com/colinhacks/zod/pull/6588">#6588</a>)</li>
<li><a
href="https://github.com/colinhacks/zod/commit/2bb08717"><code>2bb08717</code></a>
chore: re-pin the integration peers to the workspace zod after the 4.6.4
bump</li>
<li><a
href="https://github.com/colinhacks/zod/commit/f6e1701a"><code>f6e1701a</code></a>
chore(deps): bump next to 15.5.25 and vite to 7.3.6 (<a
href="https://redirect.github.com/colinhacks/zod/pull/6153">#6153</a>)</li>
</ul>
<h2>v4.6.3</h2>
<p>A patch on top of <a
href="https://github.com/colinhacks/zod/releases/tag/v4.6.2">4.6.2</a>.</p>
<ul>
<li><a
href="https://github.com/colinhacks/zod/commit/413cce9a"><code>413cce9a</code></a>
fix(v4): make z.properties() a check again (<a
href="https://redirect.github.com/colinhacks/zod/pull/6594">#6594</a>) —
removes the standalone <code>z.properties()</code> schema from 4.6.0;
<code>z.instanceof().properties()</code> and
<code>.check(...z.properties())</code> are unchanged</li>
<li><a
href="https://github.com/colinhacks/zod/commit/75d63ee1"><code>75d63ee1</code></a>
docs: show only the <code>.properties()</code> method form in the 4.6
post</li>
<li><a
href="https://github.com/colinhacks/zod/commit/46da9572"><code>46da9572</code></a>
docs: match the error-message examples to what the parsers emit</li>
</ul>
<h2>v4.6.2</h2>
<p>A patch on top of <a
href="https://github.com/colinhacks/zod/releases/tag/v4.6.1">4.6.1</a>.</p>
<ul>
<li><a
href="https://github.com/colinhacks/zod/commit/9446b5cc"><code>9446b5cc</code></a>
fix: preserve undefined prefault outputs and object keys (<a
href="https://redirect.github.com/colinhacks/zod/pull/6587">#6587</a>) —
closes <a
href="https://redirect.github.com/colinhacks/zod/issues/6585">#6585</a></li>
<li><a
href="https://github.com/colinhacks/zod/commit/0c483c58"><code>0c483c58</code></a>
docs: the <a href="https://zod.dev/blog/zod-4-6">Zod 4.6 announcement
post</a> (<a
href="https://redirect.github.com/colinhacks/zod/pull/6546">#6546</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/colinhacks/zod/commit/9c219833c3a3764fa1f1ea9c5ecec8254a63c0b1"><code>9c21983</code></a>
ci: stage at verbose so a failed OIDC token exchange says why</li>
<li><a
href="https://github.com/colinhacks/zod/commit/529f78dd9431b07f49647fee3aa0819b3f8b66b7"><code>529f78d</code></a>
4.6.6</li>
<li><a
href="https://github.com/colinhacks/zod/commit/a0ebf970f2d752e5f58527cd2942ff48be4f46bd"><code>a0ebf97</code></a>
fix(v4): check array, set and map size bounds before parsing elements
(<a
href="https://redirect.github.com/colinhacks/zod/issues/6652">#6652</a>)</li>
<li><a
href="https://github.com/colinhacks/zod/commit/a2ba4a67986a3e6b2b131be30084a2f460d9ea67"><code>a2ba4a6</code></a>
fix(types): declare record output and input as members, not base type
argumen...</li>
<li><a
href="https://github.com/colinhacks/zod/commit/c42596bd1c87e941d7241d10e012c5c196adcc10"><code>c42596b</code></a>
release: hold the run on the release-approval environment until the
staged ve...</li>
<li><a
href="https://github.com/colinhacks/zod/commit/bacad518c41610d654658172d325874bf0ee6290"><code>bacad51</code></a>
docs: square the Code for Japan logo to match the other bronze
tiles</li>
<li><a
href="https://github.com/colinhacks/zod/commit/8fb2faf0584828a6a027d8279db36f6f52784462"><code>8fb2faf</code></a>
docs: crop the Code for Japan logo and list it first in bronze</li>
<li><a
href="https://github.com/colinhacks/zod/commit/b2ef983e0711ff6517d6020b021071e2e5653efa"><code>b2ef983</code></a>
docs: add Code for Japan to bronze sponsors</li>
<li><a
href="https://github.com/colinhacks/zod/commit/74304ce5b5796b5edb9a88ceb8e170b6ea3e82df"><code>74304ce</code></a>
release: stage npm versions instead of publishing them (<a
href="https://redirect.github.com/colinhacks/zod/issues/6622">#6622</a>)</li>
<li><a
href="https://github.com/colinhacks/zod/commit/34a626f307871ecbb0123947559328dc52ed5e3e"><code>34a626f</code></a>
docs: tighten the z.locales tree-shaking note</li>
<li>Additional commits viewable in <a
href="https://github.com/colinhacks/zod/compare/v4.4.3...v4.6.6">compare
view</a></li>
</ul>
</details>
<br />

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicky Leach <nicky@paperclip.ing>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:41:43 -08:00
Devin FoleyandPaperclip 8f79a32898 fix(connections): keep agent access consistent through setup and permission changes (#19352)
Task-hosted tool setup kept requester-only defaults, and audience edits could leave runtime installations restricted after permission bindings were widened. Large provider catalogs could also fail before creating an access card.

Use the normal agent audience defaults for new tool setup, preserve explicit selections through verified OAuth and draft resumption, and synchronize permission bindings and runtime installations atomically. Enforce agent-edit authority and preserve existing access during additive account reuse.

Validation: 915 focused regression tests, typechecks, and builds passed. All 54 CI checks passed and Greptile Apex scored the reviewed head 5/5 with no unresolved threads. Broad local test failures, recoveries, and incomplete heartbeat coverage are documented in the PR.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
canary/v2026.1106.0-canary.1
2026-11-05 17:38:32 -08:00
DottaandClaude Opus 5.5 36643d9378 fix(native): retire finished owners whose continued turn rotated authority (#19373)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runner runs per-turn agent sessions in managed sandboxes.
A controller keeps a durable session root for each session.
> - When a per-turn owner finishes without a saved runner checkpoint,
the next turn retires that owner. Retirement first checks that the
session root holds only known entries.
> - The runner transport writes two entries that the check does not
know: `authority-epochs/` and `runtime-context.json`. A continued turn
writes `authority-epochs/` when it rotates authority from the turn
before it.
> - So the third turn of an ephemeral-lease task cannot retire the
second turn's owner. The run fails with
`native_continuation_requires_reconciliation` and the task waits for a
user message.
> - This pull request classifies every session-root entry in one
contract module, and adds a test that scans the writers so the list
cannot drift again.
> - The benefit is that follow-up turns continue without manual help,
and any future refusal names the unexpected entry.

## Linked Issues or Issue Description

Refs #18846, Refs #19061, Refs #19355

**What happened?**
On staging, the third turn of every task for one agent (gpt-5.6-sol,
native runner, ephemeral sandbox lease) failed. The error was
`runner_state_identity_mismatch:
prior_owner_terminal_state_indeterminate;
retirement_refused=unexpected_root_entry`. The first two turns
succeeded. The task then went to `blocked` with "Saved work recovered.
Send a message to continue."

**Expected behavior**
The third turn retires the finished second-turn owner and continues the
conversation.

**Steps to reproduce**
1. Assign a task to a native agent that uses an ephemeral sandbox lease.
2. Let the first turn finish. Reopen the task with a comment so a second
turn runs and finishes.
3. Reopen the task again. The third turn fails with the error above.

**Paperclip version or commit**
Staging near 9a79384958; reproduced locally on master 7c0922a7b6.

## What Changed

- Add
`server/src/services/native-runtime/native-session-root-entries.ts`. It
is the single contract for top-level session-root entries:
- Continuity state (moves with a continuity break): the existing
entries, plus `authority-epochs` and `launch-preparation.json`.
- Launch entries (stay in place, carry no authority):
`runtime-context.json`, `diagnostics`.
- Held entries (retirement still refuses): `plugin-runner-state.json`,
`cleanup-activation.json`, `cursor-cloud`.
- `readSettledRunnerdProjection` uses this contract. The refusal now
names the entries. A held entry reports as
`held_root_entry(cursor-cloud)`. An unclassified entry reports as
`unexpected_root_entry(unrecognized-state)`. Names are bounded to three,
limited to safe characters, and have identifiers replaced with `<id>`.
- Add `native-session-root-entries.test.ts`. It scans every local root
writer (the executor, the Cursor Cloud runtime, the provider-pack
authority, the controller migration, the runner transport, and the
launch-preparation store). It reads string and template literals,
same-file constants, `posix.join` and `path.join`, and the harness
persistence directory names. It fails when a writer adds an unclassified
entry, or reaches the root through a new unresolved variable.
- Add three cases to `native-recovered-owner-followup.test.ts`: a
finished owner whose root has `runtime-context.json`, an archived
authority epoch, `diagnostics`, and `launch-preparation.json` is
retired, and the epoch and launch preparation move into the archive; an
unknown entry still holds the owner and is named; a held entry is named
as held.

## Verification

- Red before the fix (same string as staging): `retires a finished owner
whose continued turn left runtime context and an archived authority
epoch` received `runner_state_identity_mismatch:
prior_owner_terminal_state_indeterminate;
retirement_refused=unexpected_root_entry`.
- Green after the fix:
- `cd server && npx vitest run
src/services/native-runtime/native-recovered-owner-followup.test.ts
src/services/native-runtime/native-session-root-entries.test.ts` (34
passed)
- `cd server && npx vitest run
src/__tests__/native-harness-backup-expiration.test.ts
src/services/native-runtime/native-session-resume.test.ts
src/services/native-runtime/native-session-executor.test.ts` (1313
passed)
- Live confirmation on staging is pending. After deploy, run a
three-turn task on an ephemeral-lease codex agent. The third turn must
succeed. Its log must show "finished without a saved runner checkpoint;
starting a new runner".

## Risks

- Retirement now accepts two more entries. `authority-epochs` only holds
epochs that the transport archived after it verified a suspended runner,
so it carries no live owner claim. `runtime-context.json` and
`diagnostics` are launch inputs and logs. Unknown entries still refuse
retirement.
- A continuity break now also moves `authority-epochs` and
`launch-preparation.json` into the archive. This is the intended owner:
these files belong to the retired authority lineage. If they stayed, a
later cold rotation could read an archived epoch of the retired lineage.
- I could not read the staging sandbox. The evidence for the exact entry
is from code: the second turn's transport writes `authority-epochs/`
during authority rotation, and every launch writes
`runtime-context.json`. The refusal now names the entry, so any other
entry will be visible in the next failure.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edit, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 19:37:01 -06:00
DottaandClaude Opus 5.5 de1143448b fix(queue): clear interaction receipts a run already delivered (#19376)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents ask people for input through thread cards: questions,
confirmations, plan approvals and connection requests
> - When a person resolves a card, Paperclip queues the response for the
agent. The task chat shows it above the composer as a read-only receipt,
for example "Answered: Questions"
> - A second wake can carry the same receipt. One example is a recovery
continuation that raced the response's own wake. The agent receives the
response from the first run, but the second wake stays deferred
> - The composer then shows a receipt that the agent already has. The
user cannot edit, discard or clear it. The same receipt also blocks the
agent's handback with `interaction_response_queued`
> - This pull request treats a receipt as pending only until a run that
carries the interaction has started
> - The benefit is that an answered card leaves the queue once the agent
has it, for every card kind

## Linked Issues or Issue Description

No public issue exists. A hands-on onboarding test on a staging cloud
workspace (Claude subscription path) found this bug.

**What happened?**
During onboarding, a queued item stayed above the task-chat composer. It
said "Answered: Questions", but the user had already answered the
questions. Later items also stayed after the interview answers and after
the user connected GitHub. The user could not trash, steer or edit them.

Evidence from the staging workspace (interactions, wake diagnostics and
run records for the onboarding task):
- The first questions card (no title, so its label is "Answered:
Questions") was answered at 17:37:48.
- Run `ba3bc7d6` (`issue.interaction.respond`) delivered that answer
from 17:37:48 to 17:38:27.
- A recovery wake (`issue.interaction_continuation_recovery`) for the
same card was requested at 17:37:48 too. It stayed deferred behind
`ba3bc7d6` and then ran again as `cacb095d` (17:38:33 to 17:39:17).
During that time the composer showed the stale "Answered: Questions"
receipt.
- The plan approval had the same pattern from 17:42:06 to 18:20 (fixed
at its source by #19332). The declined Slack connection receipt sat
behind that chain.

**Expected behavior**
A receipt leaves the composer queue as soon as a run has received it.
The agent can hand back the task after it has received the response.

**Steps to reproduce**
1. An agent asks a question card on a task.
2. A user answers it. The response wake starts a run that carries the
answer.
3. A second wake for the same card is deferred behind that run or a
later run.
4. `GET /api/issues/:id/queued-comments` returns the second wake's
receipt as an immutable entry until it runs.

## What Changed

- `server/src/services/queued-interaction-response.ts`:
`readQueuedInteractionResponse` returns no receipt when a run for the
same issue has started with this interaction in its context and is
`running` or `succeeded`. A cancelled or failed run does not count, so
that response is still owed. `hasQueuedInteractionResponse` (the
handback guard) uses the same rule.
- `server/src/routes/issues.ts`: `findQueuedCommentWake` skips a
delivered receipt, so a later undelivered receipt can still show.
- `server/src/__tests__/issue-queued-comments-routes.test.ts`: red/green
tests for every card kind and outcome: answered questions, accepted and
rejected confirmation, accepted plan (checkbox) approval, connected and
declined connection. Each runs with a `succeeded` and with a `running`
delivering run. Controls check that an undelivered receipt still shows,
that a later undelivered receipt shows instead of a delivered one, and
that the handback guard ignores a delivered receipt.
- N-02 (hover shows raw JSON): `master` already shows only the one-line
summary in the label and hover (`queuedMessageLabel`). This PR does not
change the UI.

## Verification

Red, on `master` code with only the new tests added:

```
$ cd server && npx vitest run src/__tests__/issue-queued-comments-routes.test.ts -t "receipt"
     × clears a ask_user_questions answered receipt once a succeeded run already delivered it 1309ms
     × clears a ask_user_questions answered receipt once a running run already delivered it 1041ms
     × clears a request_confirmation accepted receipt once a succeeded run already delivered it 1436ms
     × clears a request_confirmation accepted receipt once a running run already delivered it 829ms
     × clears a request_confirmation rejected receipt once a succeeded run already delivered it 666ms
     × clears a request_confirmation rejected receipt once a running run already delivered it 776ms
     × clears a request_checkbox_confirmation accepted receipt once a succeeded run already delivered it 1179ms
     × clears a request_checkbox_confirmation accepted receipt once a running run already delivered it 4227ms
     × clears a connection_intent accepted receipt once a succeeded run already delivered it 1954ms
     × clears a connection_intent accepted receipt once a running run already delivered it 1634ms
     × clears a connection_intent rejected receipt once a succeeded run already delivered it 1505ms
     × clears a connection_intent rejected receipt once a running run already delivered it 1064ms
     × shows a later undelivered receipt instead of an earlier delivered one 480ms
     × does not hold agent handback for a receipt the agent already received 1272ms
⎯⎯⎯⎯⎯⎯ Failed Tests 14 ⎯⎯⎯⎯⎯⎯⎯
AssertionError: expected [ 'Answered: Questions' ] to deeply equal []
AssertionError: expected [ 'Accepted: Approve hiring 2 agents' ] to deeply equal []
AssertionError: expected [ 'Rejected: Approve hiring 2 agents' ] to deeply equal []
AssertionError: expected [ Array(1) ] to deeply equal []
AssertionError: expected [ 'Connected: GitHub connection' ] to deeply equal []
AssertionError: expected [ 'Declined: Slack connection' ] to deeply equal []
AssertionError: expected [ Array(1) ] to deeply equal [ 'Declined: Slack connection' ]
AssertionError: expected true to be false // Object.is equality
      Tests  14 failed | 11 passed | 85 skipped (110)
```

Green, with the fix:

```
$ cd server && npx vitest run src/__tests__/issue-queued-comments-routes.test.ts -t "receipt"
      Tests  25 passed | 85 skipped (110)
```

Neighbouring suites, with the fix:
- `issue-queued-comments-routes.test.ts` (full file, on current
`master`): 110 passed.
- `question-response-delivery`, `queued-comment-consumption`,
`issue-thread-interaction-routes`, `satisfied-connection-intents`,
`issue-comment-reopen-routes`, `issue-execution-policy-routes` together
with the queued-comments file: 494 tests passed. One run hit an
embedded-Postgres `afterAll` cleanup timeout under 7-file parallel load.
The same file passed alone twice.
- `heartbeat-process-recovery.test.ts -t "plan approval|accepted
interaction|answered question|answered interaction|onboarding first
task"`: 14 passed.
- `pnpm --filter @paperclipai/server typecheck`: passes.

## Risks

- Low risk. The change only hides a receipt that a started run already
carries. An undelivered receipt keeps its current behavior.
- The redundant wake itself is not cancelled here. It still runs later
as a normal continuation, but it no longer holds the composer queue or
blocks handback. #19332 removes the main source of these duplicate
wakes.
- Each receipt read adds one indexed-by-company lookup on
`heartbeat_runs` for the issue.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, file edits, browser inspection of the staging
workspace API).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 19:28:45 -06:00
Devin FoleyandClaude Opus 5.5 434d224c3a fix(ui): retry stale chunk reloads while a deploy settles (#19361)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The board UI loads each page as a lazy chunk. After a deploy, an
open tab can ask for a chunk from its old build.
> - #17349 added a recovery: the app reloads the page one time when a
chunk fails to load. A loop guard permits only one reload in five
minutes.
> - Staging error monitoring still reports `Failed to fetch dynamically
imported module` from builds that include #17349. The events come 1–10
seconds after page load, and 1–10 minutes after page load.
> - During a deploy, the old server and the new server can both answer
for some seconds. The reloaded page can then miss a chunk again. Also, a
second deploy can occur inside the five-minute window. In both cases the
guard stops the recovery, and the user sees the error page. One more
reload a few seconds later would recover.
> - #18518 keeps old chunks available after a deploy, but only on
deployments that enable shared UI assets. Other deployments still depend
on the reload recovery.
> - This pull request permits up to three reloads in the guard window.
The reloads wait 0 s, 2 s and 5 s, so that the deploy can settle. After
the last reload, the error page shows and reports as before.
> - The benefit is that a stale tab recovers through a normal deploy
without an error page and without error-monitoring noise. A chunk that
stays broken still shows the error after a small, fixed number of
reloads.

## Linked Issues or Issue Description

Refs #17349
Refs #18518

**What happened?**
A browser tab stayed open during a staging deploy. A lazy page chunk
failed to load with `TypeError: Failed to fetch dynamically imported
module`. The route error boundary showed "This page hit an error" and
reported the error. The release included the one-time reload from
#17349. The reload did not occur because the loop guard had already used
its single reload in the last five minutes.

**Expected behavior**
The app reloads again after a short wait while the deploy settles, and
the page loads. The error page shows only when the chunk stays broken.

**Steps to reproduce**
1. Open a board page on a deployment without shared UI assets.
2. Deploy a new build. While the old and new servers both answer, go to
a page that was not loaded yet.
3. The chunk fails, and the page reloads one time.
4. The reloaded page gets its HTML from one server and a chunk request
goes to the other server. The chunk fails again.
5. The guard stops the recovery, and the route error page shows.

**Paperclip version or commit**
`master` at 5ac05871b5

## What Changed

- `ui/src/lib/chunk-load-recovery.ts`: The recovery permits up to three
reloads in the five-minute guard window. The reloads wait 0 s, 2 s and 5
s (`RELOAD_DELAYS_MS`). After the third reload, the recovery returns
`false`, and the boundary shows and reports the error as before.
- Storage stays compatible with older builds.
`paperclip:chunk-load-reload-at` still holds only the bare start time of
the window, so an older build keeps its one-reload guard during a
deploy. The retry count is in a separate key,
`paperclip:chunk-load-reload-attempts` (`<window start>:<attempts>`).
The count applies only to the matching window. A window that an older
build started counts as one reload.
- A delayed retry keeps its timer. The new
`cancelPendingChunkLoadReload()` stops a retry that has not started.
`RouteErrorBoundary` calls it when the user moves to another route, and
when the boundary unmounts. A working page is therefore not reloaded
later.
- The abandonment timer starts after the delay, so a delayed reload gets
the full 10 s before the boundary reports.
- A reload that does not leave the document (for example, the user
cancels it) is not tried again from the same document. This keeps the
behavior from #17349.
- Tests: unit tests for delayed retries, the retry limit, compatibility
with older builds, a stale retry count, and cancellation. Boundary tests
start from a delayed attempt and cancel it on navigation. The e2e spec
for a lazy CSS asset that stays broken now expects 4 page loads before
the error page.

## Verification

- `cd ui && npx vitest run src/lib/chunk-load-recovery.test.ts
src/components/RouteErrorBoundary.test.tsx
src/components/AppErrorBoundary.test.tsx`: 3 files, 32 tests pass.
- Red before the fix: with the new tests, the first PR head fails 5
tests (older-build key, cancellation, cancel on navigation). The
original `master` code fails the retry tests. With this head, all pass.
- `tsc --noEmit -p ui/tsconfig.json` shows no errors in the changed
files.
- `tests/e2e/chunk-load-recovery.spec.ts` runs in CI.
- Manual check: on a deployment without shared UI assets, open a page,
deploy, and go to a page that was not loaded. The page reloads, and it
loads after the deploy settles. If you move to another page during the
wait, the page does not reload.

## Risks

- Low risk. The change affects only the chunk-load recovery path.
- A chunk that stays broken now causes up to three reloads (about 7 s of
waits in total) before the error page shows. Before, it caused one
reload. The limit stays fixed in each five-minute window, so a reload
loop cannot occur, also when old and new builds alternate.
- A retry that the `vite:preloadError` listener starts is not tied to a
route, so navigation does not cancel it. This matches the behavior
before this change, where that reload occurred at once.
- No server, data, or contract changes.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, used through Claude
Code with tool use (shell, file edits and GitHub connector). Standard
context window, with reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:21:08 -08:00
DottaandClaude Opus 5.5 9ceaec6f97 fix(issues): stop showing resource waits as failed attempts (#19369)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The task page shows one interruption banner. The server builds it
from the task's latest run.
> - An auto-retry that waits for a busy workspace, AI connection or
runner update is not a failure. The dispatcher re-checks the resource,
so the run can stay `scheduled_retry` after its retry time.
> - The projection did not know about these waits. After the retry time
passed, it fell through to "Couldn't finish this attempt" with a Retry
button.
> - Retry cannot free the resource. The banner cleared for the 10-second
grace and then came back. Users had no way to fix it.
> - This pull request projects a scheduled resource wait as `working`,
with no Retry.
> - The benefit is that the failed-attempt banner shows only for real
failures.

## Linked Issues or Issue Description

**What happened?**
During onboarding on Paperclip Cloud staging (Claude subscription and
Codex subscription paths), the onboarding task showed "Couldn't finish
this attempt. Your message is saved." with a Retry button. The banner
did not name an attempt. Retry cleared it briefly, then it came back.

**Steps to reproduce**
1. Start a task run while another run holds the same workspace.
2. The run is cancelled with `workspace_busy`, and a `scheduled_retry`
run is created with `scheduledRetryReason: "workspace_busy"`.
3. Keep the workspace busy after `scheduledRetryAt`. On the staging
company, one retry stayed scheduled for 16 minutes past its retry time.
4. Open the task. The failed-attempt banner shows. Click Retry. The
banner comes back.

**Expected behavior**
No failed-attempt banner while the system waits for a shared resource.
The run status already says "Waiting for workspace"
(`execution-projection.ts`).

## What Changed

- `server/src/services/task-interruption-presentation.ts`: a
`scheduled_retry` run whose `scheduledRetryReason` is `workspace_busy`,
`ai_connection_busy`, `ai_connection_pool_wait` or
`runner_revision_wait` (and that has no system recovery) projects as
`state: "working"`, `canRetry: false`. User gates and failed Retry
preparations stay authoritative because they are checked first.
- The batched query now selects `scheduledRetryReason`.
- Tests: a Postgres integration test (`task-interruption-query.test.ts`)
that seeds the staging run chain for each reason, and unit tests for the
guard cases (failure-lane retries, paused agent, failed Retry
preparation).

## Verification

Red on `master` (fix not applied), `cd server && npx vitest run
src/__tests__/task-interruption-query.test.ts -t "overdue"`:

```
     × does not present an overdue workspace_busy wait as a failed attempt 213ms
     × does not present an overdue ai_connection_busy wait as a failed attempt 25ms
     × does not present an overdue ai_connection_pool_wait wait as a failed attempt 13ms
     × does not present an overdue runner_revision_wait wait as a failed attempt 11ms
- Expected
+ Received
-   "canRetry": false,
+   "canRetry": true,
-   "state": "working",
+   "state": "failed",
      Tests  4 failed | 11 skipped (15)
```

Green with the fix, same tests plus the new unit tests:

```
 ✓ task-interruption-query.test.ts > does not present an overdue workspace_busy wait as a failed attempt
 ✓ task-interruption-query.test.ts > does not present an overdue ai_connection_busy wait as a failed attempt
 ✓ task-interruption-query.test.ts > does not present an overdue ai_connection_pool_wait wait as a failed attempt
 ✓ task-interruption-query.test.ts > does not present an overdue runner_revision_wait wait as a failed attempt
 ✓ task-interruption-presentation.test.ts > scheduled resource wait presentation > keeps an overdue workspace wait out of the failed-attempt UI
 ✓ task-interruption-presentation.test.ts > scheduled resource wait presentation > keeps failure retries, user gates and failed Retry preparation authoritative
```

Neighbouring suites: `task-interruption-presentation`,
`task-interruption-query`, `workspace-recovery`, `agent-project-access`,
`heartbeat-dependency-scheduling`: 5 files, 458 tests passed.

Server `tsc --noEmit` reports no errors in changed files. The only
errors locally come from the unbuilt `@paperclipai/paperclip-runner`
vendor package, which this change does not touch. No UI change, so `pnpm
check:token-gates` is not applicable.

Evidence: the staging onboarding task's run history shows
`workspace_busy` cancellations at 17:46, 17:52 and 18:10, each followed
by a `scheduled_retry`. One had `scheduledRetryAt` 17:53:57 and started
at 18:10:20. The next run succeeded at 18:20.

## Risks

- Low risk. Presentation only; dispatch and admission do not change.
- If a resource wait never clears, the task shows no failed-attempt
banner. The run status still shows "Waiting for workspace", and the
existing retry-attempt limits still end the wait.
- Not in scope: genuine failures without a provider or workspace
explanation still show the generic text, and an overdue failure-lane
retry still shows as failed.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, browser for read-only staging evidence).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 19:08:27 -06:00
4550253dc9 feat(ui): rename Needs you to Inbox and move Mine into Tasks (#19363)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Coordinator gives people navigation for conversations, tasks, and
decisions.
> - The decision entry is called Needs you, while a separate Inbox
remains under More.
> - Coordinator's personal task filter does not retain the full behavior
of the original Mine view.
> - This pull request names the decision entry Inbox and hosts the
original personal work views inside Tasks.
> - People keep their triage tools and saved choices without moving
between two task destinations.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Coordinator navigation and the personal Mine view in Tasks.

**Subsystem affected**

UI navigation, task view routing, task collection preferences, and
browser coverage.

**Current behavior**

Needs you opens the decision feed. More contains another Inbox
destination. The default Coordinator task filter matches task creators
and assignees. It does not use the full personal participation rules,
archive behavior, shortcuts, grouping, or saved columns of the original
Mine view. The organization task table also resets its column choices on
reload.

**Proposed behavior**

Inbox opens the same decision feed with the inbox icon. Tasks defaults
to Mine and remembers a person's selected view per company. Mine uses
the existing personal Inbox implementation and preferences. It retains
read/unread title weights, keyboard navigation, read/unread shortcuts,
archive and undo, mobile swipe-to-archive, grouping, and saved columns.
Old Inbox links open the matching task view. The organization task table
saves its columns and order per person and company.

**Reason and benefit**

People get Chat, Tasks, and Inbox as distinct navigation entries.
Personal work remains a view within Tasks without losing the existing
triage behavior.

**Breaking changes**

Coordinator no longer offers the previous Inbox destination under More.
Old Inbox URLs redirect into Tasks and retain query parameters and
fragments. The Needs you label becomes Inbox; its decision feed, count,
history, and resolution paths stay in place. Classic retains its
separate optional combined-list setting.

**Additional context**

Refs: #19360, the workspace-defaults PR now integrated from master.
Refs: #10620, related Inbox attention ordering. Duplicate searches found
no open PR for this navigation and Mine integration. Roadmap review
found no separate new execution feature in this scope.

## What Changed

- Rename Coordinator's desktop and mobile decision entry and page to
Inbox. Use the inbox icon. Remove the old Inbox item under More. Clarify
that the combined-list experiment applies only to Classic.
- Add one Coordinator task-view router. Host Mine, Unread, Recent,
Blocked, and Everything through the existing Inbox owner. Keep
organization views in the task table and board. Normalize legacy
filtered URLs before mounting a collection so clearing the last filter
preserves the current layout and local filters.
- Reuse the existing Mine API, participation, read state, archive cache,
undo, keyboard, mobile gesture, grouping, and preference paths. Existing
personal column choices carry over.
- Save organization task-table columns and order per person and company.
Keep controls usable if browser storage fails.
- Update navigation tests, routing and persistence regressions,
exact-run retry browser coverage for each workspace variant, product
documentation, and the affected Storybook notes. Keep existing story
IDs.

## Verification

- Preference, routing, navigation, task table, and inherited Inbox
regression tests pass in focused runs. After integrating master and
addressing the review finding, all 187 focused tests, UI typecheck, and
token gates pass.
- Full build and repository typecheck pass after integrating master. The
full UI suite previously passed: 894 files, 11,169 tests passed, and one
skipped. All 14 desktop/mobile triage and exact-run retry browser cases,
including a fresh production UI build, pass after integrating master.
- The full repository test suite was not run locally for this UI-only
change; full PR CI is running.
- Browser checks use a fresh isolated local instance and a deterministic
process agent for real run-attributed unread activity. They exercise
legacy URLs, title weights, saved grouping and columns, read/unread
shortcuts, archive/undo, and a real mobile touch gesture.
- Earlier browser fixtures were corrected: own activity is read, agent
comments need valid run attribution, and undo uses `u` after the archive
refresh completes. The Inbox content heading is scoped separately from
the layout heading.
- The final settings wording correction passes all 63 settings tests.
- The [next CI browser
run](https://github.com/paperclipai/paperclip/actions/runs/42137377960/job/127629100535)
caught an old URL assertion for denied run retries. The test now pins
each workspace variant and covers Coordinator Tasks explicitly. The old
expectation fails deterministically for Coordinator; all 12 retry cases
pass with the corrected destination, preserving exact run identity and
truthful queued, deferred, and denied feedback.
- Token gates, map checks, and Storybook catalog checks pass. All 30
map/catalog script tests pass. Storybook play tests were not run.
- Greptile reviewed [the final
commit](https://github.com/paperclipai/paperclip/commit/5f2f3789b83af259f694f379afb769465688fb72)
at **5/5** with no new actionable findings. Its sort-only link finding
is fixed and resolved: three cases fail before the fix and pass after
it. `sortField` and `sortDir` keep organization links on the
organization table. [Current-head
CI](https://github.com/paperclipai/paperclip/actions/runs/42139668774)
is running.
- [CI exposed a routing
race](https://github.com/paperclipai/paperclip/actions/runs/42135293421/job/127621981587)
when removing the last label from a legacy Everything link. Four
deterministic regressions failed before the fix and now pass:
collections only mount after URL normalization. The existing end-to-end
component cases confirm local filters, board layout, fragments, and
history survive the change.
- The [initial server shard 4
run](https://github.com/paperclipai/paperclip/actions/runs/42131163087/job/127608751620)
failed an unchanged monitor-drain idleness assertion: two active
operations remained instead of zero. The complete scheduling suite
passes locally (54 tests). One isolated CI retry passed, including its
dependent verification gate. The original timing failure's root cause is
not established.

## Risks

- Mine includes the existing tasks, approvals, failed runs, and join
requests, preserving the original personal participation and attention
rules.
- Shared URLs with organization filters or table sort parameters still
open the organization task collection. Task back-links retain their
original fragments.
- Saved preferences are browser-local. Mine keeps its existing storage
keys; the organization table and selected Coordinator view use
person/company scopes.
- No database or server API contract changes are included. Existing
company access, personal participation, and decision permissions retain
their owners.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository tools, code
execution, and browser automation. The exact serving revision and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 17:07:24 -08:00
DottaandPaperclip 46414fcfa4 test(evals): pin timeline and governed Company approval at native cap two (#19381)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The chat catalog uses a fixed source and explicit execution
conditions.
> - A correct timeline response failed because its completed edit and
attached command result were not recognized.
> - Company credential setup can require a real human approval after the
secure save.
> - The controller must declare that optional approval and its run
bounds before it admits the source.
> - This pull request pins the reviewed source and caps native catalog
overlap at two.
> - The next run can test these conditions without treating local
controls as native acceptance.

## Linked Issues or Issue Description

Refs: #19357, #19086, #18971, #19093.

This follows the prior fixed-source catalog pin. It changes only
controller metadata, admission, tests, and load policy. The complete
catalog and separate core feature PRs remain unmerged.

## What Changed

- Pin immutable source `fe547be84826b4611d3ba9ee1a7d7aa0bd0a2843`, tree
`a23723ea5244e6ce8cee11e9c4b476acb6978fcd`.
- Advance only timeline v10 to v11 and Company v11 to v12. Add the
declared Company proposal helper to its source closure.
- Declare Company minimum two and maximum three native runs. Retain two
base human inputs and permit at most one independently verified UI
approval. Retain its original 480000ms attempt.
- Carry the literal Company-only approval marker from the descriptor
into the manifest. Reject foreign markers, changed bounds, and a
mismatch with static source task metadata.
- Cap native catalog execution at two workers. Keep the external lane
serial and retain zero retries.
- Keep all244 requests, contexts, expected outcomes, model, clocks,
credentials and other case bounds unchanged.

## Verification

- Source:421 focused controls across seven suites pass. Full runner E2E
TypeScript check passes.
- Explicit model-free approval UI canary passes one test with three
local scenarios. It checks accepted approval, wrong human, and failed
response. The first sandbox attempt stopped at Chrome launch; the
allowed execution passed.
- Controller:40 manifest, adapter and reconciliation controls pass.
- SDK506 inputs remain byte-identical to the accepted registry
`efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`.
- Launcher and trusted private evidence wire remain byte-identical.
Source contract/flow accounting changes are explicit.
- Source descriptor SHA256:
`50e90d8c02ad57f444e27b522efe694db9734d47bdbc3a058475af77bac8b2ca`.
- Controller manifest SHA256:
`0447315601ed21ba15d6ea0ecd7fb89bc79ad6262db2594cd12ac774aa8d6c21`.
- Review the two changed rows. The next planned experiment selects
timeline and Company once at cap2 before a full ten-case run. No native
run on this source has passed yet.

- Exact controller head `d49dfc59ab7fddfe1a1de7e81bdb9c4e13400f30`:
normal CI has56 successful checks and three configured skips. Protected
`ci / verify` and `ci / e2e` both pass. Greptile5/5 and zero open
threads are verified for this head.

## Risks

- Original timeline5056 and Company5057 remain FAILED. Timeline5056
supplied the correct account; the local revised check passes its
diagnostic only. Company5057 saved the secret but left the binding
approval pending.
- Company native3 completion within480s is unproved. This change does
not raise its clock or claim to fix latency.
- Two-worker execution is a changed load condition. It does not prove a
concurrency cause, reliability, or eight-worker qualification.
- Secret journeys require independent semantic safety review before pass
credit. The documented closing-comma warning and arbitrary
metadata-prefill classifier limitations remain follow-ups in the
functional secret PR; a native failure is not waived.
- Candidate core PRs remain unmerged. The full-catalog196/244 admission
gate remains fixed. Local controls do not qualify all244 journeys.
- Local checks use Node26.4.0. They do not replace actual AWS runtime
qualification.

- The separate Paperclip Review app remains queued while its assigned
task waits for maintenance. It has no completed assessment. GitHub
reports `UNSTABLE` and `MERGEABLE`; this is not an all-checks PASS
claim. No review retry or staging mutation was performed.

## Model Used

OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, repository
tools and code execution. The host does not expose an exact
context-window size. No model or paid provider was called for local
controls.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 19:06:42 -06:00
DottaandClaude Opus 5.5 7440cfc9b1 fix(issues): accept queued Send while the previous run has not started (#19370)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A task chat queues a message while the agent is busy. The
queued-message bar shows a Send button when no turn is running.
> - Send calls `POST /issues/:id/queued-comments/interrupt` with no
target run.
> - The issue can hold an execution run that has not started yet
(`queued`) or that waits for a retry (`scheduled_retry`). The GET
response then shows a deferred queue with no target run, so the UI shows
Send.
> - The interrupt route refused that click with 409. The UI then showed
"Couldn't send. Message is still queued."
> - This pull request saves the click as interrupt intent in that state.
The queue is delivered when the previous run stops.
> - The benefit is that the button the UI offers now works, and the
message goes next.

## Linked Issues or Issue Description

**What happened?**
Cloud onboarding test, Codex subscription path (N-06): the tester
clicked Send in the onboarding task. The UI showed "Couldn't send.
Message is still queued."

**Expected behavior**
Send is accepted. The message stays in the queue and is delivered next,
when the previous run stops.

**Steps to reproduce**
1. Have an issue whose execution run is `queued` (not started) or
`scheduled_retry`.
2. Post a message. It is deferred behind that run.
3. Click Send on the queued message.
4. The route returns 409: "This queue does not support interruption"
(paperclip_runner) or "The previous run has not stopped" (legacy
adapters).

**Paperclip version or commit**
`master` at b68f69a801.

Root cause: `GET /queued-comments` resolves the target run from running
runs only, so the UI offers Send. `POST /queued-comments/interrupt`
locks `issue.executionRunId` at any status. It accepted only a terminal
run or no run. A queued or retry-waiting run fell between those two
rules.

Relation to other fixes: #19331 (declined connection receipt) changed
which wake the queue projects. It does not touch this check, and current
`master` still fails. The sibling N-07 problem (steering a queued
message on Codex) uses the `/steer` route. This PR does not change that
route. I found no shared cause.

Evidence limits: the tester's notes are the only evidence. The staging
company that I could read uses the Claude subscription path, so I could
not see the Codex run's status at the time of the click. The reproduced
state is the only server path that gives "Couldn't send" for a Send
click with no running turn. A slow-starting or retrying Codex run gives
that state.

## What Changed

- `server/src/routes/issues.ts`: the interrupt route no longer refuses a
Send when the issue's run is `queued` or `scheduled_retry`. For
paperclip_runner queues, a run that is not running is handled like a
stopped run. The legacy "previous run has not stopped" 409 is removed.
The route saves `queuedCommentInterrupt` intent, does not cancel the
not-yet-started run, and returns the queue in the `deferred` state. The
UI already shows "Messages are still queued. They will send when the
task is ready." for that response.
- `server/src/__tests__/issue-queued-comments-routes.test.ts`: new route
test for native and legacy queues, with `queued` and `scheduled_retry`
runs. It checks that GET offers Send, that the POST is accepted without
a run cancel, that the intent is saved, and that the queue is delivered
to one successor run after the previous run stops.

## Verification

Red, on `master` (new test, before the fix):

```
pnpm vitest run server/src/__tests__/issue-queued-comments-routes.test.ts -t "accepts Send while"
     × accepts Send while the previous native run is queued and delivers it once that run stops
     × accepts Send while the previous native run is scheduled_retry and delivers it once that run stops
     × accepts Send while the previous legacy run is queued and delivers it once that run stops
     × accepts Send while the previous legacy run is scheduled_retry and delivers it once that run stops
AssertionError: {"error":"This queue does not support interruption"}: expected 409 to be 200
AssertionError: {"error":"The previous run has not stopped"}: expected 409 to be 200
      Tests  4 failed | 90 skipped (94)
```

Green, after the fix (same command):

```
 Test Files  1 passed (1)
      Tests  4 passed | 90 skipped (94)
```

Neighbouring suites:

- `server/src/__tests__/issue-queued-comments-routes.test.ts` (complete
file): 94/94 passed.
- `server/src/__tests__/agent-conversations.test.ts`,
`server/src/services/issue-queued-comment-queue.test.ts`,
`ui/src/components/task-chat/TaskChatQueuedMessages.test.tsx`: passed.
- When I ran the two embedded-Postgres route files in one Vitest
process, every test passed. Then the queued-comment file's `afterAll`
database cleanup timed out after 30 s, which is a local resource limit.
The file passes when it runs alone.
- `pnpm --filter @paperclipai/server typecheck`: passed.
- I made no UI changes, so I did not run `pnpm check:token-gates`.

## Risks

- Low risk. Send was refused only for `queued` and `scheduled_retry`
runs. Now that click is saved and the run is not cancelled. A running
turn is unchanged: paperclip_runner still steers it, and legacy still
interrupts it.
- When a run waits for a retry, a saved Send waits for that retry to
finish. The route does not skip the retry. This matches how the queue
already works for messages without a click.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits, browser for read-only staging evidence).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 18:59:16 -06:00
Devin FoleyandClaude Opus 5.5 33269e1fec fix(observability): report directory-merge lock holder diagnostics (#17755)
A directory-merge lock timeout (ERR_WORKSPACE_RESTORE_LOCK_TIMEOUT) already
carries allowlisted holder evidence, but server Sentry capture dropped it.

- Add directoryMergeLockTimeoutDiagnosticsFor in adapter-utils: closed
  labels, booleans and bounded durations; no owner PID or paths.
- server captureException: lock timeouts get an event-local
  directory_merge_lock context and error_code=workspace_restore_lock_timeout
  tag; default grouping and message unchanged.
- run-failure-diagnostics restoreLock* fields use the shared reader
  (same allowlist and order).
- Document the record in doc/workspace-restore-locks.md.

Diagnostics only; does not change lock acquisition, timeouts or retries.

Refs PAPERCLIP-BACKEND-4Q

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:48:46 -08:00
Nicky LeachandClaude Opus 5.5 3a8f95d928 fix(ci): let verified Dependabot lockfiles merge through the reporting-lock gate (#19356)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Only trusted environments may change `pnpm-lock.yaml`: the lockfile
refresh workflow and Dependabot. Contributors never commit it.
> - Paid evaluation jobs also install a lock only when the
reporting-lock proof (`tests/runner-e2e/report-lock.json` and `.patch`)
admits it.
> - #19283 added a CI test that checks the committed lock against that
proof. Only the refresh workflow writes the proof, so every Dependabot
lockfile pull request now fails CI. Examples are #17364, #17172, and
#17911.
> - A merged Dependabot lockfile also breaks the refresh workflow.
`prepare-report-lock.mjs --refresh` refuses to renew the proof for a
lock that the proof does not already admit.
> - This pull request lets a verified Dependabot lockfile pass, and it
keeps both lockfile rules strict. It also closes a gap in the
contributor rule.
> - The benefit is that Dependabot security updates can merge again, and
only trusted environments write lock content.

## Linked Issues or Issue Description

Refs #19283, #17364, #17172, #17911, #19303.

**What happened?**
`server/scripts/reporting-lock-policy.test.mjs` fails on each Dependabot
pull request that changes `pnpm-lock.yaml`. The error is "Reporting lock
changed; review the dependency correction before install". A Dependabot
pull request cannot change the proof files, so it can never pass. After
a merge, the lockfile refresh also stops at "Committed report lock is
not reviewed; refusing to refresh its proof".

**Expected behavior**
- Contributors cannot change `pnpm-lock.yaml` or the reporting-lock
proof.
- Dependabot can change `pnpm-lock.yaml`.
- The lockfile refresh workflow can change `pnpm-lock.yaml` and the
proof.
- After a Dependabot lockfile merges, the refresh workflow admits it and
renews the proof.

**Steps to reproduce**
1. Open a Dependabot pull request that changes `pnpm-lock.yaml`, such as
#17364.
2. Let CI run. The `server (9/12)` shard fails on the reporting-lock
policy test.

**Security gap found during this work**
The old rule skipped the lockfile check when `github.head_ref` was
`chore/refresh-lockfile`. A contributor could open a pull request from a
fork branch with that name and commit a lockfile change. The old rule
also exempted any pull request that Dependabot opened, including commits
that other people pushed to its branch.

## What Changed

- Add `.github/scripts/lockfile-trust.mjs`. It decides which writers may
change lock content:
- The refresh workflow: the head is `chore/refresh-lockfile` in this
repository, and `github-actions[bot]` opened the pull request.
- Dependabot: Dependabot opened the pull request from this repository,
and the head `pnpm-lock.yaml` matches a Dependabot commit. That commit
must have `dependabot[bot]` as its author, GitHub (`web-flow`) as its
committer, and a verified signature. The script reads these fields from
the GitHub API.
- The reporting-lock proof files count as lock content. Only the refresh
workflow may change them.
- `pr-trusted.yml` policy job: replace the inline `head_ref` and author
exemption with `lockfile-trust.mjs pr`. The job runs the script copy
from the base commit, so a pull request cannot change the check that
judges it. If the base predates the script, the job uses the earlier
rule.
- `refresh-lockfile.yml`: add `lockfile-trust.mjs merged`. It checks
that the last lockfile change on `master` is a GitHub-signed merge of a
Dependabot pull request, and that the lockfile at the pull request head
matches a verified Dependabot commit. If both checks pass, it gives the
lockfile SHA-256 to `prepare-report-lock.mjs --refresh
--trusted-lock-sha256`.
- `refresh-lockfile.yml` PR step: check `pnpm-lock.yaml` and both proof
files before it exits as unchanged. A refresh that renews only the proof
now opens a pull request.
- `prepare-report-lock.mjs` and its `.d.mts` declaration:
`refreshReportLock` accepts a trusted lock hash. If the trusted lock
already resolves, the proof admits it exactly with an empty patch. If
not, the proof records the resolution as the reviewed correction, as
before.
- `reporting-lock-policy.test.mjs`: skip on Dependabot pull requests.
The policy job verifies those lockfiles. The general test job gets
`PAPERCLIP_CI_PR_AUTHOR` for this purpose. This keeps the test, so it
still merges cleanly with #19303.
- `check-pr-lockfile.mjs` (advisory gate): allow Dependabot lockfile
changes, and flag changes to the proof files.
- `CODEOWNERS`: add the two proof files.
- `doc/RELEASE-AUTOMATION-SETUP.md`: describe the writers, the
`@dependabot recreate` path, and the proof renewal after merge.

## Verification

- `node --test '.github/scripts/tests/*.test.mjs'`: 707 passed. This
includes 17 new tests in `lockfile-trust.test.mjs` and 3 new tests in
`check-pr-lockfile.test.mjs`. The new tests run the `pr` command against
real git histories with a stub `gh`.
- `tests/runner-e2e/prepare-report-lock.test.ts`: 10 passed, including 3
new trusted-lock refresh tests. I ran it with a minimal Vitest config,
because the runner-e2e config builds a provider bundle first.
- `server/scripts/reporting-lock-policy.test.mjs`: passes normally, and
skips with `PAPERCLIP_CI_PR_AUTHOR=dependabot[bot]`.
- Live check of the `pr` mode against #17364 with the real GitHub API:
- With the real author, it reports "pnpm-lock.yaml matches verified
Dependabot commit 83c328c15c…" and `writer=dependabot`.
- With a contributor as the author and `chore/refresh-lockfile` as the
head ref, it fails.
- Live check of the `merged` mode at the real merge commits of #16018,
#16020, and #15361. All three report a trusted lock hash. For #16018 and
#15361, `master` changed after Dependabot's commit, so GitHub's merge
produced different lockfile bytes. An exact-bytes rule would refuse
them. That is why the rule trusts GitHub-signed merges of a verified
Dependabot head.
- Greptile reviewed `513ca2def8` at 4/5 with one P1: a proof-only
refresh was never committed, because the PR step checked only
`pnpm-lock.yaml`. `edbeb1ea0d` fixes it. A new test in
`lockfile-refresh-merge.test.mjs` runs the step's change check on a
proof-only change. The test fails without the fix.
- CI `Typecheck + Release Registry` failed at `edbeb1ea0d`:
`prepare-report-lock.d.mts` did not declare the new options argument
(TS2554 twice in `prepare-report-lock.test.ts`). `9139aaad85` updates
the declaration. A local `tsc -p tests/runner-e2e/tsconfig.json` shows
the 2 errors with the old declaration and none with the new one.
- CI at `9139aaad85` failed in six general test shards. None of them run
code that this pull request changes, and PR CI loads `pr-trusted.yml`
from `master`. The failures:
- `heartbeat-plugin-environment.test.ts`: "expected 'running' to be
'succeeded'"
  - `status-card-evaluation-work.test.ts`: a deadline comparison
  - `isolated-repository-rehome.test.ts`: timed out after 15 seconds
- `chat-channels.integration.test.ts`: GitHub webhook recovery
scheduling
- `company-import-export-e2e.test.ts`: child server start timeout, which
also fails on unrelated branches
- `workspaces-b`: every test passed, but an `EPIPE` error came after
`acpx-engine/execute.test.ts`
  I started a rerun of the failed jobs.
- Superagent found a P1 at `9139aaad85`: a verified signature proves who
committed, not who authored. A collaborator could sign a commit with
their own key and set Dependabot as the author. `4fe180d6c1` also
requires GitHub (`web-flow`) as the committer. New tests cover the
spoofed-author case. #17364, #16018, #16020 and #15361 still verify
against the GitHub API.
- `git merge-file` merges this test change with #19303 without
conflicts.
- Not run: the full repository test suite and typecheck. These changes
touch only CI scripts, workflows, one helper, and tests. GitHub CI
covers full regression.

## Risks

- `pr-trusted.yml` loads from `master`, so the new policy takes effect
only after merge. This pull request's own CI uses the earlier rule.
- A Dependabot pull request that is behind `master` now fails if someone
merges `master` into it and the lockfile changes. Use `@dependabot
recreate`. The policy job error says this.
- Paid evaluation jobs refuse a merged Dependabot lockfile until the
refresh pull request merges. The refresh merges without review when it
adds no new package versions. If the release-age gate holds it, the wait
lasts until a maintainer merges it or the daily refresh runs. During
that time, the reporting-lock policy test also fails on other pull
requests, which is the current behavior for any unrenewed lock.
- The rule trusts GitHub as the signer of Dependabot commits and of
merge commits (`web-flow`). Superagent uses the same trust root.
- People with write access can still push to `chore/refresh-lockfile`.
This was already true. The bot merges only its own head commit
(`--match-head-commit`), and code-owner review covers anything else.
- A future manual proof renewal, such as #19283, now needs the refresh
workflow or an admin merge. A contributor pull request that changes the
proof files fails.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution, in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused tests; see
Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:42:26 -08:00
Nicky LeachandClaude Opus 5.5 e6c74982fe fix(workspaces): report why a workspace provider operation failed (#19362)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A project can store its task workspaces with a workspace storage
provider plugin. Core calls the plugin's `workspaceOperation` method to
prepare, inspect, finalize and remove a workspace.
> - `callWorkspaceProvider` catches every rejection of that call and
throws the same conflict: "Workspace storage operation is incomplete;
retry the same workspace".
> - A provider error, an RPC timeout, a crashed worker and a worker that
never got the call all look the same in the run, the log and the API.
The message tells the user to retry, also when the failure is a fixed
configuration problem that a retry cannot fix.
> - In one diagnosis, three different configuration failures in a row
showed this same message. Finding each cause took trace span counts and
database reads.
> - This pull request lets a provider send a reason from a closed set,
and makes Core tell transport failures apart from provider failures.
> - The benefit is that a failed run says what failed and whether a
retry can help. Provider text and values still never cross into Core.

## Linked Issues or Issue Description

**What happened?**
A workspace provider's `workspaceOperation` failed. The run failed with
"Workspace storage operation is incomplete; retry the same workspace",
`errorCode: setup_failed`, and no reason in the run record or the
`heartbeat execution setup failed` log. Retrying did not help, because
the cause was a provider configuration problem.

**Expected behavior**
Core tells these cases apart: the worker did not receive the call, the
call timed out, the worker stopped during the call, or the provider
reported a failure. For a provider failure, Core shows a bounded reason.
Core tells the user to retry only when a retry can help.

**Steps to reproduce**
1. Install a workspace storage provider plugin whose
`onWorkspaceOperation` throws during `prepare` (for example, because of
missing provider settings).
2. Run an agent task in a project that uses that provider.
3. See the failed run: the error is the generic retry message, and the
run record and logs do not show the cause.

**Paperclip version / commit**
`master` at 5ac05871b5.

**Deployment mode**
Any deployment with a workspace storage provider plugin.

## What Changed

- **SDK reason contract.** New
`packages/plugins/sdk/src/workspace-operation-error.ts`:
`WORKSPACE_OPERATION_FAILURE_REASONS` (11 reasons),
`PluginWorkspaceOperationError`, `workspaceOperationErrorData` and
`readWorkspaceOperationFailureReason`. It uses the same envelope design
as `environment-task-error.ts`. The worker host attaches the envelope
only to `workspaceOperation` errors. It copies only the reason, never
the message.
- **Duck-typed reason.** The worker host also forwards an own
`workspaceFailureReason` property from any error, when the value is in
the closed set. A provider that builds against an older SDK can set the
reason before it can import the class. An inherited or unknown value is
ignored. The value can only select one of Core's fixed messages.
- **Core classification.** New
`server/src/services/workspace-provider-failure.ts`.
`callWorkspaceProvider` maps each rejection to a
`WorkspaceProviderFailureError` (409) with `kind`, `reason` and
`retryable`:
- `worker_unavailable`: no request was sent. The
`PluginWorkerUnavailableError` stays as the cause, so run setup retries
it as `plugin_worker_unavailable` (the existing bounded bootstrap
retry).
- `timeout` and `worker_exited`: the outcome is unknown, so the message
still says to retry the same workspace.
  - `transport`: any other rejection. The message is the old message.
- `provider` with a reason: a fixed message per reason. Only transient
reasons say to retry.
- `provider` without a known reason: "Workspace storage provider
reported a failure; retry the same workspace".
- **Run settlement.** `failRunSetup` records
`resultJson.workspaceProviderFailure` (`kind`, `reason`, `retryable`).
The `heartbeat execution setup failed` log carries the same fields and
the run's `errorCode`. The run keeps `errorCode: setup_failed` (except
for the worker-not-running case above), so the existing setup-failure
retry, recovery and chat paths do not change.
- **Span allowlist.** The host accepts a `plugin.workspace.failure` span
and a `plugin.failure_reason` attribute. The value must be a reason from
the closed set or `unknown`.
- **Docs.** `doc/plugins/PLUGIN_RUNNER_EXECUTION.md` describes the
failure kinds and the reasons. `doc/observability.md` lists the new span
and attribute.

## Verification

- `cd packages/plugins/sdk && npx vitest run`: 21 files, 261 tests pass.
The new `tests/workspace-operation-error.test.ts` sends errors through
the real worker RPC host.
- `cd server && npx vitest run
src/__tests__/workspace-provider-failure.test.ts
src/__tests__/plugin-host-services-span.test.ts
src/services/heartbeat/run-completion.test.ts`: 3 files, 118 tests pass.
These tests cover:
- every transport kind, and that a worker refusal is still found as the
cause;
  - a real worker RPC rejection with a reason, and without a reason;
  - a fixed message and retry decision for every reason;
  - span clamping of `plugin.failure_reason`;
- run setup for a provider reason (no retry, reason in `resultJson` and
in the log) and for a stopped worker (bounded retry).
- `cd server && npx vitest run` for the 4 plugin workspace integration
suites (`plugin-workspaces`, `plugin-fresh-workspaces`,
`project-workspace-provider`, `workspace-delivery-cleanup`): 91 tests
pass. Their lost-reply fixtures still get the old "incomplete" message.
- `npx tsc --noEmit` in `packages/plugins/sdk`: no errors. `npx tsc
--noEmit -p server/tsconfig.json` shows no errors in the changed files.
The remaining errors come from workspace packages that I did not build
locally.
- Not run locally: the full `pnpm -r typecheck`, `pnpm test:run` and
`pnpm build`. CI runs them.

## Risks

- **Behavior change for a stopped worker.** A `workspaceOperation` call
that the worker did not receive now gets `errorCode:
plugin_worker_unavailable` and the bounded bootstrap retry, as other
plugin calls already do. Before, it was a `setup_failed` run with no
retry. No request reached the provider, so the retry is safe.
- **Message changes.** Provider-reported failures and timeouts get new
fixed messages. Code that matches the exact old message only sees it for
`transport` and `remote_incomplete` failures. I did not find a match on
this message outside tests.
- **No new error codes.** Provider failures keep `setup_failed`. Retry,
recovery and chat-retry routing does not change.
- **Duck typing.** Any error with an own `workspaceFailureReason` from
the closed set is forwarded. A provider can only select one of Core's
fixed messages. It cannot send text or values.
- **Observability.** The new span attribute is a closed set and stays
inside the host allowlist. No Telemetry change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edits, test runs). Extended thinking was on.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Related:
[paperclipai/paperclip#19326](https://github.com/paperclipai/paperclip/pull/19326)
added the same envelope design for Runner revision mismatches. I found
no duplicate PR for this change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
canary/v2026.1106.0-canary.0
2026-11-05 16:28:49 -08:00
f2593b7905 feat: make Coordinator and Agent Chat standard and speed up navigation (#19360)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People enter that work through the workspace, agent conversations,
and tasks.
> - Coordinator and Agent Chat still require experimental enrollment.
> - Task link prefetches do not warm the compact queries used by the
default task view. Chat also waits for a request when its navigation
list already holds the conversation.
> - This pull request makes Coordinator and Agent Chat standard and
reuses the reads needed for navigation.
> - People can choose Classic workspace in Experimental settings. Cached
work can appear while fresh data loads.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Workspace selection, Agent Chat availability, and navigation between
Chat, Tasks, and task details.

**Subsystem affected**

Cross-cutting: UI, workspace preferences, instance settings, and the
database migration.

**Current behavior**

People must enable Coordinator and Agent Chat in Experimental settings.
Opening a task can repeat requests because prefetch warms the full
detail and comment queries, while the default view reads compact task
data and a recent comment window. Opening Chat waits for a detail
request even when the current person's conversation list contains it.

**Proposed behavior**

Coordinator is the standard workspace for everyone. Experimental
settings offer Classic workspace, off by default. Old Coordinator
experiment values do not select Classic. Agent Chat is available to
everyone, including instances with an old saved or managed off value.
Task link intent warms the queries used by the mounted view. Chat can
use the current person's company-scoped conversation list during
refresh.

**Reason and benefit**

People can use the standard workspace without setup. Repeat navigation
can show cached content while the server refreshes it.

**Breaking changes**

Existing Coordinator experiment and Agent Chat off values no longer
disable these surfaces. Classic requires an explicit new opt-in. The
workspace API keeps its existing response shape and audit checks. The
migration retains legacy values and adds a separate Classic preference.
Execution permissions, company access, and chat privacy checks still
apply.

**Additional context**

Roadmap check: this graduates existing surfaces and improves their
navigation. It does not add a new agent execution capability. Duplicate
searches found no open Coordinator workspace PR. Related server cache
work, Refs: #17828, addresses compact issue-list invalidation; this PR
addresses client navigation queries.

## What Changed

- Add a default-off Classic workspace preference. Derive the existing
workspace API response from that choice. Keep personal persistence,
account isolation, and audited writes in the workspace preference
service.
- Replace the Coordinator experiment control with Classic workspace. Use
Coordinator while preferences load. Update the existing settings
examples and navigation recipe.
- Retire the Agent Chat experiment control. Ignore old stored and
managed off values. Retain the compatibility key and the existing
execution and access checks.
- Reuse account- and company-scoped chat list data with its original
freshness timestamp.
- Share compact task and recent-comment query options between prefetch
and the task view. Keep full-history prefetch for Classic tasks and
expanded history.
- Add upgrade, replay, cache, account isolation, and browser switching
regressions. Update product, database, development, and roadmap
documentation.

## Verification

- Targeted preference, settings, migration, cache, and UI regression
tests passed during implementation. The cache tests hold refresh
requests pending and check that content still renders. They also check
account isolation and stale refresh behavior.
- A real browser test verified fresh Coordinator defaults, Classic
opt-in persistence, and return to Coordinator. It used an isolated
source checkout with fresh data on macOS.
- Token, migration numbering and safety, map, and Storybook catalog
checks pass after the rebase.
- Repository typecheck and build pass. Typecheck and token gates also
pass after review repairs. The broad local run passed all six chat
integration shards (1,931 tests). It then found the host-CLI-dependent
recovery fixture described below. Its remaining duplicate work was
stopped after the full current-head CI matrix passed; this is not a
claim that the broad local command completed green.
- Review repairs passed 419 UI/migration fixture tests, 19 Coordinator
navigation tests, and all 214 tests in the five affected server suites.
The 24 Chat/workspace browser tests pass, including standard Coordinator
switching and explicit Classic persistence.
- The broader browser repair run passed 85 tests. It retained two failed
assertions and skipped five later serial cases. The affected files then
passed all 11 tests after the fixes: reopen the desktop panel after
reload and open More for Audit and Connectors. This includes the five
serial cases that had not run.
- The broad local suite exposed a recovery fixture that executed the
host Codex CLI during a simulated remote probe. On a host with Codex
0.154.0, model fallback broke its fixed-model assertion. The fake
sandbox now reports Codex 0.160.0. All 20 selected recovery/provider
compatibility cases pass after the repair.
- A CLI company import/export suite timed out during server startup in
the second CI run. The port fallback is intentional in that fixture. No
root cause was established. That suite passes on the final commit in CI.
This records recovery, not a claimed product fix for the earlier
timeout.
- The first CI run exposed remaining assumptions about the retired flags
and Classic UI. The repair commit updates those fixtures and navigation
steps. It also restores the missing enabled Cases destination in
Coordinator. Earlier local recovery attempts used the wrong fixture
notification and advanced fake time during live execution; the corrected
tests use the subscribed recovery queue and wall time during execution.
- Greptile reviewed commit `98972b030d4505d838427a2fca6b196b8cad6d47` at
**5/5** with no new comments. The original finding is resolved.
- Current-head CI is green: **56 successful checks**, with three checks
skipped by workflow filters. This includes typecheck, build, the
complete Vitest matrix, all eight browser shards, Runner verification,
release checks, and security checks. [Final CI
run](https://github.com/paperclipai/paperclip/actions/runs/42132651414).
- Typecheck caught missing `pid` and `startedAt` fields in the simulated
command result. The fixture now returns the complete process contract,
and the server test-layer typecheck passes.
- Storybook catalog alignment was checked. Storybook play tests were not
run. Hosted latency and cold bundle loading were not measured.

## Risks

- The new default changes the workspace for people who previously left
the Coordinator experiment off. They can opt into Classic in
Experimental settings.
- The additive migration must run before the new server code. It is safe
to replay and preserves explicit Classic choices.
- Cached content can be stale during background refresh. Existing query
invalidation and source timestamps still control freshness.
- These changes reduce avoidable loading waits. They do not establish a
measured improvement in hosted request latency or initial bundle
download time.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository tools, code
execution, and browser automation. The exact serving revision and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 16:28:25 -08:00
Devin Foley 1a36195868 fix(storage): name the S3 read when a response body is cut off (#19284)
Record read_object_body with HTTP status, S3 request id and attempts when a GetObject body stream errors after send() resolved, and add the storage fields to run failure execution diagnostics. Diagnostics only; no retry behavior changes.
2026-11-05 16:19:26 -08:00
Devin FoleyandPaperclip 8080620fc9 fix(workspaces): size archive deadlines and preserve resume budgets (#19364)
Give large workspace archives a bounded per-transfer deadline without changing
provider settings. Preserve configured post-upload command limits and the saved
provider budget when resuming an exact retained sandbox.

Verification: 341 focused Core/SDK/adapter tests and 588 Daytona tests passed.
Workspace and provider typecheck/build passed. Hosted CI: 54 successful checks,
two intended Storybook skips. Greptile 5/5 with no unresolved review threads.
The bounded full local run passed all six chat shards and stopped at twenty
minutes during general server suites; that full local run remains incomplete.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-05 16:09:32 -08:00
DottaandClaude Opus 5.5 a20dfbc695 fix(recovery): continue a message sent during Stop settlement after recovery (#19355)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A user can stop or interrupt a running native task and then send a
message to give new direction
> - When the stopped run left a workspace hold, the dispatch gate
withholds the follow-up run until recovery saves the work
> - The gate must save the user's message on its receipt so that the
deferred-receipt worker continues it after recovery
> - Two conditions made the gate skip the message instead of saving it,
so the task stayed in progress with "Saved work recovered. Send a
message to continue." although the user already sent one
> - This pull request saves the message in both conditions
> - The benefit is that the message the user already sent continues
once, automatically, after recovery

## Linked Issues or Issue Description

**What happened?**
After a user stopped or interrupted a native task with an isolated
sandbox workspace and sent a message, the follow-up run was cancelled
with `execution_reconciliation_required`. Its error said "Saved work
recovered. Send a message to continue." Nothing re-woke the task. In the
interrupt case, a periodic sweep started a run about four minutes later.
In the case where the follow-up first waited for the stopped run's
workspace save (`workspace_busy`), each later retry parked on the same
resolved recovery action, and the task did not continue.

**Expected behavior**
When the recovery that the follow-up waited on resolves, the message
that the user already sent continues once, automatically. The product
does not ask for a second message.

**Steps to reproduce**
1. Run a native task in a managed sandbox with an isolated workspace.
2. Stop the run (or send a comment with interrupt), then send a message
while the stop and workspace save are still settling.
3. Wait for the workspace recovery to resolve.
4. See that the follow-up run is cancelled with
`execution_reconciliation_required` and that no run continues the
message.

**Paperclip version or commit**
`master` before this change.

**Root cause**
`deferUnstartedUserComment` (run dispatch) refused to save the withheld
message in two cases:
1. It compared the message time with the source run's `finishedAt`. A
native run finishes only after its provider stop and workspace save
settle, which takes seconds. A message written after the Stop request
but before that settlement was treated as input written before the Stop.
The receipt was marked `skipped`.
2. It required the run's own receipt to be user-owned. A workspace-busy
retry or an admission retry carries the user's message under a
system-owned receipt, so the message was dropped with the retry.

## What Changed

- Add `userStopBoundaryAt(run)` in `run-cancellation.ts`. For a run
cancelled by a user Stop or interrupt, it returns the recorded Stop
request time (`cancellation.recordedAt`) when that is earlier than
`finishedAt`. For all other runs it returns `finishedAt`.
- Use this boundary in the dispatch gate's message deferral, in native
explicit-continuation admission, and in the remote-Stop continuation
check, so the three places agree on "message written after the Stop".
- When the withheld run is a system-owned retry, walk a bounded chain
(max 8) of retries that did not reach the provider (`workspace_busy`
with `providerWorkStarted: false`, or a verified pre-dispatch
`execution_reconciliation_required`) to the originating user receipt.
Save the message on that receipt with the user's original wake context.
Mark the retry's system receipt `skipped`.
- Update the "input written before the Stop" test so that the Stop
request, and not only the settlement, comes after the message.

## Verification

- New regressions in `server/src/services/workspace-recovery.test.ts`
(`continues a message ... without a second message, after recovery
already resolved`): a direct follow-up and a workspace-busy retry. Each
runs recovery to completion first, then lets the gate withhold the
follow-up, then runs only the deferred-receipt worker. Each asserts that
exactly one successor run is queued with the user's message and the
user's authority, and that no provider work starts.
- Red before the fix: both tests fail because the receipt is `skipped`
instead of `deferred_issue_execution`. Green after the fix.
- `npx vitest run src/services/workspace-recovery.test.ts` (384 passed).
- `npx vitest run src/services/explicit-native-continuation.test.ts
src/modules/run-dispatch src/services/execution-continuation.test.ts
src/services/run-cancellation.test.ts` (581 passed).
- 19 more heartbeat, retry, queued-comment, and native recovery suites
(1272 passed). One failure, `heartbeat-process-recovery > redacts opaque
environment-bound credentials from Sentry diagnostics: result`, also
fails on unchanged `master` and is not related.

## Risks

- A message written after the Stop request but before the stopped run
settled now continues. The stopping run cannot consume new input after
its Stop request, and admission still removes comments that a run
already delivered (`undeliveredLegacyUserCommentIds`). All admission
gates (holds, stop proofs, approvals, budget) still apply.
- Only user Stops and interrupts move the boundary. Other cancellations
keep `finishedAt`.
- The retry walk is bounded, stays in one company, agent, and issue, and
accepts only retries without provider work. It revives a user receipt
only from `cancelled` or `skipped`, so a second retry cannot save the
same receipt twice.
- Not in scope: a run that fails during native preparation with no user
message (for example `runner_state_identity_mismatch:
prior_owner_terminal_state_indeterminate;
retirement_refused=unexpected_root_entry` on a system wake). After its
recovery resolves, the task still waits for a user message. That is a
separate root cause in the native session executor's prior-owner
retirement proof.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 18:08:24 -06:00
DottaandPaperclip 7c0922a7b6 fix(native): preserve provider backups during slow workspace saves (#19359)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote native sessions save provider state so another web server can
continue the same conversation.
> - A provider can suspend while its accepted result is still saving a
large workspace.
> - The run has released its live lease at that point. A new
backup-owner lookup rejects this uncommitted state.
> - This pull request captures the real owner during backend creation
and retains it with the current session authority.
> - The existing database checks still reject a changed boot,
generation, result, or successor before inspection and during
publication.
> - The benefit is that a slow workspace save no longer prevents the
provider backup.

## Linked Issues or Issue Description

**What happened?**

A remote session can fail to save its provider backup with
`native_harness_storage_owner_missing`. The provider has suspended, but
its accepted result is still in `workspace_finalizing`. The close
callback tries to acquire a new owner after the run released its lease.
A later allocation cannot use a backup that was never published.

**Expected behavior**

Save the suspended provider with the authority acquired during
execution. Recheck that authority before provider inspection and before
and after upload. Keep result finalization and provider-work permissions
unchanged.

**Steps to reproduce**

1. Start a remote native session with durable provider storage.
2. Accept its result and release the live lease while its workspace save
remains pending.
3. Close the provider transport. The old owner lookup rejects the save
before provider inspection.

**Paperclip version or commit**

Reproduced on master `bf10bc695abaef6b20e85b287f456b5ce2edb266`.

**Deployment mode**

A web server with object storage and remote managed Runner sessions.

Related: #19225 retains failed-turn ownership through close. #18436
restores an external controller checkpoint before a changed warm process
is replaced. This change fixes late backup-owner acquisition while an
accepted result saves its workspace.

## What Changed

- Capture the provider backup owner when the remote backend still owns
its execution lease.
- Bind that owner to the current session authority. An older close
callback selects the current warm attachment's owner. Revalidate the
captured fence before inspecting or skipping provider state.
- Add a regression for the close callback during workspace finalization.
- Restore encrypted provider files with a captured owner while workspace
finalization remains pending. Reject boot, generation, result, and
successor changes during upload.
- Document the ownership contract.

## Verification

- Observed the callback regression fail on the old code with
`native_harness_storage_owner_missing`. The same test passes with this
fix.
- `cd server && node ../node_modules/vitest/vitest.mjs run --config
vitest.config.ts
src/services/native-runtime/native-harness-backup-storage.integration.test.ts`:
59 passed, using real PostgreSQL and encrypted object write/readback.
- `node scripts/check-module-boundaries.mjs`: passed.
- Full workspace typecheck: passed with the complete production fix.
- Native execution: 1,151 passed. Managed warm lifecycle: 9 passed.
These tests require a lost-owner error during shutdown, including before
an incomplete provider checkpoint is skipped. The final change to the
lifecycle test also requires rejection after lease renewal is lost.
- Full local `pnpm build`: passed on the final head.
- The full local `pnpm test:run` stopped in chat shard 6 on five
existing deadline assertions. Earlier chat shards passed; the PR CI chat
shards passed. This is not a full local-suite pass.
- The first CI server shard 4 attempt failed the unchanged legacy
environment-inheritance fixture at its five-second completion wait. The
same failure reproduced locally on the repair head and on an earlier
checkout without this repair. One unchanged CI retry passed. No fixture
assertion or timeout change is shipped.
- Latest-head CI has passed. Greptile reviewed the final head at 5/5
with no open threads. The separate Paperclip Review is paused. This PR
does not claim staging acceptance.

## Risks

The backup owner now lives with its session authority. A wrong authority
selection must fail rather than publish provider state. Database
publication still checks the original company, task, run, boot,
generation, and accepted result before and after upload. Tests change
the owner during upload and verify that the selected backup stays
intact. No schema, data migration, or new provider-work permission
changes.

## Model Used

OpenAI Codex, GPT-6 model family, with reasoning and code execution
tools. The host does not expose the exact runtime model ID or
context-window limit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-05 18:04:47 -06:00
Dotta b68f69a801 test(evals): pin author self-use and credential guard corrections (#19357)
## Thinking Path

> - Paperclip manages agents and their work.
> - The protected chat catalog selects an immutable evaluated source.
> - The measured parent produced seven independently reviewed passes out
of ten.
> - The authoring observer rejected valid author-only use, and two
credential wording guards needed stricter boundaries.
> - The reviewed source corrects those observers and declares each
affected oracle version.
> - This PR pins that source without changing the original questions or
run limits.

## Linked Issues or Issue Description

Refs #19346, #19095, #19093.

**What happened?**

The measured authoring response described real saved author-only use.
The old observer required zero assignments and wrongly rejected an
audited author-only assignment. The credential scanner also needed to
retain an unsafe comma-separated destination and mixed
reference/password requests. These are evaluation-layer corrections. The
parent timeline and company runs did not complete; no causal latency fix
is claimed here.

**Expected behavior**

Select the reviewed immutable observer source. Keep all original
scenario identities, questions, conditions, actor/resource proof,
counts, clocks, credentials, model, permissions and zero retries
unchanged.

**Steps to reproduce**

Run the source authoring and shared credential consumer controls.
Compare the committed candidate descriptor snapshot with source
`d1ab241cac0db7f700fadaaf62ffb3d2b51cf0c3`.

## What Changed

- Scope the private Company creation observer to the exact Create secret
dialog. Its successful close can open a separate details Sheet without
making the observer wait for that Sheet.
- Admit blank or bounded non-secret metadata defaults only for the
Company saved-secret-name question variant. Verify the same canonical
payload in independent human accounting.
- Pin source `d1ab241cac0db7f700fadaaf62ffb3d2b51cf0c3`, tree
`c4efaef62405cbb37236ad01db1187cf8a156868`.
- Store exact descriptor SHA256
`60db015337e807ff53a376b6db11c375a239097b224d2253dad68ecd0b0f61f3`.
- Update only seven oracle-version leaves: authoring v5, company v11,
personal v12, and four shared external consumers v6. Source-file lists
remain fixed.
- Retain provenance anchors, historical conditions and failures. Update
the single expected-target assertion. Exactly three metadata/test files
change.

## Verification

- 41 manifest, adapter, reconciliation and CLI controls pass on this
actual pin projection.
- Qualified source: one actual-worker offline browser canary passes all
three creation, failed-POST and missing-form mechanisms. Full E2E types
pass on this successor. Reused unchanged proof: 286 distinct controls
across authoring, Company saved-name question, human accounting and
shared-consumer/declaration suites pass. Full E2E typecheck passes. All
242 declarations load with zero fixture callbacks.
- SDK506 inputs and registry `efa70621` remain unchanged. Launcher
SHA256 `2b9a6953`, private transport interfaces, production/UI/CLI
source, lockfile and workflows remain unchanged from the measured
source.
- All 244 YAML queries, context and expected outcomes are byte-identical
to the parent. All descriptor native/human counts, clocks,
authentication, setup policy and source-file lists remain fixed.
- The parent source has seven measured semantic passes out of ten. The
new authoring and credential oracle versions are native-unrun. Existing
failed grades remain failed.
- The isolated company concurrency diagnostic
[run42124035785](https://github.com/paperclipai/paperclip/actions/runs/42124035785)
is a separate measurement. This pin does not alter concurrency or claim
to fix native noncompletion.
- Repo-wide build/tests were not repeated for this three-file metadata
update. Normal PR CI remains required.

## Risks

- The name character set does not prove secrecy. The saved-key-name
variant can admit a credential-shaped synthetic default; this general
classifier limit remains unfixed and tracked in #19093. These are finite
experimental journeys only. Independent review must reject any actual
credential-like or unknown-value prefill without clear
public-metadata/proposed-name grounding, even if autograding is green.
If safety cannot be established, the journey cannot receive PASS credit.
Secure UI save, human answer, canonical payload/hash, audit, binding and
byte-leak checks remain mandatory.
- Earlier CI includes three explicit runner-shutdown cancellations and
one deferred-observer test assertion at line573. Those outcomes remain
retained and unresolved; the Company dialog repair makes no claim to fix
them. New exact-head CI is required.
- The Company dialog mismatch was established from production source and
an actual-worker model-free regression before the next secure-save step.
The prior isolated attempt did not reach private entry, so it is not
evidence of an observed private-save failure.
- A valid review warning remains as a documented follow-up in #19093:
safe closing comma exclusions such as “only there” can be falsely
rejected by the credential wording guard. The pin authorizes bounded
experimental journeys only. If a valid actual handoff hits that guard,
its failed grade is retained; no regex failure is waived or counted as
PASS. Independent semantic review remains required for both secret
handoffs.
- Isolated Company original5055 remains FAILED. A real succeeded native
took255949ms and produced a valid saved-name question, but the observer
rejected its proposed non-secret metadata default. Secure entry and the
second native were not reached. This successor repairs that assertion
boundary only.
- The next selected74/466/578 run is an explicit bounded
lower-concurrency functional experiment at the existing cap2, same
model, original clocks and zero retries. It is not evidence of a latency
fix or reliability improvement.
- Local observer proof is not a new native PASS. Timeline and company
noncompletion remain unresolved, and no clock increase is included.
- A secure handoff still requires independent semantic review before
PASS credit. Actual saved resources, owner/audit evidence, provider
outcomes and privacy gates remain required.
- The source catalog remains separate from master. This PR does not
merge the core or full catalog feature changes into master.

## Model Used

OpenAI Codex, `gpt-6.1-sol`, with repository tools and code execution.
The host does not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-05 17:57:18 -06:00
Devin FoleyandClaude Opus 5.5 0f2c01b63a fix(server): retry one Cloud portfolio connection reset (#19033)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - On Cloud instances, `GET /api/cloud/stacks` gets the user's stack
portfolio from the Cloud control plane.
> - The route sends one `fetch` to `/v1/tenant/portfolio`. It has a 10 s
deadline.
> - The Cloud edge can reset a pooled keep-alive connection when the
request is written. The request then fails in a few milliseconds and the
user gets a 502.
> - The diagnostics from #15340 now show this failure mode on staging.
> - This pull request retries the idempotent GET one time for a
pre-response connection reset, inside the same deadline.
> - The benefit is that a transient connection reset no longer shows an
error to the user. Real Cloud failures and deadlines are still reported.

## Linked Issues or Issue Description

Refs #15340 (bounded portfolio diagnostics). Refs #10850 (portfolio
proxy). I searched the open and closed PRs for "portfolio". No other PR
fixes this failure.

**What happened?**

On the hosted staging pool stack, one `GET /api/cloud/stacks` returned
502. The diagnostics were `phase: fetch`, `networkCode: ECONNRESET`,
`upstreamStatus: null`, `elapsedMs: 19`. The Core process had started
about 7 minutes before. CPU and memory were normal.

The Railway HTTP logs for the Cloud harness show no failed or missing
portfolio request at that time. All portfolio requests in a 12-minute
window returned 200 in 15–91 ms. The next Core portfolio fetch, 0.4 s
later, reached Cloud and returned 200. Thus the failed request did not
get to the Cloud application. The connection was reset between Core and
the Cloud edge.

This failure is different from the older stalls on the same stack. Those
failures had `DEADLINE_EXCEEDED` after more than 60 s, during
whole-process freezes. This change does not retry those failures.

**Steps to reproduce**

1. Start an HTTP server that resets (or closes) the socket for the first
request and returns JSON for the second request.
2. Set `PAPERCLIP_CLOUD_API_ORIGIN` to that server and call `GET
/api/cloud/stacks` with the native `fetch`.
3. Before this change, the route returns 502 with `networkCode:
ECONNRESET` (or `UND_ERR_SOCKET`). Native `fetch` does not retry.

**Expected behavior**

The route retries the idempotent GET one time when the connection is
reset before a response. The retry uses the same 10 s signal. Other
failures stay unchanged.

## What Changed

- `server/src/routes/cloud.ts`: If the fetch phase fails with
`ECONNRESET` or `UND_ERR_SOCKET`, the route sends the same GET one more
time. The retry uses the same `AbortSignal.timeout(10_000)`. The total
time stays at or below 10 s.
- The route classifies the first error with the existing bounded
`CloudPortfolioError` diagnostics. Thus a reset after the deadline stays
`DEADLINE_EXCEEDED` and gets no retry.
- The first reset gets a warning log with the same safe fields
(`Paperclip Cloud portfolio connection reset; retrying once`). It does
not send a Sentry event. Thus the logs still show how frequently this
occurs.
- A second reset, a deadline, HTTP error statuses, body failures,
`ECONNREFUSED`, `ENOTFOUND`, connect timeouts and unknown errors behave
as before: one report and no retry. Failures are still not cached.
- `server/src/__tests__/cloud-routes.test.ts`: The old test "classifies
a fetch reset without retrying" now expects one retry. New tests cover a
second reset, the codes that get no retry, a reset after the deadline,
and two real-socket cases.

`doc/observability.md` describes the diagnostic context. Its fields and
semantics do not change. Per `CONTRIBUTING.md`, the change adds no code
comments; the retry rationale is in this description.

## Verification

- Red before the fix: with the old route and the new tests, `pnpm exec
vitest run server/src/__tests__/cloud-routes.test.ts` gave 5 failed, 19
passed. The failures were the two mocked retry cases, the second-reset
case and the two real-socket cases. Thus native `fetch` on Node 25.9
does not retry these resets itself.
- Green after the fix: `pnpm exec vitest run
server/src/__tests__/cloud-routes.test.ts
server/src/__tests__/cloud-portfolio-error.test.ts` gave 31 passed.
- Real dependency: the real-socket tests use `node:http` and the native
global `fetch`. `socket.resetAndDestroy()` gives `ECONNRESET`.
`socket.destroy()` gives `UND_ERR_SOCKET`. Each test makes sure that the
second request reaches the server with the same authorization, the route
returns 200, and no Sentry event occurs.
- The second commit only removes the four-line comment that Greptile
flagged. It has no runtime effect; a syntax check passed locally and CI
reruns the suite.
- Not run locally: `server/src/__tests__/sentry.test.ts` fails to load
in my sandbox on the unchanged base and with this change. The error is
the same: `Failed to resolve entry for package
"@paperclipai/plugin-sdk"`. The workspace packages were not built. The
full `tsc -p server` was stopped by the 8 GB sandbox memory limit. A
typecheck of only the changed files gave the same output on the base and
with this change, apart from a 5-line offset. CI runs the full typecheck
and test suite.
- Remaining limit: the staging evidence shows that the request did not
reach the Cloud application. It does not show if the reset socket was a
reused keep-alive connection or a new connection. The retry is correct
in both cases.

## Risks

- Low risk. The retry applies only to an idempotent GET that has no side
effects on Cloud. It occurs only for a fetch-phase reset before a
response.
- The worst case is one more portfolio request to Cloud for each reset.
The deadline is the same as before.
- If the Cloud edge resets all connections, the user sees the same 502
as before, after two fast attempts.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool
use (shell, code execution, Sentry, Railway and GitHub connectors).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:39:18 -08:00
Devin FoleyandClaude Opus 5.5 db95539d46 fix(agents): refuse agent purge when cost history references it (#19022)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A board user can permanently delete (purge) a terminated agent
> - The purge deletes the agent's heartbeat runs, but cost and finance
events still point to those runs
> - Postgres then rejects the delete with a foreign key error, and the
API returns a 500
> - Cost and finance events are accounting history. The purge must not
remove them
> - This pull request makes the purge check for this history first and
return a clear 409
> - The benefit is a clear, expected response instead of a 500, and
accounting history stays intact

## Linked Issues or Issue Description

**What happened?**
Permanent agent deletion failed with `Failed query: delete from
"heartbeat_runs" where "heartbeat_runs"."agent_id" = $1`. The cause is
`update or delete on table "heartbeat_runs" violates foreign key
constraint "cost_events_heartbeat_run_id_heartbeat_runs_id_fk" on table
"cost_events"`. The server reported this as an unexpected error. Sentry:
PAPERCLIP-BACKEND-4Z.

**Steps to reproduce**
1. Run an agent so that a heartbeat run records a cost event.
2. Terminate the agent.
3. Permanently delete (purge) the agent.

**Expected behavior**
The purge stops before it deletes anything. It returns a clear conflict,
and the agent stays terminated. This matches the existing
`agent_case_history_retained` behavior.

**Actual behavior**
The transaction fails on the `heartbeat_runs` delete with a raw foreign
key error (500).

Related, older approaches: #9130 and #1987 tried to delete cost and
finance rows during agent removal. This PR keeps the ledger instead.

## What Changed

- `deleteAgentDependencies` now checks for `cost_events` and
`finance_events` rows that reference the agent or one of its heartbeat
runs. It does this before any destructive cleanup.
- If such rows exist, it throws `409 agent_cost_history_retained`.
Termination stays available.
- The activity-log cleanup reuses the same agent-runs subquery.
- New embedded-Postgres regression test
`server/src/services/agent-cost-history-deletion.test.ts`. It covers a
cost event on a run, a cost event attributed to the agent, a finance
event on a run, and a normal purge with no cost history.

## Verification

- Without the fix, 3 of the 4 new tests fail with the exact Sentry
error. Examples: `cost_events_heartbeat_run_id_heartbeat_runs_id_fk`,
`cost_events_agent_id_agents_id_fk`,
`finance_events_heartbeat_run_id_heartbeat_runs_id_fk`.
- With the fix, all 4 pass.
- Related suites pass (8 files, 249 tests): `lifecycle.test.ts`,
`execution-admission/adapters/postgres.test.ts`,
`decision-models.test.ts`, `prompt-injection-prevention.test.ts`,
`fast-responses.test.ts`, `agent-timer-deadlines.test.ts`,
`case-deletion.test.ts`, and the new test.
- `tsc --noEmit` in `server/` reports 0 errors.

## Risks

- Behavior change: agents with recorded cost or finance history can no
longer be purged. Before this change, these purges already failed with a
500, so no purge that used to succeed is now blocked.
- No schema change and no migration.
- Docs follow-up: `doc/SPEC-implementation.md`, `doc/DEVELOPING.md` and
`maps/agent-access-map/agents.md` describe
`agent_case_history_retained`. They do not yet mention the new
`agent_cost_history_retained` code. I can add that in this PR if
reviewers want it.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:30:36 -08:00