Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-12 06:10:44 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,997 Commits 1,827 Branches 2,432 Tags
3ff62fc6dbe4c3c2937f98ab3c019ffe0a364b64
Commit Graph
2545 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 3ff62fc6db feat(recovery): model an unclaimed pending-cleanup item as owned by the cleanup queue (#19980) 
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100): short sentences, one instruction per sentence, simple
approved vocabulary, and the active voice. -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A sandbox that still needs teardown waits as a `pending_cleanup`
environment lease until the cleanup queue retries it
> - The ownership reconciler checks invariant I1: every in-progress
record names an owner
> - An unclaimed pending-cleanup item names no process by design, so I1
reported every one (61 in the Phase 2 data)
> - That noise hides real ownership gaps
> - This pull request models the item honestly: the cleanup queue owns
it, with a visible next action at its saved retry deadline
> - The benefit is that I1 measures real gaps, and a queue that stops
acting shows up as an I7 gap instead of silence

## Linked Issues or Issue Description

Refs #19933

**What happened?**
`environment_pending_cleanup` listed every unclaimed pending-cleanup
item with owner `none`, so I1 reported each as `no_owner_stamp`.

**Expected behavior**
The item names the queue that acts on it and its next due time. It is
reported only when the queue is not acting on it (long past the
deadline). No owner is invented, and nothing is released.

**Steps to reproduce**
1. Let a sandbox teardown fail so its lease waits in `pending_cleanup`.
2. Run the ownership reconciliation pass.
3. See an I1 `no_owner_stamp` violation for a healthy queue item.

## What Changed

- `recovery/ownership-registry.ts`: new owner kind `queue` (`queue`,
`dueAt`, `graceMs`). `environment_pending_cleanup` names the environment
cleanup queue. Its due time is the same retry deadline the cleanup
worker sleeps until, including the at-once first attempt of a settled
orphan.
- `recovery/ownership-invariant-report.ts`: a queue owner is alive
(`queue_scheduled`) while its deadline is current. When it is more than
two retry intervals past, it is unprovable (`queue_overdue`) and
reported as `I7_visible_next_action`. A queue is never proven gone and
nothing is settled.
- The listing reads the in-flight flag as text, so a malformed value
cannot break it.

## Verification

- `cd server && npx vitest run
src/__tests__/environment-pending-cleanup-queue-owner.test.ts
src/__tests__/ownership-reconciliation-report.test.ts` (embedded
Postgres).
- Red on master, green here (written after the code): a current item
names the queue (no I1); a long-overdue item is an I7 `queue_overdue`
gap.
- Green on both: an in-flight item is not listed here (it is a teardown
claim).
- Full typecheck is left to CI (local memory limit).

## Risks

- Low risk. Report only: no record is changed.
- The grace (two retry intervals) is a reporting threshold, not a
release rule. A worker that sleeps longer than that for a legitimate
reason would show as an I7 gap.
- `#19966` changes the same listing predicate to the same text
comparison; a rebase of either resolves trivially.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edit, embedded Postgres tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-08 00:39:11 -06:00
DottaandClaude Opus 5.5 95f3cab0cd fix(recovery): settle a workspace finalize attempt whose process died mid-copyback (#19985) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The durable-ownership plan settles in-progress records whose owner
is proven gone, on one shared path.
> - The crash-point suite (#19946) found two windows where a workspace
finalize attempt stays `running` forever after a restart, even though
its run succeeded through a later attempt.
> - #19807's supersession covers only native-finalizer attempts. The
local live-finalize path writes rows without that marker.
> - This pull request stamps each finalize attempt's owner and settles
an abandoned attempt on the shared path.
> - The benefit is that no finalize row is left `running` with a dead
owner, and the run's outcome never changes.

## Linked Issues or Issue Description

Refs #19946 (crash-point suite), #19807 (RC-64 supersession), #19933
(shared settle path).

**What happened?**
At crash points `workspace.finalize.after_start_before_copyback` and
`workspace.finalize.after_copyback_before_done` (run
`cp-20261108T040253`, tasks t07 and t08), the run succeeded after the
restart. The original `workspace_finalize` operation stayed `running`,
and the ownership pass reported I2 (`owner_run_terminal`).

**What did you expect?**
The interrupted attempt is settled once its owner is proven gone, and
the run outcome is left alone.

**Steps to reproduce**
1. `node scripts/crash-points.mjs --point
workspace.finalize.after_start_before_copyback`. Or run the new report
test case below.

## What Changed

- `workspace-operations.ts`: every `workspace_finalize` attempt records
`finalizeOwner` (the process stamp, with no I/O) and is tracked in a
process-local set while it runs (`isWorkspaceFinalizeOperationLive`).
- `ownership-registry.ts`: `native_workspace_finalize_attempt` takes its
owner from the stamp, or from the run for rows written before the stamp.
It now registers a settle.
- New `recovery/workspace-finalize-attempt-settlement.ts`: a
compare-and-swap that marks the attempt `failed` with
`abandonedFinalization` provenance and logs a
`workspace_operation.reconciled` activity. It skips an attempt live in
this process, an uncertain export and a save-work record. It does not
change the run.

## Verification

- `cd server && npx vitest run
src/__tests__/ownership-reconciliation-report.test.ts`: 19 pass. The new
case covers a dead stamped owner (settled), a current-process owner
(kept), another PID space (kept), an uncertain export (kept), a
pre-stamp row of a terminal run (settled) and of a running run (kept), a
read-only pass (no change), and the run status unchanged.
- The finalize-related suites pass (100 tests in 5 files):
`heartbeat-workspace-finalize-branch`,
`native-workspace-finalization-work`,
`workspace-operations-reconciliation`,
`offline-workspace-operation-reconciliation`,
`finalized-workspace-recovery-migration`.
- Red/green: the new case fails on master, where the attempt stays
`running`. I wrote the test after the code.
- Crash-point re-proof with `--point` for both windows: to be run once
the harness slot is free, and reported on this PR.

## Risks

- Fail-open risk: settle needs a `gone` proof from the shared prover,
and another PID space is never proof. It re-checks the stamp in the
transaction and skips attempts this process is running.
- Previous-release story: rows without `finalizeOwner` are owned by
their run, and they settle only when that run is terminal.
- Nothing runs before `turn.start`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-08 00:17:02 -06:00
DottaandClaude Opus 5.5 ccce55ab11 fix(issues): tell callers when a task list page is capped (#19972) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents read company tasks through `GET
/api/companies/{companyId}/issues`, often through the runner `call_api`
tool
> - The route caps each page at `limit` (500 by default, 1000 at most)
and returns a bare array
> - A capped page and a complete list look the same, so an agent cannot
tell that rows are missing
> - In an eval with 501 open tasks, the agent got 500 rows and reported
498 tasks as the complete list
> - This pull request adds a "more results exist" signal and the
next-page position, and surfaces it to agents
> - The benefit is that agents page until the list is complete instead
of presenting a partial list

## Linked Issues or Issue Description

**What happened**

The chat eval `tasks.organize-open-work` seeds 501 open Launch tasks.
The agent called `GET /api/companies/{companyId}/issues` through
`call_api`. The route returned exactly 500 rows (the default limit) and
gave no signal that more rows existed. The agent noted that exactly 500
rows was suspicious and that some tasks were missing. It still reported
the partial list as complete.

**Expected behavior**

A capped list response tells the caller that more results exist and how
to get the next page. The agent-facing tool result shows the same signal
next to the data.

**Steps to reproduce**

1. Create 501 tasks with status `todo` in one company.
2. Call `GET /api/companies/{companyId}/issues?status=todo`.
3. See that the response has 500 rows and no header or field that says
more rows exist.

## What Changed

- `server/src/routes/issues.ts`: the company issue list fetches one row
past the page. It then sets `X-Paperclip-Has-More` (`true`/`false`) and
`X-Paperclip-Limit` on every page. A capped page also sets
`X-Paperclip-Next-Offset`, and `X-Paperclip-Next-After-Id` for
`sortField=id&sortDir=asc` cursor paging. Pagination is reported only to
actors who can read the whole company's task list; actors whose rows are
filtered after the query (such as low-trust agents) get no pagination
headers, so the raw page cannot reveal tasks they cannot read.
`oneChildPerParent` lists are ordered by parent first, so they page by
offset and never advertise an id cursor. The body is unchanged (still an
array). The Mine inbox path does not change.
- `server/src/services/native-runtime/runner-api-client.ts`: `call_api`
results for successful responses that carry these headers include
`pagination: { hasMore, truncated, limit, nextOffset, nextAfterId?,
guidance? }`. The guidance tells the agent to repeat the call with the
next `offset`/`afterId` until `hasMore` is false. The field is also
present when the body is saved as an artifact. Routes without the
headers get no new field.
- `server/src/routes/openapi.ts`: the API catalog entry documents
`limit`, `offset`, `sortField`, `sortDir`, `afterId`, the headers and
the `call_api` pagination field.
- Tests for capped, cursor, last, exact-size and default-limit (501
rows) pages in all list projections, for offset-only paging of
`oneChildPerParent` lists, for a low-trust agent receiving no pagination
headers, and for the `call_api` pagination field.

## Verification

- `cd server && npx vitest run
src/__tests__/issues-list-query-parsing.test.ts
src/__tests__/low-trust-red-team-routes.test.ts` — 31 passed.
- `cd server && npx vitest run
src/__tests__/issues-list-query-parsing.test.ts
src/services/native-runtime/runner-api.test.ts` — passes. Without the
source change, 8 of the new tests fail.
- `cd server && npx vitest run src/__tests__/inbox-mine-routes.test.ts
src/__tests__/issue-list-assignee-filter-routes.test.ts
src/__tests__/issue-list-updatedsince-filter-routes.test.ts
src/__tests__/openapi-routes.test.ts
src/services/native-runtime/runner-api*` — 2237 passed, 2 skipped.
- `tsc --noEmit -p server/tsconfig.json` shows no errors in the changed
files.

## Risks

- Low risk. The response body shape does not change. Existing clients
ignore the new headers.
- Each list query reads one more row. The extra row is enriched and then
dropped. The cost is small.
- `call_api` results gain an optional `pagination` field. Consumers that
do not know the field ignore it.
- Agents whose task reads are filtered after the query (low-trust
agents) still get no truncation signal, as before; a raw-page signal
would leak hidden tasks. Pushing their boundary into the list query
would let them have one later.
- Other capped list routes do not set the headers yet. They can adopt
the same headers later.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:50:08 -06:00
DottaandClaude Opus 5.5 bf0d45a4d2 feat(recovery): settle teardown claims of dead claimants on the shared ownership path (#19966) 
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100): short sentences, one instruction per sentence, simple
approved vocabulary, and the active voice. -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A sandbox teardown writes a claim on its environment lease that
fences other cleanup for 15 minutes
> - After a restart, the cleanup sweep ended the fence of a claim whose
claimant had exited, with its own process check
(`releaseLostPendingCleanupClaims`)
> - Phase 2 of the durable ownership plan moves each in-progress record
kind onto one shared owner proof and deletes its bespoke check
> - This pull request registers a settle for
`environment_teardown_claim` and calls the shared path where the sweep
called the bespoke release
> - The benefit is one proof for every kind: a local claimant proves
death by its stamp; another host stays a fence handoff

## Linked Issues or Issue Description

Refs #19965
Refs #19964
Refs #19933

**What happened?**
The teardown claim had its own liveness check
(`controllerProcessExited`) in the pending-cleanup sweep. Each kind with
its own check can drift from the shared rules (PID space, PID reuse,
current boot).

**Expected behavior**
The shared prover judges the claimant stamped for this exact attempt. On
gone, the fence ends now and a counted sweep attempt is refunded once. A
claim held as a fence (another claimant or another host) waits for its
deadline.

**Steps to reproduce**
1. Start a sandbox teardown so the lease holds an in-flight claim.
2. Kill the server.
3. Restart. The claim must be released by the shared ownership path, not
by a separate check.

## What Changed

- `environment-teardown-claim-settlement.ts` (new): compare-and-swap
settle on the exact attempt and claimant. It ends the fence, refunds a
counted attempt, and is single-use. It never acts on a fence owner.
- `recovery/ownership-registry.ts`: `environment_teardown_claim`
registers the settle.
- `heartbeat/recovery.ts`: the sweep calls `settleOwnableKind(db,
"environment_teardown_claim")`. Deleted
`releaseLostPendingCleanupClaims` and its use of
`controllerProcessExited`.
- `environment-cleanup-state.ts`: the attempt cap, attempt key and their
SQL readers move here, so the settle and the sweep share one reader.
- `heartbeat-pending-cleanup-sweep.test.ts`: the overlap barrier now
holds the shared prover instead of the deleted check.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-pending-cleanup-sweep.test.ts
src/__tests__/environment-lease-owner-stamps.test.ts
src/__tests__/ownership-reconciliation-report.test.ts` (embedded
Postgres).
- The existing lost-claim matrix stays green on the shared path: retry
now; no failed-attempt charge for a dead server; one refund when sweeps
overlap; keep a live claim; keep a newer claim that kept an older
record; keep another host's claim; keep a claim without identity; page
past fenced claims.
- New, red before and green here (written after the code): the
reconciler pass ends a dead claimant's fence and refunds its counted
attempt.
- New counter-cases (green): a read-only pass never settles; a live
claimant, another PID space, and an unstamped fence claim keep their
fence.
- Full typecheck is left to CI (local memory limit).

## Risks

- Low risk. The settle keeps the previous release semantics (fence ends,
lease stays `pending_cleanup` and in flight until the sweep claims it
again).
- The stop-only claim still does not renew its fence; a live stop that
runs longer than 15 minutes can still be taken over. This change does
not alter that.
- Stacked on #19965 and #19964. Merge those first.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edit, embedded Postgres tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:48:43 -06:00
DottaandClaude Opus 5.5 d33b3ac263 fix(wake-queue): save temporarily refused work intents and promote them through one path (#19956) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Users send work to agents as assignments, comments and Retries. The
wake queue delivers each one as a run
> - When an admission check refused a wake for a temporary reason, the
wake often ended as a terminal skip, so the work was lost
> - Restart-chaos waves found dropped comments and Retries (RC-37,
s7532) and double delivery (RC-53). Each fix covered one hold type
> - The durable ownership plan (section E) asks for one outbox rule: a
closed set of terminal outcomes, a saved deferral for every temporary
refusal, and one promotion path
> - This pull request adds that rule. It saves messages refused by a
pause, promotes deferred intents from every hold-release event through
one function, and adds a reconciler backstop
> - The benefit is that an accepted message is delivered exactly once
after a pause lifts, and the reconciler reports any intent that is stuck
or dropped

## Linked Issues or Issue Description

Refs #19831 (plan, section E), #19838 (inventory, wake-queue rows and
section 1b), #19927 (Phase 2 data: the I4 false positive), #19702,
#19741, #19829, #19826. Builds on #19944 (section D, merged).

**What happened?**
- A user commented while the agent was paused, or while the company was
paused. Admission refused the wake. Nothing saved the message as
pending, so when the pause lifted, nothing delivered it.
- Hold-release events woke different subsets of workers. A saved intent
then waited out a 30-second refusal backoff after its hold cleared.
- The I4 reconciler check matched free text with a regular expression.
Phase 2 found a false positive: a delivered retry whose error said
"retrying once".

**Expected behavior**
An accepted work intent ends only as delivered, consumed by a later run,
task cancelled, or hard stop. Every temporary refusal saves the intent
as `deferred` with its reason and its release trigger. Every
hold-release event promotes deferred intents through one function
(invariant 4).

**Steps to reproduce**
Pause an agent. Post a user comment on its task. Resume the agent. On
`master` the comment is never delivered as a wake.

## What Changed

- New `server/src/services/work-intent-outbox.ts`:
- `WORK_INTENT_TERMINAL_OUTCOMES`: the closed set of terminal outcomes.
- `TEMPORARY_REFUSAL_TRIGGERS`: maps each temporary refusal code to its
release trigger.
- `promoteDeferredWorkIntents`: the one promotion function. It removes
the 30-second refusal backoff from the matching deferred intents. It
then wakes run dispatch, the saved-receipt worker and stranded-work
recovery, and hints each saved intent of the issue.
- `heartbeat/queue.ts`: the #19829 save-as-deferred block is now a
shared helper. A user's message refused by a paused agent, a paused
company, or a budget pause that is not a reached hard stop is saved with
`executionWait: { reason, trigger, message }`. The caller still gets a
409. The message is saved only when the task's latest run is terminal,
because the release drain promotes a saved message behind that run. A
task that never ran keeps the message in the task, and stranded-work
recovery starts the task once the pause lifts. A `pending_approval`
agent and a task-tree hold are not changed here: the release drain still
fails a promoted wake for a pending approval, and tree holds have their
own resume path.
- Hold-release events call `promoteDeferredWorkIntents`:
  - accounting settled (`run-cost-accounting.ts`);
  - environment lease released (`environments.ts`, two sites);
  - native run projected terminal (`native-run-projection.ts`);
  - budget pause lifted (`budgets.ts`);
  - company reactivated (`companies.ts`);
  - task-tree hold released (`issue-tree-control.ts`);
  - agent lifecycle ready (`agent-lifecycle-driver.ts`).
- Reconciler (`recovery/ownership-registry.ts`,
`ownership-invariant-report.ts`, `native-finalization-reconciler.ts`):
- registers three new kinds: `wake_request_deferred`, `queued_run` and
`scheduled_retry_run`;
- I4 now reads the closed refusal code (the wake `reason`) of accepted
intents, not free text;
- a new I4 check reports a deferred pause refusal whose pause has lifted
for more than 5 minutes. The worker then promotes it as a backstop;
- a lifted pause inside its 5-minute grace window returns its deadline
through `nextDueAt`, and the worker also wakes on agent-lifecycle
signals (company and agent pause changes). So a missed release event is
still checked.
- `doc/SPEC-implementation.md`: adds the rule "Accepted work intents are
delivered exactly once".

## Verification

Each focused file ran alone in one vitest process (`--maxWorkers=1`) on
embedded Postgres. Results at `2875c85fd2`:

| Question | Loop | Elapsed | Result |
|---|---|---|---|
| Is a comment refused by a pause saved, then delivered once after the
pause lifts? | `heartbeat-held-comment-wake.test.ts -t "pause lifts"` |
8 s | 2 pass. With `queue.ts` from `master`, both fail: no intent is
saved |
| Does I4 use the closed code, and does it catch a stuck deferral? |
`ownership-reconciliation-report.test.ts` | 45 s | 20 pass. This
includes the Phase 2 "retrying once" false positive and the case
"paused, then lifted" |
| No regression on the paths that changed | held-comment (15),
archived-company guard (8), undispatched execution wait (32), deferred
receipt work (45), recovery pause durability (16), native run projection
work (55), tree-control routes and services (19, 1, 7) | 8–334 s each |
All pass |

- Exactly-once: each pause test runs two promotions (a release event,
then the reconciler backstop) and many dispatch passes. It asserts one
delivery. The existing consumption tests (RC-53) still pass.
- Server package typecheck passes.
- Speed: nothing new runs before `turn.start`. The new admission work
happens only on a refusal. Promotion runs in the release transaction and
in the background reconciler.
- Not run here: chaos waves. The coordinating session schedules them on
the shared harness.

## Risks

- `promoteDeferredWorkIntents` moves `updated_at` back for deferred
intents that were refused in the last 30 seconds. A busy company with
many release events can retry refused intents more often. Each retry
re-runs normal admission, and the refusal resets the backoff.
- More release sites now wake run dispatch and stranded-work recovery.
These notifications are coalesced per company.
- Lifting a pause is now the operator's explicit authorization to send
what the pause refused, as Send is after a Stop. When the pause stopped
the task's run, the lifted pause issues the task's next explicit wake,
which adopts the saved message once. Other messages queued behind an
acknowledged Stop keep the existing rule. If Stop recovery still holds
the task, the message waits for that recovery.
- An agent pause's stop sweep keeps the messages that the pause deferred
(`preservePauseDeferred`). Terminate and cleanup stops still end them.
- The backstop check reads only pause refusals (`company.inactive`,
`agent.not_invokable`), because those holds can be checked exactly.
Other deferred kinds keep their existing promotion paths.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:43:17 -06:00
DottaandClaude Opus 5.5 5742c42b75 fix(chat): tell agents to write company links as Markdown links (#19981) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - In chat, an agent often gives the person a link to the next place to
act: a task, an approval, a run or an agent profile.
> - Chat guidance gives the correct paths, but it does not say how to
write them.
> - The chat makes links from Markdown links and issue identifiers. It
does not make a link from a bare `/<prefix>/approvals/<id>` path.
> - An agent proposed a hire and wrote the approval as a bare path. The
person could not click it to decide the approval.
> - This pull request adds one sentence to the chat guidance: write
every company link as a Markdown link.
> - The benefit is that the person gets a link they can open.

## Linked Issues or Issue Description

Refs #19344 (similar guidance for completion feedback about created
resources).

**What happened**

In the `agents.hire-needs-approval` chat eval, the CEO proposed a hire
and replied with `- Review the pending hire: /RUN/approvals/<id>`. The
chat rendered the path as plain text. No link to the approval was
rendered. In an earlier run of the same case, the agent wrote
`[/RUN/approvals/<id>](/RUN/approvals/<id>)`, which rendered as a link.
The guidance did not say which form to use.

**Expected behavior**

Company links in chat replies are Markdown links that the person can
click.

**Steps to reproduce**

1. Set a company to require board approval for new agents.
2. Ask the CEO in chat to hire a QA engineer.
3. Look at the reply. Sometimes the approval path is plain text, and you
cannot click it.

## What Changed

- `server/src/services/agent-conversations.ts`: the chat directive now
says to write every company link as a Markdown link, for example
`[Review the approval](/<prefix>/approvals/<approvalId>)`. It says that
a bare path or a path in inline code is not clickable.
- `server/src/__tests__/agent-conversations.test.ts`: assert the new
guidance.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/agent-conversations.test.ts
src/__tests__/codex-local-execute.test.ts`: 71 passed.
- An end-to-end rerun of the `agents.hire-needs-approval` chat eval with
this change follows on the eval branch.

## Risks

- Low risk. This is a change to guidance text only. The prompt gets one
sentence longer.
- The model can still write a bare path. This change makes that less
likely. It does not rewrite replies.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:37:07 -06:00
DottaandClaude Opus 5.5 05db5fdd9a fix(agents): hired OpenCode agents inherit the hiring agent's OpenRouter key reference (#19977) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents can hire teammates. A hired agent inherits the credential
references of the agent that hired it, so it can run without a separate
setup step.
> - The inheritance map covers Claude, Codex and Grok. It does not cover
OpenCode.
> - An OpenCode CEO hires a QA agent. The board approves the hire. The
new agent has no OpenRouter key, and verification fails with
`harness_auth_failed`.
> - The person approved a working hire, but they get a broken agent that
they must fix by hand.
> - This pull request adds `OPENROUTER_API_KEY` to the inheritance map
for `opencode_local`.
> - The benefit is that an approved OpenCode hire becomes ready with the
same account as the agent that proposed it.

## Linked Issues or Issue Description

Refs #13268 (the change that added credential inheritance for hires).

**What happened**

A native OpenCode CEO (provider `opencode`, `OPENROUTER_API_KEY` stored
as a `secret_ref`) proposed a QA hire with `inheritRuntimeFrom: caller`.
The company requires board approval for new agents. The board approved
the hire. The agent lifecycle verification then failed with
`failureCode: "harness_auth_failed"`, and the agent did not become
ready. The hired agent's `adapterConfig` had no `env`.

**Expected behavior**

The hired agent gets the same OpenRouter secret reference as the hiring
agent. After approval, verification passes and the agent becomes ready.

**Steps to reproduce**

1. Create a company with `requireBoardApprovalForNewAgents: true`.
2. Create a `paperclip_runner` agent with `provider: "opencode"` and
`env.OPENROUTER_API_KEY` as a `secret_ref`.
3. As that agent, call `POST /api/companies/:companyId/agent-hires` with
`adapterType: "paperclip_runner"` and `inheritRuntimeFrom: "caller"`.
4. Read the new agent. Its `adapterConfig.env` has no
`OPENROUTER_API_KEY`.
5. Approve the hire. Verification fails with `harness_auth_failed`.

## What Changed

- `server/src/routes/agents.ts`: add `opencode_local:
["OPENROUTER_API_KEY"]` to `INHERITABLE_AGENT_CREDENTIAL_ENV_KEYS`. This
is the key that AI connections bind for `opencode_local`
(`packages/shared/src/ai-connections.ts`).
- `server/src/__tests__/agent-hire-auth-inheritance-routes.test.ts`: add
a regression test. A native OpenCode parent hires with
`inheritRuntimeFrom: caller` under board approval. The child gets the
same `secret_ref` (with its version). The plain
`OPENCODE_CONFIG_CONTENT` value is not copied.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/agent-hire-auth-inheritance-routes.test.ts`: 27 passed.
The new test fails without the route change (`expected undefined to
match object { type: 'secret_ref', … }`).
- `cd server && pnpm exec vitest run
src/__tests__/agent-adapter-validation-routes.test.ts
src/__tests__/agent-permissions-routes.test.ts
src/__tests__/agent-hire-ai-connections.test.ts
src/__tests__/ai-connections.test.ts`: 446 passed.
- The failure was seen in the `agents.hire-needs-approval` chat eval
(runner-opencode profile). The approved hire failed verification with
`harness_auth_failed`. An end-to-end rerun with this change follows on
the eval branch.

## Risks

- Low risk. The existing inheritance rules still apply: only
`secret_ref` and `user_secret_ref` values are copied, a key the hire
request supplies wins, the parent must be in the same company and use
the same harness, and a parent with a managed AI connection passes
nothing.
- An OpenCode hire can now use the parent's OpenRouter account. This
matches the existing behavior for Claude, Codex and Grok hires.
- Other OpenCode provider keys (for example `OPENAI_API_KEY`) are not
added. AI connections bind only OpenRouter for `opencode_local`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:37:03 -06:00
DottaandClaude Opus 5.5 576ba29fff feat(recovery): settle the legacy controller lease on the shared ownership path (#19953) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Legacy (non-native) runs hold a controller lease on `heartbeat_runs`
while the server prepares and dispatches the adapter. The shared
ownership reconciler now settles proven-gone owners (Phase 2 of the
durable ownership plan)
> - The lease stamped only a boot id. The reconciler could never prove
its owner gone (`missing_identity` in the Phase 2 data), and recovery
released it by age: lease expiry (60 s), then a 5-minute age guard on
dispatched runs
> - A hard-killed server's legacy run therefore stayed `running` for up
to 6 minutes after the next boot
> - This pull request records the controller process with the claim,
registers a compare-and-swap settle, and makes live-run recovery use the
shared prover instead of the age guard when the controller is proven
gone
> - The benefit is that a dead controller's run is settled at once,
while a live, unprovable, or previous-release controller keeps today's
behavior

## Linked Issues or Issue Description

Refs #19933 (the settle hook, merged). Stacked on #19950 (migration
0446, registry) and targets its branch until it merges; then it
retargets to master.

**What happened?**
After a hard kill, a dispatched legacy run stayed `running` until its
controller lease expired and its last update was 5 minutes old.

**What did you expect?**
The next boot proves the old controller gone and fails the run as a lost
process at once.

**Steps to reproduce**
1. Start a legacy (non-native) run and let it dispatch the adapter.
2. Kill the server (SIGKILL).
3. Start the server. The run stays `running` for up to 6 minutes.

## What Changed

- `heartbeat_runs.controller_process` (jsonb, migration 0447): `{bootId,
pid, hostHash, processStartedAt}` of the claiming server process. It
names the controller only while its `bootId` equals
`controller_boot_id`. Revocation swaps the boot id, so a revoked lease
never keeps it.
- `legacyControllerClaim` writes it in the same UPDATE (from
`currentControllerProcessStamp()`, no I/O).
- `legacy-controller-lease.ts`: `legacyControllerOwnerStamp`,
`legacyControllerProvenGone` (calls `proveOwnerGone`), and
`settleLegacyControllerLease` (compare-and-swap expiry of the proven
controller's outstanding lease after it wakes live-run recovery).
- `ownership-registry.ts`: `legacy_controller_lease` registers `settle`
and `listForRuns`. The owner is the bound process; other rows are
boot-id fences. The listing covers outstanding leases only.
- `heartbeat/recovery.ts` (3 small hunks): a controller of another boot
that the shared prover proves gone has its live lease settled through
`settleOwnableKind` and skips the age guard. The local child-process
check after it is unchanged, so a live child still keeps the run
running.
- Migration 0447 builds the partial index with `CREATE INDEX
CONCURRENTLY` (heartbeat_runs is a known-large table) and is added to
`MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`.

Bespoke code deleted: none. The kind had no liveness proof; the age
guard stays for controllers that are not proven gone.

## Verification

- `cd server && npx vitest run
src/services/legacy-controller-lease.test.ts`: 33 passed.
- `cd server && npx vitest run
src/__tests__/ownership-reconciliation-report.test.ts`: 17 passed.
- Red with `recovery.ts` and `ownership-registry.ts` from the base
commit (tests, schema and lease module from this branch): 4 failed, 29
passed. Red: the crash-point reaper test, the expired-lease age-guard
test, the reconciler settle test, the exact-controller CAS test. The
tests were written after the code.
- New tests:
  - the claim stamps this process, and a changed boot id unbinds it;
- crash point: a controller dispatched the adapter and was killed with
its lease live; the startup reaper (default 5-minute age guard) fails
the run as `process_lost` at once;
  - a dead controller's expired lease skips the age guard;
- counter-cases keep the run and lease through the reaper and a
reconciler pass: a live earlier-boot PID, a dead PID in another PID
space, and a previous-release row with no `controller_process`; an
expired lease of a live controller stays behind the age guard;
- a read-only pass (`record: false`) never settles; a recording pass
expires the lease and leaves the run to live-run recovery;
  - settle refuses a controller that changed after the proof.
- Not run locally: typecheck (CI runs it) and the wide suites.

## Risks

- Previous-release story (rules/upgrade-safe-run-state.mdc): rows saved
by the previous release have no `controller_process`. They stay boot-id
fences with today's release (expiry, revocation, age guard). A rollback
ignores the column.
- Unprovable controllers (another host or PID space, unknown space, a
live PID with an unreadable start time) never release early. Their
handoff stays the lease expiry plus revocation, and the age guard. A
different host is never proof.
- Behavior change: a dispatched run whose controller is proven gone is
failed as `process_lost` without the 5-minute wait. The retry decision
is unchanged.
- The settle expires the lease only. Live-run recovery still settles the
run under its own checks. Accounting and workspace obligations are not
touched.
- The index build is concurrent. A cancelled build is retried by the
migrator's invalid-index handling.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:27:54 -06:00
DottaandClaude Opus 5.5 0cb9770d1c feat(recovery): name the owner of teardown claims and login/probe leases (#19965) 
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100): short sentences, one instruction per sentence, simple
approved vocabulary, and the active voice. -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Environment leases carry several in-progress records: login/probe
leases and in-flight teardown claims among them
> - The ownership reconciler (durable ownership plan, Phase 2) must know
who owns each record, or invariant I1 reports a gap
> - The ownership data showed one teardown claim and one login/probe
lease with no owner stamp
> - A teardown claim could also show the wrong owner: a controller stamp
left by an earlier attempt named a dead process while a live claimant
held the claim
> - This pull request names an honest owner for both kinds, without a
new write before `turn.start`
> - The benefit is that I1 measures real gaps, and the later settle for
these kinds can trust the owner it is given

## Linked Issues or Issue Description

Refs #19964
Refs #19933

**What happened?**
The ownership reconciler reported `environment_teardown_claim` and
`environment_login_probe_lease` records as I1 `no_owner_stamp`.
Workspace recovery and native export retry claim teardown without a
controller stamp. A login/probe lease never had an owner stamp. A
teardown claim that kept an older attempt's controller stamp was
attributed to that older process.

**Expected behavior**
Each in-progress record names the owner that can still move it: a
process proved by identity, or a fence whose expiry is never proof by
itself. A stamp for a different attempt never names the owner.

**Steps to reproduce**
1. Let workspace recovery claim a pending-cleanup lease (no controller
stamp), or open a login session lease.
2. Run the ownership reconciliation pass.
3. See `no_owner_stamp` for the record, or a dead earlier controller for
a claim that a live claimant holds.

## What Changed

- `recovery/ownership-registry.ts`, `environment_login_probe_lease`: the
owner is the holder process stamped by the write that makes the lease
active (`metadata.leaseController`, added in #19964). A lease saved by
the previous release still names no owner; it ends with its login
session.
- `recovery/ownership-registry.ts`, `environment_teardown_claim`: a
controller stamp counts only when its `attemptId` equals the claim's
`pendingCleanupAttemptId`. Any other claim with an attempt id is a
fence: the attempt id is the token and `pendingCleanupLeaseExpiresAtMs`
(renewed by the claimant) is the expiry. An expired fence stays
unprovable. A claim with no attempt id names no owner.
- Report-only. No settle is registered for these kinds in this change.

## Verification

- `cd server && npx vitest run
src/__tests__/environment-lease-owner-stamps.test.ts
src/__tests__/ownership-reconciliation-report.test.ts` (embedded
Postgres).
- Red on the previous registry, green here (tests written after the
code):
  - login/probe lease is owned by the process whose write made it active
  - login/probe lease is proven gone when that process has exited
- teardown claim never reads a controller stamped for an earlier attempt
(fence held by its token)
- an unstamped teardown claim whose fence expired stays unprovable,
never gone
- Green on both: a stamped claim for this attempt with a dead PID is
gone; an unstamped login lease and an attempt-less claim report I1.
- Full typecheck is left to CI (local memory limit).

## Risks

- Low risk. The change only alters the owner the report-only reconciler
reads. No record is changed.
- Stacked on #19964 (the lease holder stamp). Merge #19964 first.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edit, embedded Postgres tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:17:40 -06:00
DottaandClaude Opus 5.5 f858f79678 fix(agents): keep agent termination board-only on the PATCH status route (#19955) 
## Thinking Path

- Paperclip lets a board govern which agents exist; terminating an agent
is irreversible and is meant to be a board-only governed action.
- `POST /api/agents/:id/terminate` enforced that with `assertBoard`, but
`PATCH /api/agents/:id` with `{ "status": "terminated" }` ran the same
termination after only the agent-configuration permission check.
- The `agents.retire-agent` chat eval caught it: the CEO agent got 403
from `POST /terminate`, then terminated Morgan through `PATCH` and got
200.
- This change puts one board-only gate in front of both routes, and
makes the denial useful by giving a ready link to the agent's profile,
where a person can terminate it.

## Linked Issues or Issue Description

**What happened**

An agent that is allowed to configure another agent could terminate it
with `PATCH /api/agents/:id` `{ "status": "terminated" }`, bypassing the
board-only check on `POST /api/agents/:id/terminate`.

**Expected behavior**

Termination is board-only on every route. A non-board actor gets 403
with a link to the agent's profile so it can send a person to the
Terminate control.

**Steps to reproduce**

1. As an agent with agent-configuration permission over another agent,
call `POST /api/agents/{id}/terminate` (403).
2. Call `PATCH /api/agents/{id}` with `{ "status": "terminated" }` (200
before this change; the agent is terminated).

## What Changed

- `server/src/routes/agents.ts`: new `assertBoardCanTerminateAgent`,
used by `POST /terminate` and by `PATCH` whenever the requested status
is `terminated`. Board actors pass; others get 403 "Board access
required: only a board user can terminate an agent…" with a Markdown
link to `/<prefix>/agents/<id>`.
- `server/src/__tests__/agent-permissions-routes.test.ts`: both routes
are denied for an agent allowed to configure the target, and neither
terminates, updates, cancels invocations nor logs activity. A board user
can still terminate through `PATCH` status.

## Verification

- `pnpm exec vitest run src/__tests__/agent-permissions-routes.test.ts`
(server): 104 passed; the denial cases fail without the change.
- End to end: the `agents.retire-agent` chat eval (run 42445668869) now
sees 403 on both routes, Morgan stays active, and the agent links
Morgan's profile with the Terminate step.

## Risks

- Any non-board automation that terminated agents through `PATCH` status
now receives 403 and must use a board session. That is the intended
governance boundary.

## Model Used

Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code, with
tool use (shell, file editing).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:08:44 -06:00
DottaandClaude Opus 5.5 fda58d8cc1 fix(chat): post the complete final reply, not a 4,000-character run-log preview (#19971) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People talk to agents in chat. The agent's final message becomes the
chat reply.
> - The runner cuts every completed agent message to a 4,000-character
run-log preview. The server then posts that preview as the reply.
> - A long answer, for example a list of 501 open tasks, reaches the
person cut off with "…[truncated]".
> - PR #19959 fixed only the case where the accepted summary starts with
the exact preview text. It missed runs where the final prose differs
from the summary.
> - This pull request keeps the complete final answer in the runner, and
makes the server never prefer a preview over a complete source.
> - The benefit is that the person receives the agent's complete reply.

## Linked Issues or Issue Description

Supersedes #19959.

**What happened**

A chat agent wrote a final answer of about 4,985 characters (all 501
tasks). The posted chat reply was a 4,000-character preview that ended
with "…[truncated]".

**Expected behavior**

The person receives the agent's complete final message. A preview of a
message is never posted when a complete text exists.

**Steps to reproduce**

1. Ask a chat agent (OpenCode or Codex) to list a large set of open
work, so that its final answer is longer than 4,000 characters.
2. Let the run finish.
3. Read the chat reply. It ends with "…[truncated]" at 4,000 characters.

Root cause:

-
`packages/paperclip-runner/runner/crates/runner-core/src/provider_events.rs`:
the codex/OpenCode facade projection of `item/completed` and the ACPX
`AssistantMessage` projection cut the final answer with
`bounded_text(text, MAX_TEXT_CHARS)` (4,000 characters).
- `durable/state.rs` `sanitize_value` caps every string again at 4,096
bytes before the event goes to the server.
- `server/src/services/heartbeat-run-summary.ts`
`resolveHeartbeatRunResponse` gives an explicit `final` agent message
top precedence, so it posts the preview.

## What Changed

- Runner (`provider_events.rs`): a completed `final_answer` agent
message keeps its complete redacted text, up to a new 32,000-character
`MAX_FINAL_AGENT_MESSAGE_CHARS` bound. Progress prose and in-flight
items keep the 4,000-character preview.
- Runner (`durable/state.rs`): `preserve_final_agent_message_text` keeps
the complete redacted text of a completed final agent message when it
fits in half of the transport frame. Otherwise the marked preview stays,
so a large message never fails the frame check.
- Server (`heartbeat-run-summary.ts`):
`readCompletedAssistantMessageCandidate` marks a stored preview as
`bounded`. The resolver never posts a bounded preview while a complete
source exists. It posts a marked adapter final response or the accepted
semantic summary instead
(`bounded_final_agent_message_replaced_by_semantic_summary`). It posts
the marked preview only when no complete text exists
(`bounded_final_agent_message_without_complete_source`). This also fixes
replies from runners that are already deployed.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-run-summary.test.ts`: 86 passed. Six new tests:
a complete final message longer than 4,000 characters is posted in full;
the old exact-prefix case posts the complete accepted answer; a preview
whose prose differs from the summary posts the complete summary (also
with chat precedence); a marked adapter final response wins over a
preview; the marked preview is posted only when nothing complete exists;
a stored preview candidate is marked `bounded`. Five of the six fail
against the `master` resolver.
- `cargo test -p paperclip-runner-core --lib -- final_answer
complete_final`: 3 passed. Both new runner tests fail when the fix is
reverted.
- `cargo test -p paperclip-runner-core --lib`: 581 passed, 3 failed. The
failures are `task_memory::tests::monitor_*` (OOM-kill monitor timing
tests). They do not touch the changed code.
- `cargo test -p paperclip-runner-core --test acpx_event_projection
--test acpx_rich_events --test local_integrity_boundary_golden --test
acpx_provider_events`: all passed. `cargo fmt --check` passes.
- Evidence from the failed eval run: the `item.completed` final
`agentMessage` text was exactly 4,000 characters (3,988 retained plus
the marker). The `run.result.proposed` summary was complete at 4,985
characters.

## Risks

- Larger durable events: a final answer can now be up to 32,000
characters (at most about 128 KB). The outbox keeps the preview when the
text exceeds half of the frame size, so the frame check does not fail.
- Redaction stays the same. The complete text goes through
`redact_sensitive_text_values`, which the preview also used.
- The server treats any final message that ends with "…[truncated]" as a
preview. If an agent writes that text itself, the server posts the
complete accepted summary instead. That summary is also the agent's own
complete answer.
- The runner part reaches users only after a runner release. The server
part fixes replies at once when an accepted summary exists.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 23:01:00 -06:00
DottaandClaude Opus 5.5 cc072e7c4e feat(recovery): settle orphaned active environment leases on the shared ownership path (#19964) 
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100): short sentences, one instruction per sentence, simple
approved vocabulary, and the active voice. -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs hold environment leases (local workspaces and remote
sandboxes) while they execute
> - A crash between the terminal run write and the lease release leaves
the lease `active` with no holder
> - A sweep released such a lease when the run was terminal and the
lease was idle for five minutes. The age was the only guard against a
race with the live executor, which releases the lease itself with the
run's resource disposition
> - Phase 2 of the durable ownership plan moves each in-progress record
kind onto one shared owner proof (#19933 added the settle hook)
> - This pull request moves `environment_run_lease_active` onto that
path: the lease names its holder, the shared prover judges the holder,
and the settle never destroys a sandbox
> - The benefit is that a dead holder's lease settles on the next pass,
a live holder is never raced, and the five-minute age gate is gone

## Linked Issues or Issue Description

Refs #19933

**What happened?**
An orphaned active environment lease was released by age: "run terminal
plus five minutes" (the reaper in `heartbeat/recovery.ts` and the
live-recovery `orphanLease` worker). The ownership data showed one such
lease settle 119 s after it was ready. Without the age gate the sweep
could flip a lease to `pending_cleanup` (destroy) while the executor or
a recovery path was about to retain it.

**Expected behavior**
A run lease settles when its holder is proven gone, by identity and not
by age. A live holder is never raced. A settle never destroys a sandbox
whose files may still be needed.

**Steps to reproduce**
1. Start a run that holds an environment lease.
2. Kill the server after the run status becomes terminal and before the
lease release.
3. Restart. The lease stays `active` until the five-minute age passes,
then the sweep destroys the sandbox.

## What Changed

- `environments.ts`: the write that makes a lease active (insert and
same-run reacquire) stamps the holder process in
`metadata.leaseController`. The stamp does no I/O
(`environment-lease-controller.ts`).
- `recovery/ownership-registry.ts`: `environment_run_lease_active` lists
leases with their run status. The owner is the run while it is active,
then the stamped holder. It registers `settle` and `listForRuns`.
- `environment-run-lease-settlement.ts` (new): compare-and-swap settle
on the exact row (same run, status `active`, same metadata, terminal
run). A sandbox lease gets the stop-and-retain intent (stop and keep,
never destroy). Other leases move to `pending_cleanup` as before. It
keeps a lease that a pending native export, a legacy shutdown fence, a
shared provider resource, or an accepted native result still needs.
- `recovery/ownership-invariant-report.ts`: `settleOwnableKind` accepts
`currentProcessIdle`. A caller that passes every run this process
executes in `skipRunIds` may treat a current-process owner of another
run as gone (`current_process_idle`). The periodic pass never does this.
- `heartbeat/recovery.ts`: `sweepOrphanedActiveLeases` no longer
releases run leases by age. It runs native workspace preservation first
(a separate obligation), then calls `settleOwnableKind` with this
process's executions. Login and probe leases (no run) keep the idle
sweep until their own change.
- `heartbeat/live-recovery-work.ts`: a run lease is due now; only
run-less leases keep the five-minute idle deadline.
- `sandbox-stop-and-retain.ts`: an ownership settle records a recovery
marker on the intent and keeps `updatedAt`, so the pending-cleanup sweep
stops the sandbox in the same tick.
- Deleted: the age gate for run leases and the page-order bump for run
leases.

## Verification

- `cd server && npx vitest run
src/__tests__/environment-run-lease-ownership.test.ts
src/__tests__/heartbeat-orphaned-active-lease-sweep.test.ts
src/__tests__/ownership-reconciliation-report.test.ts` (embedded
Postgres).
- Red on unchanged master sources, green on this branch (tests were
written after the code):
  - stamps the lease holder in the write that makes a lease active
- settles a lease whose holder died after the run went terminal, on the
next pass (crash-point state: terminal run, dead holder in this PID
space)
  - stops and keeps a dead holder's sandbox instead of destroying it
- leaves a current-process lease to this process: settled only when it
is not executing the run
  - keeps the run fact alone for one release on a single-host server
- names no owner on an ephemeral server
(`PAPERCLIP_EPHEMERAL_SERVER=true`)
- the three updated sweep tests (sandbox stopped and kept, not
destroyed)
- Live-owner counter-cases (green): run still active; another live PID;
another PID space; current process while executing; read-only pass
(`record: false`) never settles; a lease re-stamped after the proof is
not settled; a legacy lease whose run this boot owns is kept.
- Also run green: `recovery.test.ts`,
`heartbeat-observed-workspace-preservation.test.ts`,
`workspace-recovery-warm-handoff.test.ts`,
`environment-service.test.ts`,
`heartbeat-pending-cleanup-sweep.test.ts`,
`native-environment-settlement-work.test.ts`,
`environment-cleanup-work.test.ts`.
- Full typecheck is left to CI (local memory limit).

## Risks

- Behavior change: an orphaned sandbox lease is now stopped and kept,
not destroyed. Workspace preservation and provider auto-delete decide
when it goes away. This can keep stopped sandboxes longer than before.
- A recovery path on another process can mark a run terminal while the
dead executor's stamp is still on the lease. The settle can then win
before that path releases. Because the settle only stops and keeps the
sandbox, files are not lost; the later release continues from the
stop-only intent.
- Previous-release rows: a single-host server keeps the run fact for one
release (a run this boot owns is excluded). An ephemeral server never
releases an unstamped lease; it stays visible as an I1 gap with no
operator action yet.
- No migration. The stamp lives in existing lease metadata.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edit, embedded Postgres tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 22:43:57 -06:00
DottaandClaude Opus 5.5 81f12c21ec fix(server): document the safe company import request body in the API catalog (#19968) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A CEO agent can preview and apply a company package import through
`POST /api/companies/{companyId}/imports/preview` and `/imports/apply`
> - Agents find API operations and their request bodies in the OpenAPI
catalog
> - The catalog entries for these two safe import routes have no request
body and no description of the safe-route limits
> - An agent then offered the person a `replace` choice that the safe
route rejects
> - This pull request registers the existing preview and import body
schemas and describes the limits
> - The benefit is that agents can resolve name conflicts with `rename`,
`skip` and `nameOverrides`, and do not offer choices that fail

## Linked Issues or Issue Description

**What happened**

A person attached a company package in chat and asked a CEO agent what
the package would add. The package had an agent named Morgan, and the
company already had a Morgan. The agent previewed the import correctly.
It then offered three choices: `skip`, `rename`, or `replace`
("overwrites the existing Morgan"). The safe import route rejects
`replace`. The catalog entries for `/imports/preview` and
`/imports/apply` have no request body, so the agent could not see the
route's fields or limits. PR #19945 fixed the same gap for `/exports`.

**Expected behavior**

The catalog documents the import body, including `source`, `target`,
`collisionStrategy` and `nameOverrides`. It states that the safe route
rejects `replace` and never overwrites an existing agent.

**Steps to reproduce**

1. Read `GET /api/openapi.json`,
`paths["/api/companies/{companyId}/imports/preview"].post`.
2. See that it has no `requestBody`. The legacy
`/api/companies/import/preview` entry has one.
3. Call the safe route with `collisionStrategy: "replace"`. It returns
403.

## What Changed

- Register `companyPortabilityPreviewSchema` and
`companyPortabilityImportSchema` as the JSON request bodies of `POST
/api/companies/{companyId}/imports/preview` and `/imports/apply` in
`server/src/routes/openapi.ts`.
- Add a description of the safe-route limits: target, allowed collision
strategies, and `nameOverrides`.
- Add a regression test in
`server/src/__tests__/openapi-routes.test.ts`.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/openapi-routes.test.ts` passes (37 tests). The new test
fails on master because the entries have no `requestBody`.
- An end-to-end check runs the `companies.import-preview` chat eval with
this commit cherry-picked onto the eval branch.

## Risks

- Low risk. This changes only API documentation. The routes already
parse these schemas. Runtime behavior does not change.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 22:35:16 -06:00
DottaandClaude Opus 5.5 a480335e69 fix(chat): tell agents how to read the company link prefix from a task identifier (#19967) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Chat agents reply with links to tasks, approvals, runs and agent
profiles
> - Each link starts with the company prefix, for example
`/RUN/agents/<id>`
> - The chat guidance said to derive the prefix from the task
identifier, but it did not say how
> - An agent decided that `RUN-1` does not give the prefix and invented
one, so its links were broken
> - This pull request states how to read the prefix and forbids prefixes
built from names, slugs, hostnames or paths
> - The benefit is that agent profile links and other company links in
chat replies open the correct page

## Linked Issues or Issue Description

**What happened**

A person asked a CEO agent in chat to preview and then import a company
package. The chat task was `RUN-1`. The agent's reasoning said: "Task
identifier RUN-1 doesn't give prefix directly." The agent then linked
agent profiles as `/runner-e2e/agents/<id>`. Each link opened
"Organization not found: No organization matches prefix RUNNER-E2E".

**Expected behavior**

The agent links `/RUN/agents/<id>`. The guidance tells the agent that
the prefix is the identifier part before the final hyphen and number.

**Steps to reproduce**

1. Open a chat with a CEO agent in a company whose prefix differs from
its slugified name.
2. Ask it to do something with another agent, for example import a
package that creates an agent.
3. See that the reply can link `/<invented-prefix>/agents/<id>`, which
opens "Organization not found".

## What Changed

- Replace the task-link sentence in `AGENT_CHAT_DIRECTIVE`
(`server/src/services/agent-conversations.ts`). It now says that the
prefix is the identifier part before the final hyphen and number
(`RUN-1` gives `RUN`). It says that all company links use that prefix.
It forbids a prefix built from a company name, slug, hostname or path.
- Add a regression test in
`server/src/__tests__/agent-conversations.test.ts`.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/agent-conversations.test.ts` passes (46 tests).
- An end-to-end check runs the `companies.import-preview` chat eval with
this commit cherry-picked onto the eval branch.

## Risks

- Low risk. This changes only prompt guidance text. Runtime behavior
does not change.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 22:32:38 -06:00
DottaandClaude Opus 5.5 bf3a067076 feat(recovery): settle the finalizer and cleanup leases on the shared ownership path (#19950) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs keep their ownership in `native_run_finalizations`, and
the shared ownership reconciler now settles proven-gone owners (Phase 2
of the durable ownership plan)
> - The finalizer lease (`native-finalizer:*`, 5 min) and the
retained-session cleanup lease (`native-cleanup:*`, 60 s) recorded only
a random token. They named no holder process
> - A server killed while it finalized or cleaned up kept the run
waiting for the whole lease. The lease was released by age alone. A
graceful shutdown also keeps an in-progress finalizer lease for the next
boot to wait out
> - This pull request records the holder process with each claim and
moves both kinds onto the shared prover and a compare-and-swap settle
> - The benefit is that a dead holder's run continues in the first pass
after boot, and a live or unprovable holder is never released

## Linked Issues or Issue Description

Refs #19933 (the settle hook and the first moved kind, merged).

**What happened?**
A finalizer or cleanup lease whose server process died stayed held until
the lease expired (5 min or 60 s). The run stayed `running` and its
follow-up work waited.

**What did you expect?**
The next boot proves the dead holder gone and frees the lease at once. A
live holder, or one that cannot be proven gone, keeps its lease.

**Steps to reproduce**
1. Accept a native result and let the finalizer claim it.
2. Kill the server (SIGKILL) before the finalizer commits.
3. Start the server. The run stays `running` for up to 5 minutes.

## What Changed

- `native_run_finalizations.lease_holder` (jsonb, migration 0446):
`{token, bootId, pid, hostHash, processStartedAt}` of the process that
took a token-only lease. It names the holder only while `token` equals
`lease_owner`, so renewals keep it and a later lease never inherits it.
The executor's `controller_*` columns are not changed.
- The finalizer claim (`native-run-finalizer.ts`) and the cleanup claim
(`native-session-executor.ts`) write it in the same UPDATE as the lease,
from `currentControllerProcessStamp()` (no I/O).
- New `native-token-lease-settlement.ts`: compare-and-swap settle that
expires exactly the proven-gone holder's outstanding lease, after it
notifies the finalization or cleanup work that the expiry frees.
- Both claims also wake the ownership worker queue in their transaction,
so a quiet worker watches the new holder (Greptile round 1).
- `ownership-registry.ts`: `native_finalizer_lease` and
`native_cleanup_lease` register `settle`. Their owner is the holder
process; rows without a holder keep the token fence. Listings cover
outstanding leases only.
- `heartbeat/recovery.ts`: restart recovery settles both kinds before it
finalizes and schedules cleanup. A failure is logged and the reconciler
pass retries it.
- Two sparse partial indexes on `lease_expires_at` for the two listings,
so the settle watch re-check stays indexed.

Bespoke code deleted: none. These kinds had no liveness check; their
only release was lease age (the takeover in the claim), which stays as
the fence for unprovable holders.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-native-restart-suspension.test.ts -t
"finalizer|cleanup|holder|exact lease"`: 12 passed (rerun after the
rebase on master). The crash-point test also asserts the ownership wake
for its run; it fails without the signal.
- `cd server && npx vitest run
src/__tests__/ownership-reconciliation-report.test.ts`: 17 passed.
- Red on the base branch (server code at `bea3bfb20b`, tests and schema
from this branch): 7 new tests fail, 5 pass (the 5 are existing tests).
Red: the crash-point restart test, the two reconciler settle tests, the
three live/unprovable counter-cases (owner was a token fence), and the
exact-lease CAS test. The tests were written after the code.
- New tests:
- crash point: a finalizer claim was written and its holder killed
before commit; the next restart finalizes the run, and this boot's claim
stamps itself as holder;
- a dead finalizer or cleanup holder is settled by a recording pass,
never by a read-only pass (`record: false`); the accepted result and
phase are untouched;
- live counter-cases: a live earlier-boot PID (`pid_alive`), the current
boot (`current_process`), a dead PID in another PID space
(`different_process_space`, unprovable) keep the lease through a pass
and a restart;
- a holder written for another token, and a holderless row of the
previous release, stay token fences, also with
`PAPERCLIP_EPHEMERAL_SERVER=true`;
- settle refuses a lease taken over after the proof
(`owner_changed_since_proof`).
- Not run locally: typecheck (CI runs it) and the wide suites.

## Risks

- Previous-release story (rules/upgrade-safe-run-state.mdc): rows saved
by the previous release have no `lease_holder`. They stay token fences
and the existing claim takes them over at expiry. Neither lease outlives
5 min, so this covers at most 5 min after the upgrade. A rollback
ignores the new column.
- Unprovable holders (other host or PID space, unknown space): no
operator action. The lease is short and self-expiring; at expiry the
next claim takes it over and the old holder's next token
compare-and-swap fails. That fence is the handoff. A different host is
never proof.
- The settle only expires a lease. Finalization and cleanup still run
under their own checks (cleanup keeps its maintenance-history proofs).
Settling ownership does not settle the accepted result, workspace
preservation, or accounting.
- Migration adds a nullable column and two partial indexes on a sparse
predicate. Low risk.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 22:16:34 -06:00
DottaandClaude Opus 5.5 f6cb3e43f4 fix(admission): admit new work independent of other runs' cleanup and accounting (#19944) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Budget admission decides if new agent work can start in a company
> - Five admission gates waited on the cleanup or accounting of other
runs
> - The restart-chaos campaign found that these gates froze companies
for up to 10 minutes, caused false budget pauses, denied in-run writes,
and dropped wakes, comments and Retries
> - The durable ownership plan (Decision 1) removes the company-wide
accounting gate. It does not narrow it again
> - This pull request removes the five gates. Unfinished accounting now
adds its bounded estimate to exposure
> - The benefit is that one run's unsettled accounting never stops the
other work in its company

## Linked Issues or Issue Description

Refs #19831 (plan, section D), #19838 (inventory, finding 3), #19927
(Phase 2 reconciler data), #19497, #19724, #19826.

**What happened?**
Admission refused new work in these cases, with no reached hard stop and
no pause:
- a terminal run awaited accounting (`waitingRunCount`);
- a native run recovered its accounting (`recoveringRunCount`);
- a budget-owned scope hold awaited reconciliation, after the hard stop
no longer applied;
- a budget input was older than 15 s (`budget_evaluation_work`);
- the predecessor run row of a task was missing (`hasPendingRunCleanup`
failed closed).

**Expected behavior**
Admission never waits on the cleanup or accounting of another run. Only
a reached hard stop, an explicit pause or an inactive company refuses
new work (invariant 3).

**Steps to reproduce**
Make a company with an active budget policy. Leave one failed run with
`cost_accounting_pending = true`. Request work for another agent. On
`master` admission refuses with "Company cannot start work while
completed runs await accounting."

## What Changed

- `budgets.ts`: admission refuses only for a reached hard stop or an
explicit pause. The code removes `observedBlocks`, the
`recoveringRunCount` check, the stale `budgetHeld` refusal and the 15 s
input-age gate. It also removes the unused
`inspectInvocationEligibility`. `reconcileScope` no longer returns
accounting holds.
- `budget-exposure.ts`: removes the `waiting` and `recovering`
aggregates and the own-task subquery. The admission query is smaller.
Unfinished accounting stays exposure: the larger of the held reservation
and the observed cost.
- `budget-evaluation-work.ts`: removes `oldestBudgetInputAt`, which
nothing reads now.
- `execution-blocker.ts`: `hasPendingRunCleanup` returns false for a
missing run row. A run row that is deleted removes its finalization
coordinator and nulls its lease references, so the row owns nothing. A
predecessor that owns a lease or a coordinator still holds the successor
of its own task.
- `recovery/ownership-invariant-report.ts`: the I3 check no longer
mirrors the removed gate. It reports a wake request or a run that was
refused with an accounting-hold reason. Any such report is a regression.
- `doc/SPEC-implementation.md`: adds the product rule "Admission is
independent of cleanup and accounting". `doc/execution-semantics.md` is
aligned.

## Verification

- New `server/src/__tests__/admission-independent-of-cleanup.test.ts`,
on embedded Postgres. It has one case per gate. All six cases fail on
`master` and pass here. A seventh case (an explicit pause still refuses)
passes on both. One case also asserts invariant 3 together with
exposure: two failed runs that each hold a 400-cent reservation fill a
1000-cent budget, and a hard stop still refuses.
- Existing tests that asserted the old gates now assert admission
(invariants 3 and 4): `closed-native-receipt-accounting`,
`budgets-service`, `budget-evaluation-work`,
`heartbeat-held-comment-wake`, `ownership-reconciliation-report`. In
`heartbeat-held-comment-wake`, each saved comment, Retry, assignment and
continuation is delivered exactly once while another run's accounting is
pending. Settling accounting later does not deliver it again.
- All six files pass locally, one file per process. The server package
typecheck passes.
- Speed: admission is on the turn path. This change only removes work
from it: two correlated aggregates, one own-task subquery and one
`budget_evaluation_work` read per policy. No wait or I/O is added before
`turn.start`.
- Not run here: the chaos waves (K13 storm, K6 crash loop, K4 long
holds). The coordinating session schedules them on the shared harness.

## Risks

- Behavior change: work starts while accounting for other runs settles.
Spend stays bounded by the reservation capacity check and the hard stop,
which read the ledger plus observed cost. A run that has no reservation
and no observed cost adds zero exposure until its charge commits. This
is the gap that Decision 1 accepts. Budget enforcement was already best
effort (SPEC 11.6).
- A budget-owned pause with no reached hard stop no longer refuses work
in budget admission. The paused company or agent is still refused by the
existing pause checks until the budget worker lifts the pause.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 21:46:39 -06:00
DottaandClaude Opus 5.5 779913333a fix(documents): say whether an exact revision read is still current (#19939) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Task documents keep a revision history, and agents read and restore
revisions through the API
> - The exact-revision read returns one revision, but it does not say if
that revision is still current
> - An agent read the revision that it restored earlier by its ID and
reported it as "latest", but another person had saved a newer edit
> - This pull request adds the document's current revision to the
exact-revision read
> - The benefit is that a pinned read cannot pass as a read of the
current document

## Linked Issues or Issue Description

**What happened**

In a chat eval, the agent restored a plan revision. Then another person
saved a new revision. The user asked the agent to read the current plan.
The agent called `GET
/api/issues/{id}/documents/plan/revisions/{revisionId}` with the ID of
its own restored revision. The response did not show that a newer
revision existed. The agent reported the old revision as the latest and
said that the read confirmed this.

**Expected behavior**

A read of one exact revision must show if that revision is still
current, and which revision is current. Then the reader can see a newer
edit and read the current document.

**Steps to reproduce**

1. Save two revisions of a task plan.
2. Call `GET
/api/issues/{id}/documents/plan/revisions/{firstRevisionId}`.
3. See that the response gives no sign that revision 2 is the current
revision.

## What Changed

- `getIssueDocumentRevision` now also returns `latestRevisionId`,
`latestRevisionNumber` and `isLatest`.
- Shared type `ExactDocumentRevision` describes this response. The UI
client uses it.
- The OpenAPI description of the exact-revision read names these fields
and points to `GET /api/issues/{id}/documents/{key}` for the current
text.
- The service test checks `isLatest` for an older and a current
revision.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/documents-service.test.ts
src/__tests__/openapi-routes.test.ts` (42 passed).
- `npx tsc --noEmit -p packages/shared` passes.

## Risks

- Low risk. The change only adds fields to a read response. No caller
removes or renames a field. No migration.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 21:29:24 -06:00
DottaandClaude Opus 5.5 a4a3e342ec fix(recovery): charge a restart attempt only when the provider accepted a turn and made no progress (#19948) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A run has an execution recovery budget: three provider attempts for
native runs, and two failure retries for legacy runs
> - In the restart chaos campaign, control-plane restarts used up that
budget for runs that were healthy (RC-22, 22b, 22c, 40 and 60)
> - #19720 and #19747 fixed two variants with watermark heuristics, and
legacy interrupted preparation still spent the failure budget
> - The durable-ownership plan (section F, invariant 5) asks for one
rule: charge an attempt only when a recorded outcome proves that
provider work started and no progress followed
> - This pull request adds that rule and moves the native restart charge
and the legacy preparation retry onto it
> - The benefit is that a restart never spends budget for work the
provider never started, while real failure loops and crash loops stay
bounded

## Linked Issues or Issue Description

Refs #19720, #19747, #19856

**What happened?**
A control-plane restart charges a provider attempt by heuristics. The
first crash restart always charges. A later restart charges unless the
previous resume had no start evidence (#19747) or completed work
(#19720). A runner that spawned but whose provider never accepted a turn
counted as started, so a K6 crash loop could still spend the budget.
Legacy runs whose preparation a restart interrupted
(`providerWorkStarted: false`) spent `failureRetries`, the same counter
as real failures (`execution-recovery-attempt.ts`).

**Expected behavior**
An attempt is charged only when the provider accepted a turn
(`turn.accepted` or `turn.started`) and no accepted turn then completed
work. A startup before spawn, a runner that spawned but was never
admitted, and a `turn.start` the provider did not accept are not
charged. The generation-12 cap still bounds uncharged restarts.

**Steps to reproduce**
Start a native run at its last attempt. Kill the controller after the
runner spawns but before the provider accepts a turn. Before this
change, the next restart claim blocks the run with
`execution_recovery_budget_exhausted`.

## What Changed

- New `server/src/services/native-runtime/provider-attempt-outcome.ts`:
the one rule. `readProviderAttemptOutcome()` returns
`provider_not_started`, `provider_started_without_progress` or
`provider_progressed` from run events. `providerAttemptChargesBudget()`
charges only the second. `providerAttemptWindowStart()` starts the
attempt window at the latest `native.process_start_requested`, so
earlier work never counts for a later attempt.
`PROVIDER_ATTEMPT_CREDIT_GENERATION_LIMIT` (12) is the shared cap.
- `restartResumeKeepsAttempt()` (`native-restart-recovery.ts`) now
applies that rule to the attempt the restart interrupted, including the
first restart. It keeps the unstarted-resume carry from #19747, the
reattachment rule (an adopted provider counts as started), the drain
exemption from #19625, and the generation-12 cap. A history entry from
an older release without a recorded position still charges.
- Legacy runs: `interrupted_preparation` is now an uncharged retry lane
(`execution-recovery-attempt.ts`). It keeps its own counter, bounded at
the same cap (12), and never changes `failureRetries`. `retries.ts` uses
the longest transient delay after the second lane attempt.
- `doc/execution-semantics.md` states the rule.
- Accounting has its own "provider work started" readers (inventory
Summary finding 5). Phase 3 owns those; this PR exposes the rule for
them to adopt and does not change them.

## Verification

- `native-runner-restart-recovery.integration.test.ts` (embedded
Postgres, real `claimNativeRestartRecoveries`): 129 passed. New cases,
red on master:
- not charged at the last attempt for a claimed remote bootstrap, a
remote bootstrap retry, and a claimed unlaunched remote run;
- not charged when the runner spawned but was never admitted, and when
the provider did not accept `turn.start`;
- not charged on the first restart when the original attempt already
made progress;
- a normal retry that spawned but was never admitted is not charged, and
earlier work still does not count for it.
- Counter-cases, green on master and with the change: an accepted
`turn.accepted` or `turn.started` with no progress is charged; the
generation cap blocks at the last attempt; crash loops with and without
progress stay bounded; the #19720 and #19747 cases still pass.
- `execution-recovery-attempt.test.ts`: new lane tests (red on master).
- `workspace-recovery-preparation-dispatch.test.ts` (25 passed),
`heartbeat-process-recovery.test.ts` (interrupted-run cases),
`run-dispatch/adapters/postgres.test.ts` (interrupted cases).
- `server` typecheck (`scripts/typecheck-layers.mjs`, low-memory mode):
no errors.
- Not run: chaos waves (the coordinator schedules them on this host),
and the full server suite (CI covers it).

## Risks

- Behavior change: the first crash restart no longer charges when no
accepted turn is recorded. If a crash lands before the controller
projects the provider's `turn.accepted`, that attempt is not charged.
This fails toward more retries, and the generation-12 cap bounds it.
- A legacy preparation crash loop now gets up to 12 retries instead of
2. Each retry still passes the normal issue gates and keeps the
transient delay.
- The new queries use the `(run_id, seq)` index, once per restart claim.
Nothing new runs before `turn.start`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test execution). Extended reasoning
enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 21:20:05 -06:00
DottaandClaude Opus 5.5 c6e4ed1547 feat(recovery): settle a claimed wake whose run finished without settling it (#19934) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A wake request is claimed by the run that handles it, and that run
marks it finished.
> - In a K4 chaos wave (#19927), native restart recovery blocked a run
and ended it, but left its assignment wake `claimed`. Nothing ever
released it.
> - The durable-ownership plan settles such records on one shared path,
by proof, never by age.
> - This pull request registers a settle for `wake_request_claimed`. It
is Phase 2's second kind and is stacked on the accepted-result lease PR.
> - The benefit is that no run-ending path can strand a claimed wake.

## Linked Issues or Issue Description

Refs #19831 (plan), #19861 (registry), #19927 (the data that found
this). Stacked on #19933 (it adds the `settle` hook).

**What happened?**
Run `7c45860b…` failed with `native_restart_recovery_blocked`
(`execution_recovery_budget_exhausted`) at 23:19:41. Its wake
`6b3c4cce…` stayed `claimed` until the end of the wave (23:44 and
later). Rule I2 flagged it.

**What did you expect?**
A claimed wake ends when its run ends, with the run's outcome.

**Steps to reproduce**
1. `node cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts
--env local --faults K4 --k4-hold-max-seconds 600 --tasks 6
--concurrency 3 --seed 9203 --keep`
2. Look for `agent_wakeup_requests` rows with status `claimed` whose run
is terminal.

## What Changed

- New `recovery/wake-request-claimed-settlement.ts`: when the owner run
is terminal, and for a native run its coordinator is `committed` or
`terminal_failure`, the claim gets the run's outcome. That is
`completed` for a succeeded run and otherwise the run's status, as run
completion and native terminal projection already do. It is
compare-and-swap on the claim (id, run, status `claimed`).
- `ownership-registry.ts`: `wake_request_claimed` registers that settle.
- `native-finalization-reconciler.ts`: the pass log line includes the
settled count per kind.

## Verification

- `cd server && npx vitest run
src/__tests__/ownership-reconciliation-report.test.ts`: 17 pass,
including the new case. It covers a blocked native run, a native run
whose projection is still pending (kept), a succeeded legacy run, a live
run (kept), and a read-only pass (no change).
- Red/green (test written after the code): the new test fails on master
and passes here.
- Server typecheck passes.

## Risks

- No persisted shape changes, so there is no previous-release story
beyond this: claims saved by an older release are settled the same way.
- Race with the run's own settle: it waits for a native coordinator to
be final. For a legacy run, both write the same status, so the race is
idempotent.
- No change adds work before `turn.start`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 21:18:22 -06:00
DottaandClaude Opus 5.5 38d8ef89ca fix(chat): link a person's own profile editor instead of attempting an agent write (#19943) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - People talk to their agents in chat. The chat guidance tells the
agent which links to give back.
> - A person can ask the agent to change their own display name. Only
the person can do this, in their profile settings.
> - The chat guidance did not say where that page is. The agent searched
the API and tried the human-only profile endpoint. The product correctly
refused it. The reply then gave only vague "profile/account settings"
directions.
> - This pull request tells chat where the personal profile editor is,
and that agent credentials cannot change a person's profile.
> - The benefit is a reply with a link that the person can open to make
the change.

## Linked Issues or Issue Description

**What happened**
In chat, a person wrote "Change my display name to Alex Chen." The agent
spent about four minutes on API searches. It tried `PATCH
/api/auth/profile`, and the product refused it. The reply told the
person to use their "profile/account settings" but gave no link.

**Expected behavior**
The agent does not try to change the person's profile. It links the
person's profile editor at
`/<prefix>/company/settings/instance/profile`, so the person can change
the name.

**Steps to reproduce**
1. Sign in as a person in an authenticated deployment.
2. Open chat with the CEO agent.
3. Send "Change my display name to Alex Chen."
4. See that the reply has no link to the profile editor.

## What Changed

- `server/src/services/agent-conversations.ts`: the chat directive now
says that a person's own name and avatar change only in their profile
settings at `/<prefix>/company/settings/instance/profile`. It says that
agent credentials cannot edit a person's profile, and that renaming an
agent, member record or company does not change it. The agent links that
page instead of trying the change.
- `server/src/__tests__/agent-conversations.test.ts`: a regression test
checks the directive and the built chat prompt.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/agent-conversations.test.ts -t "chat prompt policy"`: 10
passed.
- The chat eval case `navigation-preferences.edit-profile` checks this
end to end. It requires a link to the profile editor, and the person
follows that link to save the new name.

## Risks

- Low risk. This is a guidance change only. It adds no permission and
changes no route.
- Deployments that hide the Profile settings page get a link to a hidden
page. The page gate still applies.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 21:03:35 -06:00
DottaandClaude Opus 5.5 47dc3dd9d0 fix(chat): ask which record is meant when a change matches several (#19942) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - People use the agent chat to change records, for example task
priority
> - A request can name one record when several accessible records match
it
> - The chat guidance tells the agent to ask focused questions, but it
does not say what to do when a reference matches several records
> - In an eval run, the agent spent more than two minutes. It considered
picking one task, changing both tasks, and asking. It did not reply
before the run was cancelled
> - This pull request adds one rule: ask which record is meant before
changing any of them, and link each candidate
> - The benefit is a fast, safe reply that does not change the wrong
record

## Linked Issues or Issue Description

**What happened**

The person wrote "Mark the landing-page task as high priority." Two
accessible tasks have the title "Landing page", in the projects Launch
and Experiments. Nothing in the chat selected one. The agent searched,
read both tasks, listed projects and searched the API. Its reasoning
wavered between "ask", "change both" and "pick one". After about 130
seconds it had not replied, and the run was cancelled.

**Expected behavior**

The agent asks which task the person means. The question links both
tasks, names what makes them different, and keeps the requested change.
The agent changes no task until the person answers. Then it changes only
the selected task.

**Steps to reproduce**

1. Create two tasks with the same title in two different projects.
2. In the agent chat, ask to set "the <title> task" to high priority.
3. See that the guidance gives no rule for several matches, and the
agent can deliberate, guess, or change both.

Refs: chat-feature eval `tasks.ambiguous-task`.

## What Changed

- `server/src/services/agent-conversations.ts`: `AGENT_CHAT_DIRECTIVE`
now says: when a change request matches several accessible records and
nothing selects one, ask which one before changing any. Link each
candidate, name the difference, and restate the change. Do not pick the
first match, change every match, or keep searching for a hidden answer.
- `server/src/__tests__/agent-conversations.test.ts`: a regression test
checks the rule in the directive and in the built chat prompt.

## Verification

- `cd server && npx vitest run src/__tests__/agent-conversations.test.ts
-t "chat prompt policy"` passes (10 tests).
- End-to-end: the chat-feature eval `tasks.ambiguous-task` runs with
this commit on an eval branch. The result is in a comment below.

## Risks

- Low risk. This is a prompt-guidance change only. No API, schema or
permission changes.
- The agent can ask in more cases. The rule applies only when several
records match and nothing in the conversation selects one.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 21:00:57 -06:00
DottaandClaude Opus 5.5 f5400e2594 feat(recovery): settle the accepted-result lease on the shared ownership path (#19933) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A control-plane restart can kill the controller after an agent's
result is accepted. The result then stays leased to a dead process.
> - Today a private sweep (`native-dead-controller-result-lease.ts`,
#19713) proves that controller dead with its own liveness rules.
> - The durable-ownership plan (#19831) replaces private proofs with one
prover (`proveOwnerGone`, #19853) and one registry (#19861).
> - This pull request moves the accepted-result executor lease to that
shared path and deletes the private sweep. It is Phase 2's first kind.
> - The benefit is one tested rule for when a dead owner's result lease
is released, and the reconciler now repairs a lease that restart
recovery missed.

## Linked Issues or Issue Description

Refs #19831 (plan), #19853 (prover), #19861 (registry), #19713 (old
sweep), #19883 (legacy host fallback), #19927 (Phase 2 data).

**What happened?**
The accepted-result lease had its own liveness proof, separate from the
shared prover. The reconciler could only report a dead owner, not settle
it. When restart recovery missed a lease, it waited out the 20-minute
lease.

**What did you expect?**
One prover decides, and both restart recovery and the periodic
reconciler pass settle a lease whose controller is proven gone.

**Steps to reproduce**
1. Accept a native run result.
2. Kill the controller (SIGKILL) and start a new server.
3. If the restart sweep does not see the row, the result stays leased
until the lease expires.

## What Changed

- `ownership-registry.ts`: a kind may register `settle` and
`onUnprovable`. `native_accepted_result_lease` registers both. Other
kinds stay report-only.
- New `native-accepted-result-lease-settlement.ts`: a compare-and-swap
settle that expires exactly the proven owner's lease. It keeps the old
guards: the run must still be `running`, remote copyback stays fenced,
and a lease taken over since the proof is left alone.
- `ownership-invariant-report.ts`: the pass calls `settle` on `gone` and
`onUnprovable` on `unprovable`. A read-only pass (`record: false`) never
settles. New `settleOwnableKind()` for restart recovery.
- `heartbeat/recovery.ts`: restart recovery calls
`settleOwnableKind("native_accepted_result_lease")` instead of the old
sweep.
- Deleted `native-dead-controller-result-lease.ts`.
- `owner-proof.ts`: accepts a board attestation (`operator_attested`).
It also keeps the one-release local fallback for rows saved before their
PID space was recorded, on single-host servers only, and flags each use
so it is logged (`legacy_host_unknown_dead_pid_proof`, #19883). An
ephemeral server returns `unprovable` (`process_space_unrecorded`) and
the existing "confirm the previous server stopped" board action is
raised.

## Verification

- `cd server && npx vitest run
src/__tests__/heartbeat-native-restart-suspension.test.ts`: 27 pass, 7
of them new.
- `npx vitest run src/services/owner-proof.test.ts
src/__tests__/ownership-reconciliation-report.test.ts`: pass.
- Red/green (tests written after the code): 5 new behaviour tests fail
on master and pass here. They cover the ephemeral board action, the
reconciler-pass settle, the live-owner counter-case in that pass, the
legacy marker, and operator attestation. The stamped-dead-controller and
foreign-PID-space cases pass on both, as the old path behaved the same.
- Server typecheck passes (`npx tsc --noEmit -p server/tsconfig.json`).
- Not run locally: the full server suite (`pnpm verify` asks for broad
coverage because a file was deleted). Required CI covers it.

## Risks

- Previous-release story (rules/upgrade-safe-run-state.mdc): NRF rows
saved before #19868 have no `controller_host_hash`. They keep the same
one-release local proof as before on single-host servers, and each use
is logged. On ephemeral servers they stay unprovable and ask the board,
as before. Rows on another PID space keep their lease (the fence), as
before.
- Fail-open risk: settle is compare-and-swap on the lease token, boot,
PID, host hash and start time of the proven owner. An `unprovable` owner
is never released. Live-owner and foreign-space cases are tested.
- No change adds work before `turn.start`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 20:58:04 -06:00
DottaandClaude Opus 5.5 18644a3263 feat(recovery): instrument 20 durable write boundaries with crash points (#19938) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A control-plane restart between two durable writes leaves an
in-progress record owned by a dead process
> - #19937 added `crashPoint(name)` and a registry of 20 named
boundaries from the Phase 0 inventory
> - A name without a call site cannot fire, so the crash-point suite
cannot test it
> - This pull request puts one call at each registered boundary and adds
a test that keeps the registry and the call sites in step
> - The benefit is that the on-demand suite (next pull request) can kill
the control plane at every known write-ahead gap

## Linked Issues or Issue Description

Refs #19831 (plan, Phase 5). Refs #19846 (inventory, "Write-ahead
coverage"). Stacked on #19937 (helper and registry). Refs #19861
(ownable kind ids).

**Problem**
The registered crash points have no call sites.

**Expected behavior**
Each registered point is called at its gap: after the first durable
write or outside effect returns, before the next one starts. Disarmed
(always in production) each call is one comparison.

**Scope**
Call sites and a coverage test only. The suite runner and CI workflow
follow.

## What Changed

- Server call sites (`@paperclipai/shared/crash-points`):
- `sandbox-creation-operations.ts`: around the provider `create` call
(intent → provider → lease).
- `native-session-executor.ts`: after `native.process_start_requested`,
before spawn.
- `paperclip-control-plane-port.ts`: after the result-accept
transaction, before finalization starts (only when a result was
inserted).
- `native-run-finalizer.ts`: after `commitNativeStatusDecision`, before
the run projection.
- `workspace-operations.ts`: `workspace_finalize` only — after the
running row, and after `settle()` before the done row.
- `environment-runtime.ts`: provider release → lease row; teardown claim
→ destroy; reusable destroy → lease release; stop-and-retain claim →
stop.
- `heartbeat.ts`: lease release → runtime services; budget reservation
hold → dispatch.
- `heartbeat/queue.ts`: run claimed `running` → wake request `claimed`.
- `run-cost-accounting.ts`, `usage-receipts.ts`: accounting commit →
spool unlink; receipt spool → DB row → unlink.
- `packages/paperclip-runner/src/crash-point.ts`: a small copy of the
helper for controller code in this package, which does not depend on
`@paperclipai/shared` (adding that dependency would change the runner
content ID). It arms only `runner.*` names. runnerd never sees the
variable: its environment is an allowlist.
- Runner call sites: after the runnerd spawn, before its identity is
published (both spawn paths); `turn.start` result received → controller
journal saved → next send.
- `server/src/__tests__/crash-point-call-sites.test.ts`: every
registered point has a call site, every call names a registered point,
and every `kinds` entry is an ownable kind id of the Phase 1 registry.

## Verification

- `cd server && npx vitest run
src/__tests__/crash-point-call-sites.test.ts` → 3 passed.
- `pnpm --filter @paperclipai/server typecheck` → clean. `pnpm --filter
@paperclipai/paperclip-runner exec tsc --noEmit -p .` → clean.
- Behavior with nothing armed is unchanged: each call returns at its
first comparison (measured in #19937: about 3.7 ns).

## Risks

- Low risk. No call changes control flow unless `PAPERCLIP_CRASH_POINT`
names it and `NODE_ENV` is not `production`.
- Points on the path before `turn.start` (process start, runner spawn)
cost nanoseconds, within plan invariant 8.
- `usage-receipts.ts`: the persist-then-unlink one-liner became a block
with the same statements in the same order.
- The runner package change gives the runner a new content ID, as any
runner source change does. No persisted run state changes
(rules/upgrade-safe-run-state.mdc).

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits) and a read-only exploration subagent that
mapped the boundaries.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 20:56:33 -06:00
DottaandClaude Opus 5.5 468f22ae4b fix(server): document the company export request body in the API catalog (#19945) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A CEO agent can export the company setup through `POST
/api/companies/{companyId}/exports`
> - Agents find API operations and their request bodies in the OpenAPI
catalog
> - The catalog entries for `/exports` and `/exports/preview` have no
request body
> - The export defaults include only the company and agents, so an agent
that sends no body gets a package without projects or skills
> - This pull request registers the existing export body schema on both
entries and describes the `include` defaults
> - The benefit is that agents can request a complete reusable setup and
do not tell users that resources do not exist

## Linked Issues or Issue Description

**What happened**

A user asked a CEO agent in chat: "Export this company's setup so I can
reuse it." The company had an agent, a project, and a company skill. The
agent looked up `POST /api/companies/{companyId}/exports` in the API
catalog. Its reasoning said that the operation "has no request body
defined". It sent no body. The package had only the company and the
agent. The agent then told the user that the company had "no projects,
issues, or company skills, so nothing else was left out." That statement
was false.

**Expected behavior**

The catalog documents the export body, including `include.projects` and
`include.skills`. An agent can request a full setup export. The legacy
`/export` route already documents this body.

**Steps to reproduce**

1. Create a company with a project and a company skill.
2. Read `GET /api/openapi.json`,
`paths["/api/companies/{companyId}/exports"].post`.
3. See that it has no `requestBody`. The legacy `/export` entry has one.
4. Call `POST /exports` with no body. The package has no projects or
skills.

## What Changed

- Register `companyPortabilityExportSchema` as the request body of `POST
/api/companies/{companyId}/exports` and `/exports/preview` in
`server/src/routes/openapi.ts`.
- Add a description that states the `include` defaults and tells callers
to set `include.projects` and `include.skills` for a reusable setup.
- Add a regression test in
`server/src/__tests__/openapi-routes.test.ts`.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/openapi-routes.test.ts` passes (36 tests). The new test
fails on master because the entries have no `requestBody`.
- An end-to-end check runs the `companies.export-company` chat eval with
this commit cherry-picked onto the eval branch.

## Risks

- Low risk. This changes only API documentation. The routes already
parse this schema. Runtime behavior does not change.
- `imports/preview` and `imports/apply` also register no body. This PR
does not change them.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 20:55:44 -06:00
DottaandClaude Opus 5.5 2a2d3f6e23 fix(ui): stop refetching the @task mention pool on every task event (#19931) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI keeps its caches fresh from live events in
`LiveUpdatesProvider`
> - Each task activity event invalidates the whole `["issues",
companyId]` query prefix
> - That prefix also holds the `@task` mention pool: the 100 most
recently updated tasks, about 315 KB per response
> - A chat turn emits many comment, status and run events, so an open
task page downloads that list again for almost every event
> - This pull request keeps the pool fresh from push events without a
refetch per event: a closed server flag marks real membership or title
changes, progress on pooled tasks reorders the cache in place, and other
progress refetches at most once per 30 seconds
> - The benefit is less bandwidth and CPU for users with an open task or
chat, and smaller eval traces

## Linked Issues or Issue Description

**What happened**
While a task chat page is open, the browser repeats `GET
/api/companies/:id/issues?limit=100&sortField=updated&sortDir=desc`
(about 315 KB each). One 10-minute chat-eval attempt recorded about 194
of these requests. The trigger is `invalidateActivityQueries` in
`ui/src/context/LiveUpdatesProvider.tsx`. For every `entityType:
"issue"` activity event, it invalidates
`queryKeys.issues.list(companyId)`. That prefix matches
`queryKeys.issues.mentionPool(companyId)`, which `IssueDetail` (and
`NewIssueDialog`) keep active for the `@task` picker. Invalidation
refetches active queries and ignores `staleTime`.

**Expected behavior**
Comments, run progress and status changes on any task do not download
the full mention pool again. The pool still updates when a task that the
picker can show is created, deleted or renamed.

**Steps to reproduce**
1. Run `pnpm dev` and create a company with about 140 tasks.
2. Open a task page, for example `/REF/issues/REF-1`.
3. Every 4 seconds for 15 rounds, post comments on the open task, on a
task in the top 100, and on a task outside the top 100, and change the
open task's priority (60 events).
4. Count requests to `issues?limit=100&sortField=updated`. Before this
change there are 15 (about 3.3 MB). After this change there are 2 (about
480 KB).

## What Changed

- New shared helper `issueActivityChangesTaskIdentity`
(`packages/shared/src/issue-activity-identity.ts`). It is true for a
created, deleted, hidden, unhidden or renamed task.
- `server/src/realtime/live-events-ws.ts`: task `activity.logged` events
now carry the closed boolean `taskIdentityChanged`. Authenticated
sockets still get no `details`; the flag carries no task content.
- `ui/src/context/LiveUpdatesProvider.tsx` refreshes the mention pool
this way:
- The task identity flag is true (or full details show a create, delete,
hide, unhide or rename): the pool refetches at once.
- Other events on a task already in the pool: the task moves to the
front of the cached list. There is no request. The freshness clock does
not change, and a pending invalidation is never overwritten.
- Other events on a task outside the pool: one trailing refetch at most
every 30 seconds, because the task can now be in the top 100.
  - The picker is closed: the pool is only marked stale.
- `ui/src/lib/query-invalidation-batcher.ts`: new `excludingQueryKeys()`
keyed exclusion predicate. The batcher coalesces it and uses it for
prefix coverage. The broad `["issues", companyId]` invalidation uses it
to skip the pool, so a burst is again one broad refresh per batch.
- Tests: `ui/src/context/LiveUpdatesProvider.mention-pool.test.ts`
(authenticated event shape, hide and unhide, the 30-second window,
reorder safety, a burst through the batcher), new batcher tests,
`packages/shared/src/issue-activity-identity.test.ts`, and socket
assertions in `server/src/__tests__/live-events-toast.test.ts`.

## Verification

- `cd ui && npx vitest run src/context/LiveUpdatesProvider
src/lib/query-invalidation-batcher`: 231 tests pass.
- `cd server && npx vitest run src/__tests__/live-events-toast.test.ts`:
11 tests pass. In authenticated mode the socket keeps `details` off and
sends `taskIdentityChanged` (false for routine updates, true for a
create, rename or hide).
- `cd packages/shared && npx vitest run
src/issue-activity-identity.test.ts`: 14 tests pass.
- The new UI tests fail against the first commit of this PR: the
authenticated burst, hide, unhide, the 30-second window and the batcher
burst.
- `tsc --noEmit` for ui, server and shared: no new errors.
- Browser measurement: a local `pnpm dev` instance (local trusted),
headless Chromium on a task page, 141 tasks, 60 events over about 63
seconds (the reproduction steps above):
  - Before (master UI): **15** mention-pool requests, about 3.3 MB.
  - After: **2** requests, about 480 KB (one per 30-second window).
- Freshness after the fix: one rename, hide, unhide or create each
caused **1** refetch. A priority change caused **0**.
- I did not run a browser session in authenticated mode. The
authenticated event shape is covered by the socket test and the UI unit
tests.

## Risks

- A task that moves into the top 100 can be missing from the `@task`
picker for up to 30 seconds.
- The cached order is approximate between refetches: a moved task goes
to the front, but its `updatedAt` value is not changed.
- Other queries under `["issues", companyId]` (descendants,
created-from, summaries, onboarding first task) still refresh for each
event batch. They are small, but they can get the same treatment later.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 20:55:39 -06:00
DottaandClaude Opus 5.5 2fb4837cca fix(chat): tell chat how to link execution runs and name their agents (#19889) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Chat lets a person ask an agent about work, such as the runs behind
a task
> - The chat guidance gives a link template for tasks only. It does not
give one for execution runs
> - The issue runs API returns `agentId` and the run `id`, but no agent
name and no link
> - So when a person asks "Show me the runs behind this website task",
the agent lists raw UUIDs. It does not link each run. It does not name
the agent that did the work
> - This pull request adds the registered run route to the chat link
guidance. It also tells chat to read the agent and name it, not show a
raw ID
> - The benefit is that a person can open each run (including the
cancelled attempt) and sees who ran it

## Linked Issues or Issue Description

**What happened?**
A chat agent was asked to show the runs behind a task that had a
cancelled attempt and a successful retry. It read `GET
/api/issues/{id}/runs` correctly, but its reply showed both runs as raw
UUIDs in code spans and named the agent as "`fe03dd15…`, the Website
Brief worker" instead of its name. It gave no run links.

**What did you expect?**
Each run links to its run detail page
(`/<prefix>/agents/<agentId>/runs/<runId>`), and the reply names the
agent that executed it.

**Steps to reproduce**
1. Create a task assigned to an agent, cancel its first run, and retry
it so a second run succeeds.
2. In chat, ask "Show me the runs behind this website task."
3. The reply lists run UUIDs without links and identifies the agent by
ID.

Refs #19614 (the chat catalog eval case `activity.find-runs` shows this
failure).

## What Changed

- `server/src/services/agent-conversations.ts`: the chat link guidance
now gives the run route `/<prefix>/agents/<agentId>/runs/<runId>` and
tells chat to read and name the agent instead of showing a raw ID.
- `server/src/__tests__/agent-conversations.test.ts`: a regression test
checks that the run template matches the registered UI route and reaches
the chat task prompt.

## Verification

- `cd server && npx vitest run src/__tests__/agent-conversations.test.ts
-t "chat prompt policy"`: 7 passed. The new test fails without the
guidance change.
- `src/services/heartbeat/task-markdown.test.ts` passes.
- The embedded-Postgres suite in `agent-conversations.test.ts` did not
start in my local sandbox (Postgres init error). It does not depend on
this prompt text. CI runs it.
- End-to-end: the chat catalog case `activity.find-runs` is rerun with
this commit on its eval branch.

## Risks

- Low risk. This is prompt guidance text only. No API, schema, or UI
change.
- It conflicts textually with #17985, which edits the same guidance
line. The merge is simple: keep both sentences.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use (shell, file edits). Default reasoning mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 18:41:35 -06:00
DottaandClaude Opus 5.5 7fd98682ca fix(workspaces): give every subtask its own execution workspace (#19860) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents split work into child tasks, and each task runs in an
execution workspace (a checkout, branch, or sandbox folder)
> - Since early 2026, a new child task used its parent's realized
workspace by default: the same checkout, branch and sandbox
> - A parent and its child often run at the same time. Two writers in
one folder lose each other's files, the waking parent is cancelled with
`workspace_busy`, and sandbox lease ownership crosses from the parent
into the child, so retained-file recovery refuses the workspace as
ambiguous
> - The product rule is simple: a subtask gets its own workspace,
independent of the parent's saved files. Paperclip never needs to carry
a parent's unmerged changes into a child
> - This pull request makes that the only behaviour on every creation
path, and moves existing shared children to their own workspace on their
next run, with no user or admin action
> - The benefit is that parents and children never block or overwrite
each other, and recovery ownership stays with one task

## Linked Issues or Issue Description

**What happened?**
A child task inherited its parent's realized execution workspace by
default (`createChild` defaulted to `executionWorkspaceInheritanceMode:
"linkage"`, and `issueService.create` fell back from
`inheritExecutionWorkspaceFromIssueId` to `parentId`). When the parent
woke while the child wrote to the shared workspace, the parent run was
cancelled with `workspace_busy`. In sandbox environments, the lease
ownership chain crossed from the parent into the child, and
retained-file recovery refused the workspace with
`allocation_ownership_ambiguous`.

**Expected behavior**
Every new child task gets its own execution workspace, realized from the
project base exactly as a top-level task in that project. It never uses
the parent's checkout, branch, saved snapshot or unmerged changes. A
parent and its descendant never share a workspace by any route.

**Steps to reproduce**
1. Run a parent task in a project with isolated (git worktree) or
sandbox workspaces, so the parent has a realized execution workspace.
2. From the parent run, create a child task with only `parentId` (or
through the runner `create_task` tool, chat suggestions, decision
effects, plugins, or the New sub-task dialog).
3. See that the child's `executionWorkspaceId` is the parent's, with
`executionWorkspacePreference: "reuse_existing"`.
4. Wake the parent while the child runs in a sandbox environment. The
parent run is cancelled with `workspace_busy`.

**Additional context**
Refs #9233 (the existing `strategy_only` opt-out).
The opt-out `executionWorkspaceInheritanceMode: "strategy_only"` already
existed for accepted-plan decomposition. This change makes that
behaviour (tightened to drop a pinned existing branch) the only
behaviour.

## What Changed

- **One rule for every creation path
(`server/src/services/issues.ts`).** `issueService.create` still uses
the parent (or `inheritExecutionWorkspaceFromIssueId`) as a source for
the project, project workspace and workspace *strategy*, but it never
copies the source's `executionWorkspaceId`, `reuse_existing` preference,
or pinned existing branch. All server paths that create a task go
through `create()`/`createChild()`: issue routes (`POST /issues`, `POST
/issues/:id/children`, the run-bound default parent and run workspace
source), runner `create_task`, chat suggestion acceptance, native status
decisions, accepted-plan decomposition, decision effects, plugins
(`ctx.issues.create`), external agents, watchdog follow-ups, monitor
recovery issues, case handoffs and email channels.
- **Explicit ancestor workspaces are dropped.** If a create names an
`executionWorkspaceId` (or an `existing` workspace selection) that an
ancestor of the new task created or is bound to, `create()` drops it and
keeps only the strategy. Older clients and skills that copied the
parent's id keep working and get an independent workspace.
- **`createChild` no longer has a "linkage" mode.**
`executionWorkspaceInheritanceMode` is still accepted from internal
callers and is ignored.
- **`inheritExecutionWorkspaceFromIssueId` decision.** I kept the field
for API compatibility, but it now copies only the source issue's project
and workspace strategy. I removed its workspace-sharing behaviour for
every pair, not only parent/child: the source task and its follow-up can
run at the same time (the runner, watchdog and status-decision callers
all create the follow-up while the source run is live), so sharing would
again let two tasks write one folder. Explicit sharing between unrelated
tasks stays available through an explicit `existing` workspace
selection.
- **New `server/src/services/task-workspace-lineage.ts`.**
`buildIndependentTaskWorkspaceSettings` (strategy only: mode,
environment, runtime, network egress, shared-workspace concurrency, base
ref, branch template, commands; no `reuse_existing`, no
`operator_branch`, no `existingBranch`), `loadTaskLineageIssueIds` (the
parent chain plus, for a task that depth placement put beside its
requester, the requester's chain via `privacyParentIssueId`) and
`isAncestorExecutionWorkspace` (the workspace's `sourceIssueId` is in
that lineage, or a lineage issue is bound to it). A child in another
project does not take the parent's strategy.
- **Upgrade-safe rebind at admission.**
`executionWorkspaceService.releaseAncestorWorkspaceBindingAtAdmission`
runs at the same admission point as the pending workspace selection
(`heartbeat.ts`). For a child (or a flattened task beside its requester)
that an older release bound to an ancestor's workspace, it clears the
child's binding (keeps the project workspace and strategy, bumps the
binding revision) so the run realizes the child's own workspace. It does
nothing while another run of the child is running, while the child's
earlier run still owns workspace finalization or recovery, for native
restart/lease-owner recovery, for a process-loss retry, or for a Retry
that keeps the current workspace. The parent and its workspace are never
changed.
- **Projects without a project workspace (per-task repository
acquisition).** A parent with no project workspace worked in its own
task files, so its settings (for example `isolated_workspace`) do not
describe the project base. A child of such a parent, and an older child
released from one, takes the project's defaults exactly like a top-level
task, instead of an isolated mode that would resolve to a git worktree
with no git base ("needs a project / project workspace"). When the
child's new workspace is created at admission,
`executionWorkspaceRepositoryService.prepareForAdmission` repeats the
parent's (or requester's) repository requests as the child's own pending
requests (identity, URL, catalog id and requested ref only; no pinned
commit, branch or files). The normal acquisition path then clones a
fresh copy into the child's task files with the child's own access
checks. This runs only for a workspace created by that admission, so
existing tasks never gain repositories.
- **UI.** The New sub-task dialog no longer pre-selects the parent's
workspace. `buildSubIssueDefaults` drops `executionWorkspaceId` and
`reuse_existing`, and derives the parent's independent mode
(`shared_workspace`, `isolated_workspace` or `agent_default`) from its
preference, saved settings or current workspace. The "This sub-task will
no longer use the parent task workspace" warning is removed.
- **Contracts and docs.** Shared validator and plugin SDK comments
describe the new meaning of `inheritExecutionWorkspaceFromIssueId`.
Updated `doc/execution-semantics.md` (new rule in §6 and a wording fix
in the retained-sandbox paragraph), `doc/SPEC-implementation.md` (child
workspace rule, requester keeps *strategy* inheritance, watchdog
follow-ups), `doc/SPEC.md` (task workspace ownership),
`docs/specs/external-task-protocol.md`, `skills/paperclip/SKILL.md`
(step 9 and the workspace bullet),
`skills/paperclip/references/issue-workspaces.md`, the CEO onboarding
`HEARTBEAT.md`, and the plugin SDK README.

### Previous-release story (`rules/upgrade-safe-run-state.mdc`)

- **Migrated on next admission.** A child that the previous release
bound to its parent's (or any ancestor's) execution workspace keeps that
binding until its next normal admission. Then the server clears the
binding and the run realizes the child's own workspace from the project
base. No schema migration is needed. No user or admin action is needed.
- **In-flight writers finish first.** A running child is not
interrupted. Its current run, its native restart recovery (admitted
native input or lease owner), a process-loss retry, and a Retry that
keeps the current workspace all finish in the shared workspace. While a
child's earlier run is still running or its workspace finalization or
recovery is unsettled, the binding stays and a later admission releases
it.
- **No files are dropped.** The release changes only the child's binding
row. The parent keeps its binding, and the shared workspace row, folder,
branch and saved snapshots are not touched. Anything the child left
there stays in the parent's workspace.
- **No new refusal codes.** Saved run identity (native execution input,
leases, checkpoints) is not changed or refused; the rebind only affects
new admissions.

## Verification

- New `server/src/__tests__/subtask-workspace-independence.test.ts`
(embedded Postgres): `create` with `parentId`, `createChild` with no
mode / `"linkage"` / `"strategy_only"`,
`inheritExecutionWorkspaceFromIssueId` pointing at the parent, explicit
parent and grandparent `executionWorkspaceId`, a non-child follow-up,
and unchanged top-level defaults and explicit top-level reuse. 7
creation cases fail on `master` and pass with this change. Admission
cases: an old-release child is rebound while its parent is running and
the parent keeps its workspace; a running child is not rebound (deploy
lands mid-run); an unrelated explicit share is kept; the parent is never
rebound.
- New heartbeat case in
`server/src/__tests__/heartbeat-workspace-busy.test.ts`: an old-release
child bound to its running parent's workspace in a sandbox environment
runs to `succeeded` with no `workspace_busy`, gets a new workspace, and
the parent run and binding stay unchanged. It fails on `master` (the
child waits with `workspace_busy`).
- Updated tests that asserted the old inheritance in
`issues-service.test.ts`, `task-workspace-binding.test.ts`,
`chat-project-tools.test.ts`, `task-workspace-child-trust.test.ts`,
`heartbeat-git-restore-point-resume.test.ts` (a child bound to a closed
parent's expired workspace now starts from the project base and leaves
the parent's restore point alone), and
`ui/src/lib/subIssueDefaults.test.ts`. Remote-workspace reservation,
continuation and writer-wait suites now seed the older-release shared
shape directly (or an unrelated explicit share), so those safety paths
stay covered.
- `npx vitest run` on the changed suites and about 45 related server
suites (heartbeat workspace, recovery admission, issue routes, routines,
plugins), `ui` `subIssueDefaults` and `NewIssueDialog` tests, `pnpm
check:token-gates`, `pnpm -r typecheck`.

- New
`server/src/__tests__/heartbeat-subtask-acquired-repository.test.ts`: a
project with no project workspace, a parent whose task files hold an
acquired repository with an unmerged change, and a child (new, and the
older-release shape bound to the parent's workspace) that runs to
`succeeded` in its own task files with a fresh clone at the project base
and its own ready repository receipt. The parent's checkout keeps its
change. Both cases fail on the previous head with "needs a project /
project workspace".

## Risks

- **Project-level configuration still applies to children exactly as to
top-level tasks.** In a project configured for `shared_workspace`, every
task (top-level or child) uses the project's one folder under the
existing shared-workspace concurrency policy. A branch that the project
itself pins applies to every task in the project. This PR removes only
the parent-to-child carry-over of the realized workspace; it does not
change those project defaults.
- **Standalone follow-ups that an older release bound through
`inheritExecutionWorkspaceFromIssueId`** are migrated only when the
source is in their recorded lineage (parent or privacy parent). Others
keep their explicit share and the existing writer-wait serialization.

- **Behaviour change for agents that relied on a shared checkout.** A
child or follow-up no longer sees the parent's unpushed or uncommitted
changes. Agents must push or merge code they want a subtask to build on.
The skill and onboarding text now say this.
- **Explicit ancestor workspace ids are silently dropped** instead of
rejected, so older clients do not break. The task gets an independent
workspace.
- **First run of a migrated child starts from the project base.** If an
older child had uncommitted work in the parent's workspace, that work
stays in the parent's workspace and is not copied into the child's new
workspace.
- **Admission adds one indexed read** for tasks with a parent and a
bound workspace; the locking transaction and ancestor query run only
when the bound workspace was created by another issue.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (file edits, shell, test execution) and
extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 18:34:09 -06:00
DottaandClaude Opus 5.5 feb8dacb86 fix(skills): teach the board skill how people find things in the web UI (#19891) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People use chat with a board agent. The board agent reads the
`paperclip-board` skill.
> - People also ask the agent how to use the web UI. An example is "How
can I quickly find Launch tasks without leaving what I am doing?"
> - The skill does not describe the web UI search. The agent searched
for docs and found none. Then it said that it did not know the UI and
told the person to ask in chat.
> - The correct answer is quick search: Cmd+K or Ctrl+K opens the
command palette over the current page.
> - This pull request adds a short "Finding Things in the Web UI"
section to the board skill.
> - The benefit is that the agent gives the real entry point and
shortcut, and does not claim that it can operate the person's browser.

## Linked Issues or Issue Description

No issue exists. This is the problem description (bug template labels):

**What happened?**
A person asked the board chat agent how to quickly find Launch tasks
without leaving their page. The agent did not mention quick search or
Cmd/Ctrl+K. Its reasoning said that it did not know the exact UI
capabilities. It told the person to ask in chat.

**Steps to reproduce**
1. Open board chat in a company that has a "Launch" project.
2. Ask "How can I quickly find Launch tasks without leaving what I am
doing?"
3. Read the reply. It does not name the quick-search shortcut.

**Expected behavior**
The agent names quick search (Cmd+K on macOS, Ctrl+K on Windows/Linux,
or the sidebar search icon), operators such as `status:backlog`, and
"Search all" for full search. It tells the person to do these steps. It
does not claim to press keys for them.

Found by the chat-feature eval `navigation-preferences.quick-search`
(runner-opencode cell).

## What Changed

- `skills/paperclip-board/SKILL.md`: add a "Finding Things in the Web
UI" section. It describes the Cmd/Ctrl+K command palette, the sidebar
search icon, search operators, the "Search all" and Cmd/Ctrl+Enter
handoff to `/{prefix}/search`, and permission-scoped results. It tells
the agent that it cannot press keys or focus fields in the person's
browser.
- `skills/paperclip-board/SKILL.md`: add "help finding things in the
Paperclip web UI" to the skill description, so agents select the skill
for UI how-to questions.
- `server/src/__tests__/board-skill-ui-guidance.test.ts`: new test. It
checks that the documented shortcuts match the key bindings in
`ui/src/components/CommandPalette.tsx`, that the full-search route
exists, and that the description makes the section discoverable.

## Verification

- `pnpm exec vitest run
server/src/__tests__/board-skill-ui-guidance.test.ts
server/src/__tests__/cli-invocation-safety.test.ts
server/src/__tests__/agent-skills-routes.test.ts
server/src/__tests__/onboarding-seed-route.test.ts
server/src/__tests__/board-claim.test.ts
cli/src/__tests__/managed-agent.test.ts` passes.
- The new test fails on `master`, because the section does not exist
there.
- End-to-end: the change is cherry-picked onto the eval branch
`chat-eval/case-quick-search`, and the
`navigation-preferences.quick-search` chat eval runs on it.

## Risks

- Low risk. This is a documentation change to one skill and one new
test.
- If the palette shortcut or route changes, the new test fails. Then
update the skill and the test together.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use (shell, file edits, GitHub CLI).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 18:33:28 -06:00
Nicky Leach 35cbd15175 fix(native): recover a stopped Runner whose journal stayed ready at a settled cut (#19907) 2026-11-07 16:01:47 -08:00
Nicky Leach 91b044858c fix(runner): send the prepared task scope with retained release and resume (#19896) 2026-11-07 14:50:31 -08:00
Nicky Leach 5020f38f90 fix(runner): give a later attempt a new task when its stopped task kept no session (#19899) 2026-11-07 14:49:37 -08:00
Nicky Leach 0363ebec70 fix(connectors): stop GitHub access refreshes from advancing agent runtimes (#19902) 2026-11-07 14:43:39 -08:00
Dotta eb7c7b083a Recover Muse verification after contact, restart and access restoration (#19909) 
## Thinking Path

> - Paperclip manages AI agents and their work under company access
rules.
> - Muse readiness uses a saved connection, receiver contact and an
authenticated challenge reply.
> - A failed challenge write could leave a saved contact with no later
recovery trigger.
> - A server restart or restored access must recover that contact under
current authority.
> - This pull request uses the existing delivery coordinator and saved
Muse state for recovery.
> - It keeps the normal receiver schedule, authorization checks and real
deadlines.
> - It lets eligible verification recover and lets idle verification
stop work.

## Linked Issues or Issue Description

**What happened?**

The first Muse receiver contact could persist while readiness challenge
creation failed. The contact then had no durable retry or startup
recovery path. Later membership, lifecycle or feature restoration could
also leave pending verification without a notice.

**Expected behavior**

Retry a temporary failure through the existing worker. Recover pending
contact after restart. Recheck the current company, agent, operator,
credential and binding generation before issuing a challenge. Stop
verification work while it is ineligible or complete. Do not revive
revoked credentials.

**Steps to reproduce**

1. Use the normal invitation and OAuth registration, PKCE and token
flow.
2. Send one receiver contact and let its normal persistence schedule
run.
3. Fail the first readiness challenge operation before it commits.
4. Observe whether the existing worker retries without another contact
or a manual Verify action.
5. Restart the App with a new database owner. Check that it creates one
usable challenge from the saved contact.

**Paperclip version or commit**

The causal baseline test used the unchanged production implementation at
`90148317b2a447794ff29a4acae5648ad11a9770` plus its normal fixture. The
accepted positive test used `9d383041028776ad8469ac4af3fe14ba6c369425`.
The initial published head was
`fe518c9b72504daf6374225987f24d45bc7bcf75`. The lease/auth correction
was locally qualified at `cfd515a207416097fe8b586277d70c4fd4a679fa`. The
current candidate is `6efe722e7eb1e22acd466125c7b4a627611d0273`; its
added Cloud actor fixture correction passed the full normal owning
program and all 35 tests. Its current normal build, lifecycle owner
controls, healthy-wait and restoration measurements are described below.
Earlier results below are bound to their stated revisions.

Related public changes: Refs #19819 for original-principal
authentication. Refs #19750 for the transaction-owned external feature
read. These are related foundations, not duplicate fixes. The current
Muse title and head-branch searches found no duplicate
contact-verification recovery PR. Related maintenance work is #17000 and
terminal assignment settlement is #17793.

## What Changed

- Add a typed, exact-binding verification notice through the existing
Muse delivery queue.
- Register the first-contact notice before its durable write in the same
transaction.
- Recover pending contact from saved state at startup. Retry temporary
failures through the existing scheduler.
- Recheck current feature, lifecycle, credential and access state after
commit. Park unavailable authority without recurring verification scans.
- Add narrowly scoped notices to the existing access, lifecycle,
configuration, company, budget and settings write owners.
- Save a healthy lifecycle verification wait in the existing operation
JSONB. Keep generic unresolved-work and error retries unchanged.
- Commit authenticated confirmation and its lifecycle recheck in one
transaction. Retain a raced recheck under the active lease, and recover
saved waits once at startup.
- Retain a committed lifecycle recheck while an expired attempt still
owns the row, so its late wait cannot park accepted confirmation.
Replacement claims keep their existing owner fence and fresh fact read.
- Keep the receiver gap, persistence and per-segment contact limits
unchanged.
- Add real App OAuth, failure, retry, restart, challenge and quiet
evidence. Add two normal-route restoration tests.
- Document recovery, reconnect requirements and the existing
connected-receiver sleep restriction.

## Verification

- Causal baseline: the real invitation and OAuth flow saved one contact.
The first challenge operation failed. The normal test failed because it
observed one verification attempt instead of a retry. No setup or
cleanup failure was counted as this regression.
- At `9d383041`: normal own Shared/SDK, DB with migration 0441 assets,
Runner TypeScript core and surfaces, module boundaries, and all seven
normal App programs passed. One new redundant role comparison first
failed typecheck; that diagnostic remains recorded and the corrected
full App gate passed.
- At `9d383041`: all 17 maintenance component tests passed with zero
skips. All original deadline, rollback, drain, terminal settlement and
startup controls remain.
- At `9d383041`: the one normal App case passed. Real OAuth authorized
one receiver signal. A retry occurred about 5.01 seconds after the first
failure. A new Db/App owner created one challenge, advanced revision 2
to 3, and accepted authenticated confirmation and mailbox ACK. Query
instrumentation had an active positive control. All measured query,
callback, session and verifier counters stayed equal across the
unchanged scheduled 15-second quiet window. Token revocation and
old-token/signal rejection passed.
- Known App, worker, HTTP and database owners closed without cleanup
errors. The tests used bounded local PostgreSQL and HTTP fixtures. They
did not call a live Muse model.
- Current integrated semantics: the full normal App driver completed all
programs. Only a new test callback had two undefined-value diagnostics.
An explicit defined-value guard corrected that test. The full 416-root
owning program then passed with empty diagnostics. A separate collector
incorrectly required an explicit child `noEmit` field; the normal
inherited setting was verified without repeating the compiler. The
original failures remain recorded.
- The first restoration run failed both cases. Permission parking
recorded 120 main-pool queries without more verifier calls. The
lifecycle counter recorded two denials without their exact scope. These
observations remain unexplained at the individual-query/call level.
Source separately shows a lifecycle readiness retry every two seconds
while the real external reply is absent.
- At `fe518c9b`: normal Shared/SDK and DB producers, module policy, all
seven full App programs, and all 33 pure diagnostic controls passed. The
lifecycle owner file passed all 19 cases, including both
confirmation/park orders, rollback, duplicate notices, startup recovery
and authority controls.
- At `fe518c9b`: the normal OAuth failure/retry/new-Db reconstruction
case passed. Its new healthy-wait window measured 15,002 ms with
unchanged total activity before genuine confirmation; the existing ready
quiet and revocation checks also passed.
- At `fe518c9b`: both normal restoration tests passed in Vitest.
Ordinary viewer-to-operator restoration and Pause/Resume recovered the
same binding, grant and single contact; each committed one challenge and
advanced revision 2 to 3. All six parked, restored and revoked/disabled
quiet windows measured at least 15 seconds with active positive read
controls and unchanged total queries, callbacks, sessions and verifier
calls. Both cleanups passed.
- The restoration wrapper still recorded failure: it required lifetime
lifecycle denial count zero, whereas the measured count was one before
the parked quiet window. The reviewed fixture explicitly permits a
current-authority denial from an earlier observation and requires no
committed challenge before restoration. Precise denial timing relative
to Pause was not serialized. The original wrapper failure is retained;
no rerun or retrospective explanation of the earlier failures is
claimed.
- Review exposed an expired-lease race. The added real-store baseline
test failed after the late defer restored `waitingForSignal`; no setup
or timeout was counted as the regression. On `cfd515a`, the whole
lifecycle owner file passed all 20 tests, including that case. The full
normal production declaration program (1067 roots), verification program
(662), and auth owning program (405) passed; all 1942 declarations
remained byte-identical.
- The original remote auth-session shard failed three tests because
their partial query adapters lacked the new binding query's
join/order/limit chain. A fixture-only correction preserved the real
notifier and existing authorization assertions. All 12 auth-session
tests passed locally.
- Complete remote inspection also found 33 failures in the 35-test Cloud
actor fixture with the same missing `innerJoin` method and downstream
incomplete-auth assertions. Its original full log and exact synthetic
Source comparison are retained. At `6efe722e`, the three-line fixture
adapter repair passed the complete normal 662-root test program and the
whole 35-test file with zero failures or skips. All existing lock,
authorization, settings, debounce and unfinished-auth assertions remain.
- The earlier healthy/restoration quiet measurements remain bound to
`fe518c9b`. Those steady 15-second windows did not exercise a
still-owned attempt passing its 120-second lease. The targeted
expired-owner regression qualifies that new boundary; it is not a fresh
measurement of the earlier windows.
- The initial `fe518c9b` remote gates failed the auth fixtures and
reported a Greptile 4/5 P1. The aggregate failure reported the
underlying shard. Those actual causes were fixed and qualified above.
- Current `6efe722e` remote gates completed with **57 successful checks
and three explicit skips**. The full latest edited Greptile summary is
**5/5**, explicitly reviews this exact head, and has no new finding. The
original P1 thread is resolved. All general, inline and nested review
pages were read.
- One current-head CI attempt timed out after 30 seconds in the Runner
evidence/privacy file, with 6,864 passes and one failure. That file and
its transitive inputs are byte-identical to the CI base. A separate
local diagnostic preserved every assertion and the original timeout, ran
the full normal Runner test project and the entire 41-case file, and
passed all 41 with zero skips. Its private bounded timing records
measured the target body at 74 ms. The diagnostic was then removed,
restoring the exact original test. This is local recovery evidence; the
original CI stalled await was not recorded and its cause remains
**unproven**.
- One retry of only the failed jobs on the unchanged head passed the
Runner support suite and dependent aggregate verification. The complete
second attempt and original failure logs are preserved. No test,
timeout, privacy guard or product code was weakened. This result does
not turn the original timeout into an explained failure or a diagnosed
flake.
- Full recursive typecheck, full `test:run` and a full release build
were not repeated locally. The normal full App programs, focused
producers and tests above passed, and the required current-head remote
gates passed. Final review is against the exact unchanged head; local
quiet remains separate from deployed suspension.

Reviewer commands use the normal project entry points: `node
server/scripts/typecheck-layers.mjs`, the whole
`server/src/services/muse-maintenance.test.ts` file, and both normal App
PostgreSQL files. PostgreSQL fixtures require explicit owned
fixture/admin URLs and normal migrations before imports. Run the files
serially with one worker. The private controller retains exact Source,
output, worker and owner-closure receipts.

## Risks

- Missing or early producer notices could lose a restoration trigger.
Notices stay in the transaction that commits the write. The worker
checks current authority after commit.
- A stale notice must not grant access. Binding generation, current
membership, company, agent and credential checks remain authoritative.
Revoked credentials require reconnecting.
- The change adds no table, migration, alternate task queue or periodic
verification scan. It adds two optional fields to existing lifecycle
operation JSONB. Generic pending/error deadlines, setup holds and manual
repair remain obligations.
- Confirmation can race with an active verification attempt. The park
compare-and-set must retain the durable recheck. Startup recovery uses a
fixed cutoff and keyset pages; it must not become another recurring
scan. Wrong company, stale version, pause and revoked credentials must
stay fenced.
- The connected receiver still has its documented tenant-sleep
restriction. Worker quiet is not proof of deployed suspension or
external wake.
- The normal challenge reply is sent by an authenticated protocol
fixture. Live Muse background replies, cost/usage, full Muse lifecycle
and full hosted Dot behavior remain unqualified. Partial Dot profile ACK
and unsigned-device cutover stay withheld.
- Restoration coverage for the additional company, budget, configuration
and feature producers remains a separate qualification obligation. This
pull request does not mark the whole hosted area complete.

## Model Used

OpenAI `gpt-6.1-sol` authored the implementation and tests with local
code and tool use. A GPT-6-based Codex coordinator reviewed the changes
and owns integration. Its exact underlying model ID is not exposed.
Context window sizes are unavailable. No context size is inferred.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
 2026-11-07 16:41:15 -06:00
DottaandClaude Opus 5.5 adb7633d7b fix(status-cards): give agents the person's restore route when archiving a card (#19903) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Status cards are saved summaries of company work. People and agents
can archive and restore them.
> - A person asked a chat agent to archive a card "to reuse it next
quarter". The agent archived it correctly.
> - The archive response had no board route, and the PATCH operation had
no description. The completion guidance tells agents not to guess
routes.
> - So the agent told the person to restore the card "by clearing
`archived`". That is an API instruction, not a route a person can use.
> - This pull request gives single-card responses an `href` to the
Status page and documents archive and restore on the PATCH operation.
> - The benefit is that an agent can give the person a usable link and a
true account of restore, which starts a fresh summary.

## Linked Issues or Issue Description

**What happened**
A chat agent archived a status card. Its reply told the person to
restore the card by clearing `archived` on the card. It gave no link to
the place where a person restores it.

**Expected behavior**
The reply links the card on the Status page, where a person can show
archived cards and click Restore. It also says that restore starts a
fresh summary.

**Steps to reproduce**
1. Enable status cards and create a card as an agent.
2. Ask the CEO agent in chat to archive the card for later reuse.
3. Read the reply. It has no usable restore route, because `PATCH
/api/status-cards/{id}` returns no `href` and has no description.

## What Changed

- `POST /api/companies/{companyId}/status-cards`, `GET
/api/status-cards/{id}` and `PATCH /api/status-cards/{id}` now return
`href`, the company-scoped Status page route for the card.
- The shared `statusCardSchema` has an optional `href` field.
- The `PATCH /api/status-cards/{id}` OpenAPI operation has a
description. It says that archive keeps the card and its history, that a
person restores it on the Status page with the returned `href`, and that
restore starts a fresh full summary.
- The create route reads the company issue prefix before it creates the
card. A failed link lookup can not delete a card whose compile task is
queued.
- Regression tests cover the `href` on the archive and get responses,
the new PATCH description, and a failed prefix read on create (no card
and no task are left).

## Verification

- `pnpm exec vitest run server/src/__tests__/status-cards.test.ts
server/src/__tests__/openapi-routes.test.ts` (71 tests pass). The new
create-route test fails on the first commit and passes on the fix.
- End to end: the chat eval `status-cards.archive-card` runs with this
change on an eval branch. The result is in the eval report for that
case.

## Risks

- Low risk. The change adds one optional response field and one
description. List responses do not change. The UI ignores the new field.
- Each single-card response does one more small company read for the
issue prefix. On create, this read occurs before the card exists, so its
failure leaves nothing behind.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used in Claude Code
with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 16:28:11 -06:00
DottaandClaude Opus 5.5 0f19eb39d3 fix(chat): link the profile of an agent chat acted on (#19893) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent Chat lets a person ask the CEO agent to act on other agents,
for example to resume a paused agent
> - The chat guidance tells the agent how to link tasks, but it does not
tell the agent how to link an agent profile
> - In a chat eval, the CEO resumed Morgan and reported the result, but
the reply had no link to Morgan
> - The person then has no direct route to check the saved availability
of the agent
> - This pull request adds one guidance sentence for agent profile links
> - The benefit is that a reply about an agent gives the person a link
to that agent

## Linked Issues or Issue Description

**What happened?**
A person asked the chat agent: "Resume Morgan so they can pick up their
work." The agent resumed Morgan through the API and reported the new
status. The reply did not link Morgan's profile. The chat guidance gives
only the task-link template (`/<prefix>/issues/<identifier>`), so the
agent had no route for an agent.

**Expected behavior**
When a reply is about a specific agent, the reply links that agent's
profile at `/<prefix>/agents/<urlKey or id>`. The link is a plain
profile link. It is not an `agent://` mention, because a mention
notifies the agent.

**Steps to reproduce**
1. Pause an agent named Morgan.
2. In Agent Chat with the CEO, send "Resume Morgan so they can pick up
their work."
3. Read the reply. It reports the resume but has no link to Morgan.

Refs #19889 (adds the run-link template in the same guidance; this
change merges cleanly with it).

## What Changed

- `server/src/services/agent-conversations.ts`: add one sentence to the
chat directive. It tells chat to link an affected agent's profile at
`/<prefix>/agents/<urlKey or id>`, and to use `agent://` only to notify
an agent.
- `server/src/__tests__/agent-conversations.test.ts`: add a regression
test that checks the directive and the built task prompt contain the
profile-link guidance.

## Verification

- `cd server && pnpm exec vitest run
src/__tests__/agent-conversations.test.ts -t "chat prompt policy"`
passes (7 tests).
- The new test fails on `origin/master`, because the directive does not
contain the sentence.
- The route exists in `ui/src/App.tsx` (`agents/:agentId`), and the UI
accepts a url key or an id.
- End to end: the `agents.resume-agent` chat eval runs with this change
on its case branch.

## Risks

- Low risk. The change adds prompt text only. No API, schema, or UI
behavior changes.
- The prompt becomes one sentence longer.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with
tool use (shell, file edit), extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 16:28:00 -06:00
DottaandClaude Opus 5.5 29b6411660 fix(chat): give chat replies the approval route (#19892) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent Chat lets a person ask their agent about company state, such
as "What decisions are waiting on me?"
> - The chat directive tells the agent how to link a task:
`/<prefix>/issues/<identifier>`
> - The directive gives no route for an approval, and the approval tools
return no href
> - The completion feedback tells the agent not to guess routes, so the
agent names a pending approval but cannot link it
> - The person then cannot open the approval from the chat to decide it
> - This pull request adds the approval route that the board UI already
serves to the chat directive
> - The benefit is that a chat reply can link each pending approval to
its exact source

## Linked Issues or Issue Description

**What happened?**
In the Agent Chat eval `inbox-search.show-decisions`, the person asked
"What decisions are waiting on me?". The agent found the pending hire
approval and listed it. It did not link it. Its recorded reasoning said
that it had no known href for approvals and that it must not invent
URLs. Task sources in the same reply had working links.

**Expected behavior**
The reply links the pending approval, so the person can open it and
decide it.

**Steps to reproduce**
1. Create a pending `hire_agent` approval in a company.
2. In Agent Chat, ask "What decisions are waiting on me?".
3. See that the reply names the approval but gives no link to it.

**Paperclip version or commit**
`master` at the base of this branch.

**Agent adapter(s) involved**
Not adapter-specific (core bug). The chat directive is shared by all
runners.

## What Changed

- `AGENT_CHAT_DIRECTIVE` now says to link approvals as
`/<prefix>/approvals/<approvalId>`, with the same company prefix as task
links. The board UI serves this route (`approvals/:approvalId` in
`ui/src/App.tsx`).
- Added a regression test that the directive gives routes for both tasks
and approvals and keeps the "never invent a hostname" rule.

## Verification

- `pnpm exec vitest run
server/src/__tests__/agent-conversations.test.ts` passes (41 tests). The
new test fails without the directive change.
- End to end: the change is cherry-picked onto the eval branch
`chat-eval/case-show-decisions` and run in the Runner Full-Stack E2E
workflow for
`chat-features.runner-opencode.local.inbox-search.show-decisions`.

## Risks

- Low risk. The change adds one sentence to the chat prompt. It does not
change routes, permissions or tool results.
- The prompt becomes slightly longer for every chat turn.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use (shell, file edits, GitHub CLI).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 16:27:47 -06:00
Devin FoleyandPaperclip ee556a266d perf(storage): upload workspace snapshots in parallel verified parts (#19917) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - With remote (sandbox) execution, every agent turn ends by saving the
workspace: capture an archive, encrypt it, upload it to object storage,
and verify it.
> - On a busy S3-backed instance, the upload step is the slowest part of
a save. A 1.5 GB workspace can hold a run open for minutes after the
agent finished.
> - The upload step reads the archive several times, uploads through one
stream with 4 parts in flight, and then downloads the whole object again
to verify it.
> - Parts that use the stream path share one process-wide 32 MiB buffer
window, so saves that run at the same time slow each other down.
> - This pull request uploads parts in parallel straight from the
encrypted file, lets the backend verify every part with Content-MD5, and
replaces the full read-back with a size check for those verified
uploads.
> - The benefit is faster saves, a smaller run-finalization window, and
no interference between concurrent saves.

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Remote workspace snapshot upload (`uploadWorkspaceSnapshot`) to
S3-compatible storage.

**Subsystem affected**
Server storage (`server/src/storage/*`) and remote workspace snapshot
storage.

**Current behavior**
For one save, the upload step:
1. hashes the archive;
2. reads the archive again to encrypt it into a temporary file;
3. hashes the encrypted file;
4. uploads it as one stream with 4 parts in flight;
5. downloads the whole object to verify it.

On one hosted instance over 7 days (724 saves, median 731 MB), the
upload step ran at a median of 6.5 MB/s. With no other save running it
reached 11–14 MB/s. With six or more saves at the same time it fell to
4.4 MB/s. From the same region and with the same part settings, a plain
client reached 31 MB/s to the same kind of bucket.

**Proposed behavior**
Upload parts in parallel from byte ranges of the encrypted file, with
Content-MD5 on each part. Confirm only the stored size after a verified
upload. Digest the archive in the same pass that encrypts it.

**Reason and benefit**
Saves finish faster and hold their run open for less time. Concurrent
saves no longer serialize on the shared buffer window. A transient part
failure is retried from disk instead of failing the whole save.

**Breaking changes**
None. `StorageProvider.putObject` may now return a receipt. Providers
that return nothing keep the current behavior, including the full
read-back.

Related: none found.

## What Changed

- `storage/types.ts`, `storage/service.ts`: an upload can name its
`sourceFile`, a private file whose bytes are exactly the body.
`putObject` may return `{ integrity: "server_verified_md5" }`, and
`putFile` passes it through as `PutFileResult.integrity`.
- `storage/s3-multipart.ts`: new `putS3MultipartFile`.
- Each part reads its own byte range of the file. Up to 8 parts upload
at once, with no shared buffer window.
  - Each part carries the Content-MD5 of its range.
  - A failed part is resent from disk, up to 3 attempts.
- The first part that runs out of attempts stops its siblings. The
multipart upload is aborted only after every part settles.
- A source file whose size differs from the declared length is refused
before any request.
- `storage/s3-provider.ts`:
- A `sourceFile` larger than 16 MiB uses `putS3MultipartFile`. A smaller
one uses one PUT with Content-MD5.
- These requests use a second client with `requestChecksumCalculation:
"WHEN_REQUIRED"`, so bodies are plain and the backend verifies
Content-MD5 directly. The default client wraps bodies in aws-chunked
framing with a CRC32 trailer.
  - Other uploads are unchanged.
- `services/remote-workspace-snapshot-storage.ts`:
- Encryption digests the plaintext in the same read. An unchanged file
identity (dev, inode, size, mtime, ctime) across that read replaces the
separate digest pass. A change still fails with `archive_changed` before
any upload.
- The encrypted file is still digested from disk, so the recorded hash
covers the exact bytes that the parts read.
- The upload passes `sourceFile`. When the provider reports
`server_verified_md5`, a HEAD confirms the stored size instead of
downloading the object.
  - The full read-back stays for:
    - other providers (local disk);
    - recovery of earlier attempts;
    - the caller that keeps a verified archive file.

## Verification

- `pnpm --filter @paperclipai/server typecheck`: exit 0.
- `vitest run src/__tests__/storage-s3-multipart.test.ts
src/services/remote-workspace-snapshot-storage.test.ts`: 59 passed. New
tests:
- every part carries the Content-MD5 of its range, and 8 parts are in
flight;
  - a failed part is resent and the upload completes;
- a part that keeps failing aborts the upload only after in-flight
siblings settle;
  - a size mismatch is refused before any request;
- a small file goes as one PUT with Content-MD5, through a real HTTP
server;
- the read-back is skipped only for a server-verified upload, and the
snapshot still restores;
  - the read-back stays when the caller keeps the read-back file;
  - a stored size mismatch is rejected;
  - an archive that changes during encryption is rejected before upload.
- With the source change reverted, the three new snapshot tests fail.
- Real S3-compatible bucket (a Railway bucket, run from a workstation):
- A 3 MiB single PUT stored exactly: HEAD size, ETag and read-back
SHA-256 all matched.
  - A 96 MiB, 13-part upload stored exactly, with the same checks.
  - A part with a wrong Content-MD5 was rejected with `BadDigest`.
- The same backend stored a wrong `x-amz-checksum-sha256` without
checking it. That is why this change uses Content-MD5.
- Measured from a container in the app's region, 1 GiB with the app's
settings (8 MiB parts, 4 in flight), to the same kind of bucket: 31
MB/s. With 16 in flight: 69 MB/s.
- 77 storage-related suites: the only failures are the same 19 that fail
on unmodified `master` in this checkout. They are in 4 files: missing
bundled connector skills and an unbuilt local runner daemon.

## Risks

- Integrity now depends on the backend enforcing Content-MD5 for file
uploads. AWS S3 and the tested S3-compatible backend enforce it. A
backend that ignores Content-MD5 would accept corrupted parts without a
read-back. A later restore still fails closed: AES-GCM authentication
and the plaintext SHA-256 are checked on load.
- Up to 8 parallel part requests per save, instead of 4 shared ones.
Memory per part is a 64 KiB stream buffer.
- Archive-change detection now uses file identity instead of a second
digest. The archive lives in a private temporary directory that the
capture step creates.
- Medium risk overall: the change is limited to file-sourced S3 uploads
and the snapshot verify step.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs, S3 API probes). AI-assisted
investigation, change and tests, with human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
docs describe this path)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-07 14:04:49 -08:00
DottaandClaude Opus 5.5 fc6e1b0346 perf(native): keep local Codex and OpenCode runners warm by default (#19888) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - When someone messages an agent, the turn must start fast. Time to
first token (TTFT) is a first-class metric.
> - The Phase 0 baseline (#19846) found that every local turn launches a
new runner and loads the provider session again, because `per_turn` is
the default lifecycle. That made a follow-up message slower than a new
task.
> - Warm runners keep the runner and provider session alive between
runs. #19874 fixed the defect that broke warm OpenCode follow-ups, and
#19879 bounds an idle warm runner whose controller dies.
> - This pull request makes warm the default for local Codex and
OpenCode agents, and migrates existing agents that only carried the old
creation default.
> - The benefit: follow-up TTFT p50 drops from 12.0 s to 4.7 s (Codex)
and from 21.7 s to 4.2 s (OpenCode), with no new restart-chaos failures.

## Linked Issues or Issue Description

Refs #19846 (TTFT baseline), #19852 (startup sub-spans), #19874 (warm
completion contract), #19879 (warm reconnect grace).

**What existing behavior does this improve?**
The default runner lifecycle for native agents, and how a follow-up
message starts.

**Current behavior**
Agent creation writes `lifecycleMode: "per_turn"` into every native
agent. Every follow-up launches a new runner and restores the provider
session: 4.2 s (Codex) and 16.3 s (OpenCode) at p50 before the turn
starts.

**Proposed behavior**
New native agents store no lifecycle. On a local target, an unset
lifecycle resolves to `warm` with the default 5-minute idle window for
Codex and OpenCode. Other harnesses, remote targets and managed
providers without warm support stay `per_turn`. An explicit choice
always wins. Migration 0443 clears the stored creation default from
existing agents, except where an edit set "Turn by turn" explicitly.

**Reason and benefit**
Measured locally (`doc/plans/2026-11-07-warm-follow-up-ttft.md`). Warm
follow-ups are 2.6–5× faster for Codex and OpenCode, and restart
recovery latency is equal or lower.

## What Changed

- `native-lifecycle.ts`: an unset `agentMode` resolves to warm (5-minute
idle window) for local `codex_app_server` and `opencode_server` agents.
`runtime-selection.ts` passes `localTarget`. Explicit `per_turn` or
`warm` wins. An admitted session keeps its persisted policy. ACPX Claude
stays `per_turn`: it relaunches its provider process on every follow-up
by design, so warm did not shorten its follow-up (10.7 s warm vs 10.5 s
per_turn).
- Creation and configuration writes no longer store `lifecycleMode:
"per_turn"` for Codex, Claude or OpenCode. This covers
`normalizePaperclipRunnerAdapterConfig` (runs on every create and
update), `paperclipRunnerTransitionConfig` (switching to the native
runner), `agent-runner-selection.ts`, `codex-local` `build-config.ts`,
and the simplified new-agent setup (`paperclipRunnerCreationLifecycle`
in `@paperclipai/shared`). Grok, Pi and Cursor keep their explicit
value.
- Agent form: the Runner lifecycle field shows the unset state as
"Automatic". "Turn by turn" and "Warm session" remain.
- Migration `0443_native_warm_follow_up_default.sql`: clears
`lifecycleMode = 'per_turn'` from native Codex, OpenCode and Claude
agents unless `agent_config_revisions` shows an edit that set it to
`per_turn`. Each migrated agent gets an `agent.updated` activity entry
with the old value (`actor_id =
migration:native-warm-follow-up-default`). Creation records no revision,
so a "Turn by turn" pick made at creation cannot be told apart from the
old default. The product owner approved migrating those agents. Claude
agents follow the default, which is `per_turn` today.
- Restart chaos harness and TTFT driver: `--lifecycle
per_turn|warm|default` (saved in the wave plan). `default` removes the
catalog's pinned lifecycle, so a wave exercises the real default path.
- Docs: `doc/DEVELOPING.md` "Runner lifecycle default", and the results
in `doc/plans/2026-11-07-warm-follow-up-ttft.md`, which also documents
baseline finding 4 (Claude's resumed turn-start gap is ACPX's per-turn
process generation, not the provider).

## Verification

- `server`: `native-lifecycle.test.ts` (new default cases: local Codex
and OpenCode warm, Claude, remote, unknown target, unmeasured harness,
managed Claude, admitted per-turn session, explicit choices, non-native
adapter), `agent-runner-selection.test.ts`, `runtime-selection.test.ts`
(335 pass).
- `packages/db`: `native-warm-follow-up-default-migration.test.ts` on
embedded Postgres. Default agents are cleared, the explicit edit is
kept, other harnesses and legacy adapters are untouched, audit rows are
written once, and a rerun is a no-op. Without the explicit-edit filter
the test fails. `check:migrations` passes.
- `codex-local` `build-config.test.ts` (65), `ui`
`config-fields.test.tsx` (40), restart-chaos unit tests (134), `tsc
--noEmit` for `server`.
- `adapter-utils` `paperclip-runner-lifecycle-default.test.ts`:
configuration writes and adapter conversion store no lifecycle for
Codex, OpenCode and Claude, keep `per_turn` for the others, and preserve
explicit choices. It fails without the fix.
- End-to-end default check on this branch (`ttft-baseline.ts --lifecycle
default`, codex and opencode, 2 samples): the agents read back with no
stored lifecycle, and every follow-up reused the runner and provider
session (no `runner.process.launch`). Follow-up TTFT was 3.6–4.8 s. This
check found the per-write `per_turn` injection fixed here.
- Restart chaos (codex and opencode, A1/B1/B5, K1/K2/K6, seed 4242), 8/8
PASS each with no open signatures: forced warm, forced per_turn, and
`--lifecycle default` on this branch (agents created without a
lifecycle).
- `pnpm verify` selects broader CI (harness config is a global input),
so required CI supplies the broad coverage.

## Risks

- Behavior change: local Codex and OpenCode agents keep a runner and
provider process for up to 5 minutes after a run. Instance sleep waits
for that idle window. The process is reaped by the idle timer, by
graceful shutdown, or by the #19879 reconnect grace if its controller
dies.
- Migration: agents whose "Turn by turn" was chosen at creation become
Automatic. The activity entry records the old value; choosing "Turn by
turn" restores it.
- Previous-release story (`rules/upgrade-safe-run-state.mdc`): a task
session keeps the lifecycle policy it was admitted with, so sessions
saved before this change stay `per_turn`. New sessions get the default.
No run, checkpoint or launch state changes shape.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use (shell, file edit, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 14:37:36 -06:00
DottaandPaperclip d7f02c51b4 fix(skills): scope file protection to admitted selections (#19904) 
Allow unused skill deletion when unstarted admission entries have no retained execution. Limit unknown legacy selections to skills that could have existed when the validated immutable run finished.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
 2026-11-07 14:14:59 -06:00
DottaandClaude Opus 5.5 988a4b6749 feat(recovery): ownable-kind registry and report-only ownership reconciler (#19861) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A control-plane restart leaves in-progress records owned by dead
processes, and about 31 per-subsystem proofs decide when to release them
> - The durable ownership plan (section C) asks for one registry of
ownable record kinds and one reconciler
> - Phase 1 must measure first: how many in-progress records exist, who
owns them, and which invariants they break, with no behavior change
> - This pull request registers 46 kinds from the Phase 0 inventory's
section C and adds a report-only pass to the existing native
finalization reconciler
> - The benefit is per-kind invariant-violation counts from chaos waves
and staging. That data unlocks Phase 2 (moving kinds onto one settle
path) and the removal of the company-wide admission gate

## Linked Issues or Issue Description

Refs #19831 (plan, section C and Phase 1). Refs #19846 (Phase 0
inventory, section C). Stacked on #19853 (the prover) and #19851 (one
process identity).

**Problem**
No single place lists the control plane's in-progress records or checks
the plan's invariants. Each subsystem releases its own records by its
own rules, so nobody can count how many records break "owned", "settled"
or "visible next action".

**Expected behavior**
One report-only pass lists every registered kind, classifies each owner
as gone, alive or unprovable, and records each invariant violation once.
It changes no record and stays quiet when nothing is in progress.

**Impact**
Phase 2 moves kinds onto the shared settle path in the order these
counts show. Phase 3 removes the admission gate once the I3 data exists.

## What Changed

- New `server/src/services/recovery/ownership-registry.ts`: 46
`OwnableKind` entries (`id`, `obligation`, `storage`, `listInProgress`,
`settle: null`, `invariantChecks`). Each record names its owner as a
process stamp, a fence token, a run, an operator, or none. The
skill-cache lock is registered with a named coverage gap because its
lock files live in per-skill roots.
- New `server/src/services/recovery/ownership-invariant-report.ts`: one
pass classifies every owner with `proveOwnerGone` (or the owning run's
status), then runs:
  - I1 owned: an in-progress record with no owner;
- I2 settled: an owner proven gone on two consecutive passes while the
record is still in progress;
- I3 admission independence: terminal runs whose accounting outlives the
10 min closure, which hold company admission;
- I4 exactly-once intents: wake requests ended `skipped`/`failed` on a
temporary refusal (heuristic until section E);
- I7 visible next action: an agent-assigned `in_progress` issue with no
live run, no pending wake and no open recovery action for 5 min.
- I5, I6 and I8 are reported as named coverage gaps; I8 comes in the
speed-budget PR.
- Violations are written once per process to the activity log as
`recovery.invariant_violation` (`actorId: ownership-reconciler`,
`details.mode: report_only`), insert-only with no live-event or plugin
fan-out. Nothing goes to Telemetry (AGENTS.md §5.7).
- `native-finalization-reconciler.ts`:
`registerOwnershipReconciliationWork` runs the pass in the shared
delivery-work scheduler (new queue `ownership-invariant-report`). There
is no second loop.
- `index.ts` registers it after residual recovery and does not await its
first pass, so readiness is not delayed.

### Sleep-aware trigger (rules/sleep-aware-background-work.mdc)

- **Trigger:** one pass at registration (startup reconstruction from
durable truth). After that, a settled `active-execution-recovery`
signal, which run lifecycle, controller renewal and process-evidence
writes already emit. Signals are coalesced to at most one pass per 30 s.
No producer gained a write.
- **Pending predicate and next deadline:** a 30 s follow-up only while
the last pass saw a changing in-progress set, or an owner proven gone
that needs the second I2 pass.
- **Quiet:** an empty set or the same stuck set twice sets no deadline.
The test asserts `nextWakeAt() === null` after an idle pass.
- **Turn path:** the pass never runs before `turn.start`. Stamps read
only cached identity.

## Verification

- `npx vitest run src/__tests__/ownership-reconciliation-report.test.ts`
(embedded Postgres, in `server/`): 4 tests pass:
  - every kind is registered once with no settle path;
- verdicts per class: alive (this process's claim), gone (lease of a
failed run), unprovable (no owner; another PID space);
- each rule fires on its fixture, and I2 fires only on the second pass;
- every touched table is byte-identical before and after (report-only);
- activity rows are written once per violation, and none are written
with `record: false`;
- the worker is quiet when idle and schedules a follow-up only while
records change.
- `npx vitest run src/__tests__/delivery-work-coordinator.test.ts
src/services/owner-proof.test.ts
src/__tests__/server-startup-feedback-export.test.ts`: pass.
- `npx tsc --noEmit -p server`: no errors.

## Risks

- No record changes, so no settle risk. The risk is load: one pass runs
about 46 small indexed or `limit 201` queries, at most once per 30 s
while runs are active, and none when idle.
- I4 matches refusal text with a heuristic. It can over- or under-count
until section E gives wake requests closed terminal reasons. It is
labelled as such in the code.
- Violations dedupe per process, so a restart writes a still-open
violation again (once per boot). Chaos reports count distinct `(rule,
entityId)`.
- Previous-release story (`rules/upgrade-safe-run-state.mdc`): no
persisted run state changes. The pass reads rows of any age, and a
missing field classifies as unprovable.
- `heartbeat_runs` and `native_run_finalizations` record no host or PID
space, so foreign-boot owners there classify as unprovable
(`missing_identity`). This is the inventory's finding 2, and the report
makes it countable.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 13:51:28 -06:00
Devin FoleyClaude Opus 5.5Paperclip
 		12681d1611 fix(sentry): stop resending retained task interruption failures (#19041) 
Keep task-state Sentry reports within the recent saved-change window while
retaining reports for newly failed recovery and delayed Retry preparation.
Include the safe run error code without changing event grouping.

Verified 402 relevant local tests, every server typecheck layer, all final-head
CI checks, and Greptile 5/5 with no unresolved review threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
 2026-11-07 11:31:46 -08:00
Devin FoleyandPaperclip 25273aa69b fix(onboarding): start the agent when the opening question is answered (#19886) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A new company starts with an onboarding first task. The task opens
with a greeting and an opening question card that the assignee agent
posts.
> - Onboarding does not wake the agent when it creates the task. The
user's answer must start the agent.
> - When the user clicks an answer, run dispatch cancels the agent's run
0.4 seconds later with `issue_not_in_progress`. Answer continuations
only start on `in_progress` or `in_review` tasks, and the onboarding
task is still `todo`.
> - The user sees nothing until an unrelated bounded retry starts the
agent about two minutes later. This is the first thing every new user
does.
> - This pull request lets an answer start a `todo` task when the task
has never started and no run asked the question.
> - The benefit is that the agent starts immediately after the first
answer. The protection for parked tasks does not change.

## Linked Issues or Issue Description

**What happened?**
I created a new company and clicked an option on the onboarding opening
question ("What would you like to do?"). The agent's run was cancelled
immediately with `issue_not_in_progress`: "Cancelled because
resolved-interaction continuation issue is no longer in_progress
(current status: todo) before the queued run could start". The task
stayed in `todo`. About two minutes later, a `transient_failure` bounded
retry of the cancelled run started the agent through the normal checkout
path.

**Expected behavior**
The agent starts as soon as the user answers the opening question.

**Steps to reproduce**
1. Create a new company and hire the first agent through onboarding.
2. Open the onboarding first task.
3. Click an option on the opening question card and submit.
4. Look at the agent's runs. The first run is `cancelled` with
`issue_not_in_progress`.

**Paperclip version or commit**
`master` at `86bc3db6938`. The same code is on `00ad9503fe3`.

**Relevant logs or output**
```
heartbeat_runs: status=cancelled error_code=issue_not_in_progress
  "Cancelled because resolved-interaction continuation issue is no longer in_progress (current status: todo) before the queued run could start"
two minutes later: "Scheduled bounded retry 1/2" (retryReason transient_failure)
```

Related: #13068 added the onboarding opening question. #13318 made
answer continuations require `in_progress` or `in_review`.

## What Changed

- New `server/src/services/unstarted-task-question-answer.ts`.
`isUnstartedTaskQuestionAnswer` returns true only when all of these are
true:
- the interaction is an answered question (`ask_user_questions`, status
`answered`) on this task, and it has no `sourceRunId` (no run asked it);
  - the task is `todo`;
  - the task's `startedAt` is null.
It reads only server-owned rows. `startedAt` is never cleared, so a task
that started and was later moved back to `todo` does not match.
- `run-dispatch/adapters/postgres.ts`: the staleness facts include
`isUnstartedTaskQuestionAnswer`. The adapter reads it under the issue
lock, and only for a resolved-interaction continuation on a `todo` task.
- `run-dispatch/domain/policy.ts`: `decideQueuedRunStaleness` lets such
a continuation start from `todo`. The other answer continuations still
require `in_progress` or `in_review`.
- `services/heartbeat.ts`: the final continuation checkout adds `todo`
to the expected statuses only for the same case. The task then moves to
`in_progress` in the same guarded update.

## Verification

- `heartbeat-stale-queue-invalidation.test.ts` (embedded Postgres):
- New test: an answer to a question that no run asked, on a
never-started `todo` task, starts the assignee. The task moves to
`in_progress` under the run's execution lock. With the fix reverted,
this test fails with the reported error (`cancelled`,
`issue_not_in_progress`).
- New test: three regression cases stay cancelled with
`issue_not_in_progress`:
    - an answer on a task that started and was moved back to `todo`;
    - an answer to a question that an agent run asked;
    - a rejected confirmation that no run asked.
    Without the kind and status check, the confirmation case fails.
- `policy.test.ts`: the new fact allows `todo`. The same fact on a
`backlog` task is still cancelled.
- `pnpm --filter @paperclipai/server typecheck`: exit 0.
- `npx vitest run` on these suites: 8 files, 497 tests passed.
  - stale-queue invalidation
  - run-dispatch policy
  - onboarding first-task routes
  - onboarding first-task admission
  - question-response delivery
  - issue-thread interaction routes
  - heartbeat retry scheduling
  - legacy continuation authority

## Risks

- Low risk. The change only affects answer continuations on `todo` tasks
that have never started, where the question has no source run. All other
continuations behave as before.
- The exception is not limited to the onboarding card. Any answered
question that no run asked, on a never-started `todo` task, can now
start the assignee. This matches what a plain comment wake does on a
`todo` task. Confirmations and other interaction kinds keep the existing
rule. For example, rejecting a confirmation that no run asked still does
not start the task.
- The final checkout does one extra indexed read. It only runs for
resolved-interaction continuations on `todo` tasks.
- I did not find which component scheduled the two-minute
`transient_failure` retry. After this change, onboarding does not depend
on that retry.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs). Investigation, fix, and tests were
AI-assisted, with human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
docs describe this behavior)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-07 11:29:23 -08:00
DottaandClaude Opus 5.5 f439150cb3 fix(recovery): limit the null-host owner fallback to single-host servers and record each use (#19883) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - #19868 and #19871 made restart recovery judge a recorded pid only in
the PID space that recorded it
> - Rows saved before those host columns existed have no recorded host.
They kept the previous local dead-pid proof, so in-flight native runs
would not be stranded at upgrade
> - On an ephemeral (multi-host) server that fallback is unsafe: the row
may have been written on another host
> - This pull request limits the fallback to single-host servers,
records every use, and gives ephemeral servers a working board action
> - The benefit is that single-host upgrades still recover on their own,
multi-host servers never accept a cross-host proof, and the next release
can remove the fallback with evidence

## Linked Issues or Issue Description

**Bug description**
After #19868 and #19871, a `native_run_finalizations` row with
`controller_host_hash = null`, or a run with `process_host_hash = null`,
still accepted "the pid is dead here" as proof on every server. That
includes an ephemeral server, where the row may come from another host.
Nothing recorded how often the fallback was used, so it had no safe
removal path.

**Expected behavior**
- Single-host server: the fallback applies for one release, and each use
writes a `legacy_host_unknown_dead_pid_proof` activity-log entry.
- Ephemeral server (`PAPERCLIP_EPHEMERAL_SERVER=true`): the row is
unprovable, and the board has an action to confirm the previous server
stopped.
- A mismatched host is always unprovable.

Refs #19868, #19871, #19846

## What Changed

- `native-restart-recovery.ts` `evaluateNativeControllerTakeover`:
- A coordinator row with `controllerHostHash: null` returns
`controller_host_unrecorded` when `ephemeralServerEnabled()` is true.
- Otherwise a pid proof from that row carries `legacyHostFallback:
true`.
- `operator_attested` is accepted as proof
(`operator_attested_controller_stopped`).
  - Receipt-built owners, which have no host key, are unchanged.
- New `recordLegacyHostFallbackProof` writes the activity-log entry. It
is not a run-log event: run events are progress evidence for restart
recovery's crash-loop credit, and the record must not change any
recovery decision. It is called from restart recovery, the
dead-controller result-lease release, native retirement, and remote
warm-session recovery.
- New `native-controller-host-recovery.ts`:
- `requestNativeControllerStopConfirmation` (board-owned
`native_controller_host_unverified` recovery action, idempotent, never
displaces another active action).
- `confirmNativeControllerStopped`. It verifies the exact recorded
controller and stamps `operator_attested` with compare-and-set. It then
resolves the action, wakes live-run recovery, and logs
`issue.native_controller_stop_confirmed`.
- Route `POST /issues/:id/recovery-actions/confirm-controller-stopped`.
It is board-only and requires `runtime:confirm_owner_stop`. Generic
`restored`/`false_positive` resolution of this cause is refused. A
shared `confirmNativeControllerStoppedSchema` validates the body.
- UI: `WorkspaceExportRecovery` reuses the "Confirm the previous server
stopped" control for this cause, and the generic recovery card hides it.
Adds the `issuesApi.confirmControllerStopped` client call.
- `recovery/service.ts` orphan reaper: a null `process_host_hash` is
judged only when not ephemeral. A successful use records the entry
best-effort, after the terminal write. On an ephemeral server the run is
left to its existing Stop control.
- `ephemeralServerEnabled` moves to a leaf `ephemeral-server-mode.ts`
(re-exported by `ephemeral-server-contract.ts`). Importing the contract
module from recovery code created an import cycle in the runner child
process.
- The new route is registered in OpenAPI.
- `doc/execution-semantics.md`: documents the host rule and the
one-release sunset.

## Verification

- `native-restart-recovery.test.ts`: null host on a single-host server
is released with the flag. Null host on an ephemeral server is
unprovable. A mismatched host is unprovable in both modes. An attested
controller is accepted. Receipt owners are unchanged.
- New `native-controller-host-recovery.test.ts` (embedded Postgres):
  - The fallback writes the activity-log entry and no run event.
  - Ephemeral mode creates exactly one board action.
  - A wrong run id is refused.
- Confirmation stamps the attestation, after which takeover is allowed.
The action resolves and the activity is logged. A second confirmation is
refused.
- `recovery-stale-issue-lock-sweep.test.ts`: on a single-host server a
null host is terminalized and records one event. On an ephemeral server
it is left alone with no event. The existing audit-failure case still
clears the lock.
- `WorkspaceExportRecovery.control.test.tsx`: the controller
confirmation posts to `confirmControllerStopped` and never to the
copyback endpoint.
- Locally: these suites plus `remote-warm-session-recovery`,
`native-controller-recovery.integration` and
`native-startup-cleanup-recovery` (136 passed), and after the CI fixes
`native-runner-restart-preparation` and `native-runner-restart-recovery`
integration, `openapi-routes` and the suites above (256 passed); the UI
recovery suites (52 passed); `node scripts/typecheck-layers.mjs` with no
errors in the touched files; `pnpm check:token-gates` clean.

## Risks

- Previous-release story: no schema change. Null-host rows saved by
earlier releases recover as before on single-host servers, now with an
event, and wait for the board on ephemeral servers. The next release
removes the fallback; the event count shows whether any rows remain.
- On an ephemeral server, a null-host orphaned run process is not reaped
automatically. The operator uses Stop. That is the intended "unprovable"
outcome.
- A board confirmation is an attestation. It applies only to the exact
recorded controller (boot id + pid) and only while the host is still
unrecorded.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used with tool use and
code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 13:25:47 -06:00
DottaandPaperclip 5a70999306 Add a versioned online workspace migration command (#19882) 
## Thinking Path

> - Paperclip preserves the files and recovery state produced by its
agents.
> - Canonical online migration can preserve selected local workspaces
while unrelated work continues.
> - Operators sometimes need a newer migration owner before the serving
web process can take that release.
> - The existing packaged maintenance entry requires stopped writers.
> - This pull request adds a finite online entry from one coherent
reviewed release tree.
> - The entry delegates capture and publication to the existing owners
and keeps tool and serving identities separate.

## Linked Issues or Issue Description

Refs #19854, #19855 and #19872. The duplicate search found related owner
fixes and no separate versioned online entry.

**What happened?**

The finite online workspace importer was available as a library in the
serving image. The packaged all-phase migration entry required
maintenance mode.

**Expected behavior**

Run the reviewed online owner from a separate complete release tree
without replacing the web process, stopping unrelated work or changing
its release policy.

**Deployment mode**

A managed instance with its original local volume, database, existing
encryption key and shared S3 storage.

## What Changed

- Add strict inspect, single, cohort and exact single-journal resume
requests with explicit company, workspace, deadline and concurrency
bounds.
- Verify the private tool root and its own build stamp separately from
the serving source commit and deployment.
- Reuse the nonmutating full migration-manifest inspector and require
the additive controller host-identity column before publication.
- Hydrate existing configuration and key bindings, then use the original
ESM database pool and canonical storage, source locks and migration
owners.
- Keep multi-member inspection with the optional canonical cohort
preflight in a read-only transaction with denied storage. Releases
without that owner refuse cohort operations; single import and resume
remain available.
- Register the optional runtime bundle entry and document the supported
full-tree source mode, resource admission and reconciliation.
- Add hash-bound old-writer/reader compatibility and encrypted restore
tests.

## Verification

Final local revision: `c9345b72f27aee1158047dccd0b1060f9a0be6df`, on
master `fbe689938c3bf8c5849d2cd56b9da90029db9389`, macOS arm64 with Node
24.22.2.

- Entry and runtime packaging: **34/34 PASS**.
- Real PostgreSQL/filesystem compatibility: **6/6 PASS**, disposable
database wrapper exit 0 and cleanup complete. The frozen old source lock
owner contends with the new entry; unrelated work proceeds. The old
authority and snapshot readers decrypt and restore newly published
bytes.
- Pending and uploaded journal cases use the unchanged old persisted
format and preserve operation/snapshot IDs through resume.
- Full layered server source and test typecheck: **PASS**.
- Module boundaries and `git diff --check`: **PASS**.
- Actual source-mode module import under Node 24 succeeds without
starting the application.
- A separate temporary full-old-producer fixture did not get past module
setup: 90 seconds under Vitest and a 15-second direct Node control.
Neither is counted as an executed old-producer test or a product
failure. The smaller frozen old-writer/reader proof and old-format
journal cases are the local evidence.
- Hosted resume of the actual saved operation and Linux artifact
qualification remain operational acceptance gates. This PR does not
claim those ran.
- Review regression: a real successful import returned a null operation
ID (genuine RED). The repaired entry binds the exact returned snapshot
to its succeeded migration journal after commit (GREEN). Committed or
superseded snapshots are accepted; a missing readback requires
reconciliation rather than a blind retry.
- Final affected check: **40/40 PASS** across the entry, packaging and
actual PostgreSQL compatibility suites; wrapper exit 0 and cleanup
complete. Full layered typecheck and boundaries pass after the
user-requested rebase and current Runner rebuild.
- A second real postcommit database-read fault also has genuine
RED/GREEN proof. The succeeded journal, saved authority and original
bytes remain intact, and every readback error now preserves the
reconcile-required warning.
- Final-head [CI run
42404116001](https://github.com/paperclipai/paperclip/actions/runs/42404116001):
**PASS**, with all 59 check contexts terminal (55 success and 4
intentional skips). Fresh Greptile review is **5/5** on
`c9345b72f27aee1158047dccd0b1060f9a0be6df`, with no unresolved threads;
check and thread pagination are complete.

## Risks

- The tool must come from one authenticated complete release tree with
its own dependencies. A copied bundle chunk can retain old absolute
paths and is not a portable release.
- Concurrency is not resource admission. The operator must check memory,
CPU and scratch headroom and start with one import or two cohort
members.
- Schema must be prepared by the existing schema owner. This entry never
applies DDL.
- Cancellation awaits the canonical owner. An SSH interruption does not
prove the process stopped; durable journals must be reconciled before
retry.
- The entry preserves remote obligations and source files. It does not
prove enrollment readiness or zero-downtime operation.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools and code
execution. The exact serving snapshot and context-window size are not
exposed to this agent.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-11-07 13:20:35 -06:00
DottaandClaude Opus 5.5 5a20fca9f7 fix(recovery): judge an orphaned run's pid only in the PID space that recorded it (#19871) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The orphan reaper (`terminalizeOrphanedRunningRun`) ends a `running`
run whose recorded process is gone
> - It judged "gone" with `kill(pid, 0)` on the local host, wherever the
process actually ran: another host, another PID namespace, or a sandbox
> - So a run whose process is alive elsewhere could be terminalized
while it still works
> - #19868 fixed the same gap for native coordinator takeover and the
runtime-control sweep. This pull request finishes finding 2 for the
reaper
> - The benefit is that a dead local pid never proves an owner on
another host is gone

## Linked Issues or Issue Description

**Bug description**
The Phase 0 ownership inventory (finding 2) names the orphan reaper
`terminalizeOrphanedRunningRun` (`recovery/service.ts`) as a proof that
accepts "the pid is dead here" for a process anywhere. `heartbeat_runs`
recorded `process_pid` with no PID space, and a sandbox pid could
collide with a local one.

**Steps to reproduce**
1. A run records `process_pid` on host A, or in a sandbox.
2. On host B, that pid number is not alive.
3. B's stale-lock sweep terminalizes the run as `orphaned_running_run`.

**Expected behavior**
The process-death authority applies only in the PID space that recorded
the pid. Issue-terminal authority is unchanged.

Refs #19846, follows #19868

## What Changed

- Migration `0442`: add nullable `heartbeat_runs.process_host_hash`.
- `heartbeat/run-lifecycle.ts` `persistHeartbeatRunProcessMetadata`, the
one writer of `process_pid`, stores `currentProcessSpaceHash()` for a
local pid and `"remote"` for a sandbox pid.
- `recovery/service.ts`: the process-death authority runs only when
`process_host_hash` is null (rows saved before this column, for one
release) or `sameProcessSpace` matches.
- Tests: the reaper leaves a dead pid alone for another host, a sandbox,
and an unreadable space, and terminalizes it in this space. The writer
stamps local and remote pids.

## Verification

- `npx vitest run
src/__tests__/recovery-stale-issue-lock-sweep.test.ts`: 46 passed.
- Red before the fix: with the old `recovery/service.ts`, the three
cross-space cases fail (the run is terminalized). Green after.
- `pnpm db:generate` produced `0442_low_salo.sql` (one `ADD COLUMN`),
and the snapshot set was pruned to five. `pnpm --filter @paperclipai/db
check:migrations` passed. Server `tsc --noEmit` shows no errors in the
changed files.

## Risks

- Previous-release story: runs saved with `process_host_hash = null`
keep the previous local proof for this release. That matches #19868,
because stranding in-flight runs at upgrade would be a restart failure.
New rows are strict.
- A run whose process really lived on another host now stays `running`
until issue-terminal authority, its controller lease, or another owner
settles it.
- No new I/O before `turn.start`. The hash is computed at module load,
and the stamp rides the existing process-metadata update.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used with tool use and
code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 12:22:15 -06:00
DottaandClaude Opus 5.5 00ad9503fe feat(commentary): agents read only their own commentary unless granted commentary:view (#19842) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents can submit complaints and suggestions as commentary, and
people with commentary access read them on the Commentary page
> - When a person asks an agent in chat what agents have been
complaining about, the agent cannot read commentary and gets "Board
access required"
> - That response does not say where a person can see it, so the agent
can name the feed but cannot link it
> - Product rule: an agent cannot read other agents' commentary unless
it holds `commentary:view`, which is off by default; it may read its own
> - This pull request implements that rule and adds a remediation to the
denial naming the human page for the agent's own company
> - The benefit is a clear permission boundary and a working link for
the person instead of a dead end

## Linked Issues or Issue Description

**What happened?**
In the `activity.agent-feedback` chat eval, the agent correctly said it
could not read agent commentary and that only the person could open it,
but it gave no link, because the denial named no page.

**Expected behavior**
An agent denied a commentary read learns which page a person with
commentary access can open.

**Steps to reproduce**
1. As an agent, call `GET /api/companies/{companyId}/agent-commentary`.
2. Before this change, the 403 body is only `{ "error": "Board access
required" }`.

## What Changed

- `server/src/routes/agent-commentary.ts`:
- An agent with the `commentary:view` grant reads all commentary its
task access allows. The grant is not among the default agent grants.
- Without the grant, an agent may read its own entries: `?agentId=<its
own id>` on the list, or its own entry by ID.
- Without the grant, listing all commentary, filtering by another agent,
or reading another agent's entry returns 403 "Board access required".
The response's `remediation` names `/<issuePrefix>/activity/commentary`
for the agent's own company.
- A request for another company still fails on company access first,
with no page path.
- Board reads are unchanged: `commentary:view` for the user, plus task
read conditions.
- `agent-commentary.integration.test.ts`: one test covers the full-list
and other-agent denials with the page path, own-list and own-entry
reads, the cross-company denial, the grant allowing a full list, and
that `commentary:view` is not a default agent grant.

## Verification

- `npx vitest run src/__tests__/agent-commentary.integration.test.ts` in
`server`: 21 tests passed.
- The new test fails against the previous route.
- Negative control: with the previous route, the new test fails.
- Related commentary tests: `agent-commentary-reader`,
`agent-commentary-helper`, `commentary-permission-migration` passed.
`agent-commentary-skills` fails the same way with and without this
change (a missing skill file in a temporary directory locally).
- The chat eval rerun with this change is recorded in #19614.

## Risks

- Agents can now read their own commentary when they ask for it
explicitly; before, every agent read was denied.
- The denial's status code and message are unchanged; a `remediation`
string is added for same-company agents.
- There is no UI yet for granting `commentary:view` to an agent.
- An agent learns the company's issue prefix, which it already sees in
task identifiers.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 12:04:46 -06:00
DottaandClaude Opus 5.5 05e3309740 fix(accounting): settle a failed usage capture as unpriced at the closure bound (#19857) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Every finished run settles its usage into the cost ledger. Until it
settles, budget admission makes new company work wait
> - When usage capture fails (`accountingCaptureFailed=true`),
accounting skips the run forever. It is also not "recovering", so it
counts in `waitingRunCount`
> - So one failed capture holds new company work until the UTC budget
month rolls over
> - This pull request bounds that wait with the existing accounting
closure timeout. It then settles the run's known usage as unpriced, with
provenance that names the capture failure
> - The benefit is that admission recovers by itself, while unknown
spend stays visible and still counts under a blocking unpriced-usage
policy

## Linked Issues or Issue Description

**Bug description**
The Phase 0 ownership inventory (finding 3) found that runs with
`accountingCaptureFailed=true` are excluded from accounting
(`run-cost-accounting.ts` `actionableRunAccounting` and
`accountRunCost`). They are also excluded from "recovering" in
`budget-exposure.ts`, so they count as waiting until the month window
rolls over. Nothing else releases them.

**Steps to reproduce**
1. Finish a run whose usage capture failed (for example, a fence write
failure or usage after a source handoff).
2. Wait any length of time and run `reconcileRunCosts`.
3. The run stays `costAccountingPending`, and `getInvocationBlock` keeps
holding new company work.

**Expected behavior**
After a bounded wait, the run settles with its known usage marked
unpriced, and held work resumes. A blocking unpriced-usage policy still
treats the run as unknown spend.

Refs #19846

## What Changed

- `run-cost-accounting.ts`: new `captureFailureBoundPassed`. It applies
to a terminal capture-failed run whose native coordinator (if any)
settled, after `ACCOUNTING_CLOSURE_TIMEOUT_MS`. This is the same bound
used for missing closure evidence.
- `actionableRunAccounting` selects these runs, and
`readCostAccountingWork` schedules a wake at their bound. A past bound
never schedules a wake, as before.
- `accountRunCost`: a capture-failed run is never treated as
pre-provider work. Before the bound it stays pending. After the bound it
settles through the existing lost-receipt path as `receiptUnavailable` +
`closureTimedOut`: known cost becomes a lower-bound estimate, otherwise
the charge is unpriced, and the run gets `accountingClosureTimedOut:
true`.
- Provenance evidence and the `cost.accounting_closure_timed_out`
activity say that the usage capture failed (`details.captureFailed:
true`).
- Tests: three new cases in `closed-native-receipt-accounting.test.ts`.
`cost-data-quality.test.ts` now pins both sides of the bound for a board
retry.

## Verification

- `npx vitest run
src/__tests__/closed-native-receipt-accounting.test.ts`: 54 passed.
- `npx vitest run src/services/run-cost-accounting.test.ts
src/__tests__/idle-accounting-work.test.ts
src/__tests__/cost-data-quality.test.ts`: 40 passed.
- Red before the fix: with the old `run-cost-accounting.ts`, all 3 new
capture-failure cases fail. The run never settles, nothing schedules a
wake, and work stays held. Green after.
- Hard stop: "keeps a settled capture failure under the blocking
unpriced-usage policy" shows that a blocking policy still blocks after
settlement. Real observed spend (`budgetExposure`) counts toward the
hard stop before and after.
- Server `tsc --noEmit` shows no errors in the changed files.

## Risks

- Behavior change: capture-failed runs no longer stay pending forever.
The 2026-10-28 Cursor runner plan recorded "capture failures remain
pending" as an interim rule. This PR keeps them pending through the
closure bound and then settles them as unknown spend, never as free. A
complete receipt that arrives later still reconciles through
late-receipt handling.
- A board retry of a capture-failed run past the bound now settles it.
Before the bound it is still refused.
- Previous-release story: no persisted shape changes. Existing
capture-failed rows past the bound settle on the next reconcile pass.
That can release work that was held, and under a blocking policy it
records unknown spend that blocks.
- No new waits or I/O before `turn.start`. This is background accounting
only.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used with tool use and
code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 12:01:56 -06:00
DottaandClaude Opus 5.5 51c352ec2e fix(skills): release a runtime skill-cache lock whose holder provably exited (#19849) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Before a run starts, the server builds a read-only cache of each
company skill under a cross-process file lock
> - The lock records only a pid and a hostname, so a crash leaves a lock
that nothing can release
> - If the holder pid is dead on the same host, the next run fails and
asks for manual cleanup. If the host name differs, the next run waits 60
s and then fails
> - This is on the turn path, so one crash during a skill build breaks
later runs that use that skill
> - This pull request records the shared controller process identity in
the lock and releases the lock at once when `controllerProcessExited`
proves the holder is gone
> - The benefit is that a crash during a skill build no longer blocks
later runs, and the release never relies on lock age or another host

## Linked Issues or Issue Description

**Bug description**
The Phase 0 ownership inventory (finding 4) lists the runtime
skill-cache lock (`runtime-skill-cache.ts`) as a record that nothing
releases automatically. When a publisher crashes while it holds
`<skillId>.lock`, every later preparation of that skill fails.

**Steps to reproduce**
1. Start a skill cache build, then kill the server process while it
holds the lock.
2. Restart and start a run that uses the same skill.
3. Preparation fails with "Runtime skill cache publisher exited; remove
its stale publication lock before retrying". For a holder whose host
name differs, it fails after 60 s with "publisher is busy".

**Expected behavior**
A lock whose holder has provably exited is released, and the build
continues without waiting. A holder that cannot be proven gone is still
waited for.

Refs #19846

## What Changed

- `controller-process-identity.ts`: add
`currentControllerProcessStamp()`. It returns this process's identity
with no I/O, so the turn path gets no new wait.
- `runtime-skill-cache.ts`: the lock owner file now also stores
`controller` (boot id, pid, PID-space hash, start time if already
known). `pid` and `host` stay so older readers still work.
- New `runtimeSkillCacheLockHolderExited`: true only when
`controllerProcessExited` proves the holder gone. Another host or PID
namespace, an unknown space, a legacy lock without `controller`, a live
pid, and this process all return false.
- On proof, waiters race for one exclusive claim named by the stale
holder's unique token (`<lock>.released-<token>`). Only the winner
unlinks the lock, and only while it still has the stale inode. The claim
is kept, so a late waiter does nothing. The waiter then retries the link
right away instead of after the 20 ms poll.
- A live holder is re-proved at most once a second.
`controllerProcessExited` skips the start-time probe (`ps` on macOS)
when no start time was recorded.
- The busy error now names the lock path. The same-host "publisher
exited" hard failure is removed.

## Verification

- `npx vitest run src/__tests__/runtime-skill-cache.test.ts
src/services/controller-process-identity.test.ts`: 28 passed, rebased on
master after #19848.
- New case "takes over a release claim whose claimer exited before
removing the stale lock" covers the second Greptile finding.
- New case "never removes a newer lock for a waiter that read a stale
one late" covers the Greptile P1.
- New case "releases a publication lock at once when its holder provably
exited". On the old code it fails with "Runtime skill cache publisher
exited; remove its stale publication lock before retrying". On the new
code it builds in well under 5 s and leaves no lock.
- New case "never treats age, another host, or a legacy holder as
proof": a different host hash, a legacy `{pid, host}` lock, a live pid,
this process, and malformed content all stay held.
- Server `tsc --noEmit` shows no errors in the changed files.

## Risks

- Previous-release story: a lock written by the previous release has no
`controller` field. It is unprovable. The waiter waits 60 s and fails as
before, and the error now names the path to remove. A new lock still
carries `pid` and `host`, so a previous-release reader behaves as
before.
- Lock release race: while a stale lock exists, only the single claim
winner can remove it, and the inode check cannot match a newer lock
because the stale holder's owner file stays and pins that inode. A claim
whose claimer crashed is itself released by the same proof (depth
bounded at 2).
- Each released lock leaves one small claim file and the dead holder's
owner file next to it.
- No new I/O before `turn.start`. The stamp is synchronous. The proof
runs only when the lock is contended.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used with tool use and
code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 11:51:25 -06:00
DottaandClaude Opus 5.5 738058553f fix(recovery): resume a continuation that never launched its runner in its own sandbox (#19850) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A chat follow-up on a remote native task runs as a continuation: it
starts from its predecessor's saved provider session and, for a warm
session, copies the predecessor's runner process identity onto the new
run
> - When the continuation fails before it asks for a runner of its own,
its due same-run retry sees a recorded process identity and reattaches
to "this run's" runner. That runner never existed, so reattachment
always fails with `native_remote_recovery_lease_mismatch` and the task
is blocked
> - On staging, 34 OpenCode runs on 31 tasks failed this way in one
evening. Each first attempt failed while staging its provider pack,
recorded no runner start, and kept its own sandbox lease active
> - This pull request proves that shape from durable state, retires the
inherited identity, and resumes the run the way its failed attempt ran,
in its own retained sandbox
> - The benefit is that these retries recover automatically, without a
second sandbox and without the unsaved-work hold, while every run that
may hold its own provider work keeps verified reattachment

## Linked Issues or Issue Description

Refs #19839, Refs #19399, Refs #19755, Refs #19779, Refs #19827

**What happened?**
A remote OpenCode chat continuation failed during provider-pack staging.
The run had no runner start event, no process identity event, no
controller checkpoint, and no provider event of its own. Its saved
session (no active turn, no pending request) and its process identity
were inherited from the previous run. Its failure detail recorded
`providerEventsExist: false`, and its own sandbox lease stayed active.
The due same-run retry chose `reattach_remote_runner` because a process
identity was recorded. Reattachment failed with
`native_remote_recovery_lease_mismatch`, and the run ended in
`native_continuation_requires_reconciliation`.

**Steps to reproduce**
1. Continue a warm remote native session so the new run inherits the
predecessor's runner process identity and saved session.
2. Fail the new run with a retryable error before it requests a runner
(for example during provider-pack staging).
3. Let the same-run retry become due. Recovery requests
`reattach_remote_runner`, and reattachment fails.

**Expected behavior**
The retry resumes the run the way its failed attempt ran, in the same
sandbox, and continues the inherited provider session. A run with any
provider work, runner start, or controller checkpoint of its own keeps
verified reattachment.

## What Changed

- New
`server/src/services/native-runtime/native-inherited-runner-retry.ts`:
- `retireInheritedRemoteRunnerIdentity` locks the issue, the
coordinator, then the run (the same order as same-run resumption). It
retires the recorded process identity only when all of these hold: the
coordinator is a result-less, unowned `exact_checkpoint_resume` retry
with `providerEventsExist: false`; the run has no runner start, process
identity, adapter invocation, tool start, runner event, controller
checkpoint, provider event or provider session of its own; the saved
session has no active turn and no pending runtime request; and a settled
predecessor of the same native session recorded the exact same process
identity (PID, group, start time).
- It records `nativeInheritedRunnerRetirement` on the run profile, with
the run's own sandbox location when that lease is still active, and an
`inherited_runner_never_launched` lifecycle event.
- `readInheritedRunnerRetainedSandbox` returns that sandbox only while
it still matches the run's saved remote location.
- `heartbeat/recovery.ts`: a due remote retry calls the retirement
first. It returns `retired`, `not_applicable` or `transient`. A retired
run (including one a concurrent sweep already retired) goes to the
ordinary same-run resume (`claimNativeSessionResumptions`), with every
existing admission gate. A lock timeout or database error is
`transient`: the sweep defers the run and checks again, and never falls
back to reattachment. Any other run keeps verified reattachment.
- `heartbeat/workspace-preparation.ts`: an ordinary same-run resume with
a retired inherited runner acquires its retained sandbox through the
existing exact-lease path (`reattachRemoteLease`), which validates and
resumes that lease and never acquires a replacement. It acquires as an
ordinary resume does only when the exact-lease failure proves the
sandbox is unusable (`stage=environment_acquire|environment_resume`,
`field=lease|lease_status|lease_expired|provider_lease_id`). Any other
failure, such as a scope mismatch on a live lease, is kept and acquires
nothing.
- New `server/src/services/retained-sandbox-absent-error.ts`: a sandbox
provider's resume reply with no provider lease and `expired: true` (the
reply Daytona and other providers send when the sandbox is gone) now
throws `RetainedSandboxAbsentError`. Every other mismatched reply keeps
the existing error. The exact-lease reattachment in
`environment-run-orchestrator.ts` maps it to
`native_remote_recovery_lease_mismatch: stage=environment_resume
field=lease_expired`, so the same-run resume above acquires as an
ordinary resume when the provider has removed the retained sandbox. A
transient resume failure is not treated as absence and never acquires
another sandbox.
- `native-restart-recovery.ts` exports the existing
`hasNativeExecutionStartEvidence` so both paths use one definition of
"execution reached a runner".
- `doc/execution-semantics.md` documents the rule.

## Previous-release state

- No schema change. The new profile field is additive.
- A run saved by an older release without `providerEventsExist`, without
a native session ID, or without a matching predecessor keeps verified
reattachment. That path is unchanged. When it loses its runner, it still
requests the user's Retry once (#19399).
- A run that an older release left with its own runner start, controller
checkpoint or provider event keeps verified reattachment, because its
outcome may be ambiguous.
- A profile without the new marker never changes acquisition.

## Verification

- `npx vitest run server/src/services/heartbeat/recovery.test.ts -t
"never launched its own runner|keeps verified reattachment"` (embedded
PostgreSQL):
- Staging shape (inherited session and process identity, no runner start
of its own, saved remote workspace, active own lease, recorded PID).
Before the fix: fails, because recovery requested reattachment. After
the fix: passes. The PID fields are cleared, the marker names the
predecessor and the retained lease, `executeRun` is called once with a
plain `nativeLeaseOwner` claim, the run still has exactly one (active)
lease, and no process is signalled.
- Negatives, passing before and after: a provider turn of its own, a
runner start of its own, a controller checkpoint of its own, no matching
predecessor, a pending runtime request in the saved session, and a
failure detail without `providerEventsExist`. Each keeps verified
reattachment and gets no marker.
- `npx vitest run
server/src/services/heartbeat/workspace-preparation.test.ts -t "retained
sandbox|retired inherited"`:
- A retired inherited runner acquires its retained sandbox with
`reattachRemoteLease`. Before: fails. After: passes.
- When the retained sandbox is gone, it acquires as an ordinary resume.
Before: fails. After: passes.
- Without the marker, acquisition is unchanged (passes before and
after).
- A `nativeWorkspaceSync` reference also resumes its retained sandbox. A
live lease's `lease_scope` failure is kept and acquires nothing.
- `recovery.test.ts` also covers a second retirement call (returns
`retired`, one event) and a held issue lock (defers; no reattachment, no
dispatch, PID kept).
- `environment-runtime.test.ts`: on exact resume, only a provider reply
with no lease and `expired: true` (with or without a workspace sentinel
mismatch) is `RetainedSandboxAbsentError`; a different lease or a reply
without `expired` keeps the existing error.
`environment-run-orchestrator.test.ts`: confirmed absence becomes
`stage=environment_resume field=lease_expired` and acquires nothing.
- Affected suites pass locally: `recovery.test.ts`,
`workspace-preparation.test.ts`,
`heartbeat-native-remote-failure-retention.test.ts`,
`workspace-recovery-retry-copyback.test.ts`,
`native-runner-restart-recovery.integration.test.ts`. `pnpm --filter
@paperclipai/server typecheck` passes.

## Risks

- **No second executor.** The rule applies only when this run never
requested a runner, so no runner of this run exists. The predecessor
must be settled, and its identity is matched exactly. The resume then
takes the normal same-run claim, which holds the issue lock and
admission gates.
- **No repeated side effects.** The run has no provider event, turn or
tool start of its own. A saved session with a pending runtime request or
an active turn keeps verified reattachment. The resumed provider session
goes through the existing provider recovery, which refuses a pending
side-effecting call with a permanent, named code.
- **Bounded.** The retirement takes effect once per run, because
afterwards the run has no recorded identity. A contended retirement
waits for the next sweep. Further failures use the existing
three-attempt provider budget.
- I did not run a live Daytona sandbox, or a full preparation through a
real sandbox plugin, through this path. The orchestrator's exact-lease
checks are covered by its own unit tests. The evidence is recovery and
preparation tests on a real database with the staging run shape.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, run in Claude
Code with tool use (shell, file editing, read-only browser access to
staging).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 11:35:23 -06:00
Devin FoleyandClaude Opus 5.5 de82f93443 fix(storage): retry S3 reads that the provider times out with HTTP 408 (#19039) 
Mark HTTP 408 on GET/HEAD as retryable inside the SDK retry middleware so the standard backoff, retry quota and three-attempt limit apply. Writes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 09:34:46 -08:00