Commit Graph
4 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 89f486cdab fix(heartbeat): run one dispatch pass per agent, back off refused claims, and enforce the run limit in the claim (#21433)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The heartbeat queue starts an agent's queued runs. Each trigger (a
run finishes, a wake arrives, a sweep runs) asks for one dispatch pass
of that agent
> - An agent could run more runs than its `maxConcurrentRuns`, dispatch
passes could stack up without a limit, and every pass offered the same
unclaimable runs again
> - On hosted staging (e23549cd9c), one agent with limit 20 had 20–35
runs `running` and 170–270 queued. The logs showed 375 "offered a free
slot but could not be claimed" lines and 117 "claim was rejected" lines
in 10 minutes
> - This pull request runs one pass per agent at a time, folds triggers
into one follow-up pass, backs off refused claims, enforces the limit in
the claim transaction, and reads only the columns the pass needs
> - The benefit is that an agent never runs past its limit (also across
two servers during a deploy), and a burst of triggers costs one or two
passes, not one pass per trigger

## Linked Issues or Issue Description

Refs #21391, #21242, #21234, #21277, #21324, #21349

**What happened?**

On hosted staging (Core e23549cd9c), one agent ("Codie",
`maxConcurrentRuns` 20) had 170–270 queued runs and 20–35 runs
`running`, so it was over its limit. The logs showed a refused claim
offered again on every pass. This pull request is about correctness and
load. It is not the cause of the staging out-of-memory crashes: a
separate leak (`sessionToolAuthorityEpochs` in
`native-session-executor.ts`, about 3 MB per finished native run)
explains those, and another change fixes it.

Live again at 17:02Z on staging (Core 04f12fd2be, which includes
#21391): the same agent had 52 runs `running` against
`maxConcurrentRuns` 40. All 52 started between 16:55Z and 17:02Z, right
after a deploy released the backlog, with 112 runs still queued. So
overlapping passes exceed the limit with claimable runs, not only with
unclaimable ones.

**Why did it happen?**

1. `withAgentStartLock` chained each trigger behind the previous one and
measured its 30 s stale limit from when the predecessor was chained, not
from when it started. When triggers arrived faster than one per pass,
every trigger after about 30 s of backlog started its own pass next to
the others.
2. Each pass counted free slots by itself, outside any lock that other
passes or another server see. Overlapping passes and two servers during
a deploy swap each claimed up to the limit.
3. Each pass read every queued run with `select *`, including the large
`contextSnapshot`, and offered every refused run again with no backoff.

A local reproduction of the staging shape by another session (250
unclaimable queued runs with 8 KB snapshots, one trigger per second, on
e23549cd9c and again on master e7e28ba241): passes that run at the same
time went 1 → 12 → 39 → 79 → 139 at t = 30/45/75/116/178 s, only 4
passes finished in 178 s, and post-GC heap grew by about 2.8 MB per
stacked pass (519 MB → 1,061 MB at 190 passes, mostly the parsed
`heartbeat_runs` rows and the dispatcher facts each pass holds). 242 s
after the triggers stopped, 216 passes were still outstanding.

**Expected behavior**

At most one dispatch pass per agent runs at a time. Triggers that arrive
during a pass cause one follow-up pass. The `running` count never goes
above `maxConcurrentRuns`. A refused run is offered again after a
bounded wait, and no queued run is stranded.

**Steps to reproduce**

1. Give an agent a limit of 4, 12 aged queued runs that the chat-control
gate refuses, and 12 claimable runs.
2. Start a pass. While its first claim waits, send 40 triggers, each
after more than 30 s of the old lock's clock.
3. Count the passes, the refused offers, and the highest `running`
count.

## What Changed

- `server/src/services/agent-start-lock.ts`: replaces the stale-bypass
lock with `dispatchForAgent`. One pass runs per agent. Triggers during a
pass merge into one follow-up pass that all their callers share. A
trigger from inside the agent's own pass does not wait on itself. A pass
has a 60 s deadline on a monotonic clock, measured from when the pass
started (not from when a waiter was chained). At the deadline the pass
stops claiming (an abort signal) and the follow-up may start. A trigger
that arrives after the deadline also ends the pass, so a stalled timer
cannot keep an agent blocked. Post-pass effects run after the agent is
free, as before.
- `server/src/services/heartbeat/run-slot-admission.ts` (new):
`assertAgentRunSlotFree` counts the agent's `running` runs and compares
them with `maxConcurrentRunsSql()` inside the agent-scoped advisory lock
that every claim already takes (`withExternalAdmissionGuard`). It throws
`AgentRunSlotsFull` when the agent is full. The count leaves out the run
that the claimed run replaces (`retryOfRunId` or `previousRunId`), so a
successor that starts while its predecessor finishes is not refused.
- `server/src/services/heartbeat/queue.ts`:
- Every claim path (ordinary, queued comment, native review through
`native-review-dispatch.ts`) checks the slot inside its claim
transaction. A full agent leaves the run queued, and the pass stops.
- The pass reads only the columns the dispatcher needs, with `issueId`,
`wakeReason` and `previousRunId` taken from `contextSnapshot` in SQL. It
reads the full row only for the run it is about to claim.
- A refused claim (declined or 4xx) of a run that has waited more than
an hour (the existing aging limit) is not offered again for 5 s, then
10, 20, 40, and at most 60 s. A person's run of that age waits 2 s. A
run queued for less than an hour is never backed off, so a successor or
retry that waits for its predecessor starts as soon as the predecessor
ends. A backoff also ends when the run's task changes (status,
`updatedAt` or execution run) or when a run of that task ends in this
process. A backed-off run does not take the oldest-run share. The
queued-run sweep (`resumeQueuedRuns`, `resumeQueuedRunsAfterDrain`,
which the queued-run redrive calls) offers refused runs again regardless
of the backoff.
- A pass makes at most 24 claim attempts. When it reaches that limit it
asks for one follow-up pass that skips every run the chain already
tried, so the follow-up reaches runs further down the line. The chain
ends when a pass does not reach the limit, so refused runs cannot cause
a continuous chain of passes. Only this automatic follow-up carries the
tried list. A wake, a completion or another fresh trigger that merges
into it resets the list, so a run whose blocker cleared during the chain
is offered again.
- Admission, suppression and cutoff are read once per pass, not once per
trigger.
- Seam for a skip before the claim: in `runQueuedDispatchPass`
(`claimQueuedRunsForAgent`), right after the skip line (`if
(triedRunIds.has(candidate.id) || (!retryRefused &&
claimBackedOff(candidate.id, new Date(), taskStamp))) continue;`),
before the claim-budget check, `claimAttempts += 1` and the full-row
read. A `continue` there spends no claim attempt, reads no row, does not
enter the claim backoff or the aged-decline note, and does not call
`dispatcher.claimed`, so the run keeps its place and its oldest-run
share. The dispatcher exposes `dispatcher.personOrigin(run)` (the
person-origin fact it already derives) and `dispatcher.personRun(run)`.
The candidate rows keep `id`, `companyId`, `agentId`, `status`,
`invocationSource`, `retryOfRunId`, `wakeupRequestId`,
`scheduledRetryAt`, `scheduledRetryAttempt`, `scheduledRetryReason`,
`createdAt`, `startedAt`, and
`contextSnapshot.issueId/wakeReason/previousRunId`.
- Each pass is registered with the execution drain
(`drainActiveRunExecutions`) until it ends or reaches its deadline.
- Tests: new `server/src/services/agent-start-lock.test.ts` and
`server/src/__tests__/queued-run-dispatch-stacking.test.ts`.
`server/src/__tests__/heartbeat-start-lock.test.ts` is removed: it
asserted the 30 s stale bypass that this pull request removes.
`workspace-recovery-preparation-dispatch.test.ts` holds and counts
passes through the new function.

The chat-control fallback lookup from #21391 (`issueId = X OR
nativeIssueId = X OR contextSnapshot->>'issueId' = X`) was measured on
embedded Postgres with 200,000 runs in one company. Postgres uses a
BitmapOr of three existing indexes
(`heartbeat_runs_company_issue_created_idx`,
`heartbeat_runs_company_native_issue_contract_id_uq`,
`heartbeat_runs_company_ctx_issue_created_idx`), not a sequential scan:
0.33–0.45 ms per call for a recent previous run, a 30-day-old previous
run, and no previous run. It is not changed.

## Verification

Red/green, `server/src/__tests__/queued-run-dispatch-stacking.test.ts`
(embedded Postgres, real heartbeat queue, limit 4, 12 refused aged runs,
12 claimable runs, 41 triggers):

| Measure | master | this pull request |
|---|---|---|
| Passes | 42 | 2 |
| Peak passes at the same time | 40 | 1 |
| Peak claims in flight | 9 | 1 |
| Refused-claim offers per trigger | 12 | 0.29 (12 in total) |
| Peak `running` (limit 4) | 8 | 4 |
| Passes outstanding after the burst | — | 0 |

- "waits a bounded time before offering a declined/rejected run again":
on master 10 triggers offer the refused run 10 times. With the fix they
offer it once, and once the backoff passes it is offered again. The
rejected case uses the 422 "Unable to resolve responsible user" path
from the staging reproduction.
- "starts a run that becomes ready during a chain of passes when a fresh
trigger arrives": red at 3d48c4bed2, green now.
- "stops chaining passes once every refused run was tried": 60 refused
runs and one trigger cause exactly 3 passes, then no more.
- "starts runnable work queued behind more refused person runs than one
pass tries": 30 refused person runs ahead of 3 runnable runs, with free
slots. The runnable runs start after one trigger, in at most 3 passes.
At 68e64a2de9 the same 24 person runs were offered on every pass and the
runnable runs never started.
- `server/src/services/agent-start-lock.test.ts`: one pass for a burst
of 50 triggers plus one follow-up, no second pass after 45 s, a hung
pass stopped at its deadline without overlap, no self-wait, settle after
release, and errors passed to the caller.
- `node server/scripts/typecheck-layers.mjs`: pass.
- Vitest (embedded Postgres): `heartbeat/queue.test.ts` (102),
`recovery-wake-herd.test.ts` (35),
`heartbeat-queued-run-claim-isolation.test.ts`,
`workspace-recovery-preparation-dispatch.test.ts`,
`heartbeat-restart-claim-lapse.test.ts`,
`issue-reassignment-queued-run.test.ts`,
`queued-comment-consumption.test.ts`,
`issue-queued-comment-queue.test.ts`, `chat-run-milestone-work.test.ts`.

## Risks

- Refusals of runs queued for less than an hour are not backed off. A
backoff there delayed successors that wait for in-process predecessor
ownership, which can end without a task change (two CI tests failed with
it). Per trigger, such a run costs at most one claim attempt.
- A refused run that has waited more than an hour now waits up to 60 s
(a person's run 2 s) before an event-driven pass offers it again, unless
its task changes. The sweep and the redrive ignore the backoff.
- A successor may start while the run it replaces is still `running`.
The limit check leaves that one predecessor out of the count.
- Claims of an agent that is at its limit now fail inside the
transaction. A run that `executeRun` claims directly (for example a
native resume) stays queued when the agent is full, and the next pass
starts it.
- A pass that passes its 60 s deadline can still finish one in-flight
claim next to the follow-up pass. The limit check in the transaction
keeps the `running` count within the limit.
- No schema or migration change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use
and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 13:55:38 -06:00
DottaandPaperclip 5f0da562db fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connected chats show safe notices when a run starts, waits,
finishes, or fails.
> - A recurring scan projected these notices even when no eligible work
existed.
> - Eligibility depends on committed run state, conversation lineage,
and current authority.
> - Existing database rows can reconstruct that work after restart.
> - This PR wakes a scoped worker from those writes and removes
lifecycle polling.
> - Native progress events keep their separate existing owner until the
next migration.

## Linked Issues or Issue Description

**What happened?**

The chat maintenance timer repeatedly scanned run lifecycle state. Empty
and settled instances still queried the database. A run-status event
alone cannot replace the scan: prompt receipts, continuation receipts,
identity changes, and restored authority can make a run eligible later.

**What did you expect?**

Committed changes wake projection. Startup reconstructs eligible work. A
completed pass stops until another relevant change arrives. Failed work
retains recovery without scanning settled history forever.

**Steps to reproduce**

Configure an automatic chat endpoint. Finish its work, or leave it
unused. Observe recurring lifecycle projection queries from the chat
maintenance callback.

Stacked on #17592. Related foundation: #15625. This PR owns lifecycle
milestones only.

## What Changed

- Preserve upstream coalesced Cloud membership writes together with
locked membership restoration and its committed chat notification.

- Signal milestone eligibility from the real Slack account-link
transaction.

- Register the lifecycle worker with the existing delivery coordinator.
Remove lifecycle projection from the periodic chat callback. Keep native
progress in a separately named lane and preserve the explicit combined
API.
- Reconstruct company/issue scopes from the existing selector. Drain
bounded batches with company fairness. Retry failures, retain writes
through drain/resume, and disarm settled work.
- Signal from supported run, presentation, interaction, receipt,
lineage, conversation, recovery, and authority writers. Preserve their
transaction ownership, including nested savepoints and uncertain
settlement. No new table is added.
- Preserve provider authorization, authored-final suppression,
responsible-user continuity, deduplication, the 24-hour selection
window, and raw-output restrictions. Keep provider interpretation in its
owner.
- Preserve PostgreSQL timestamp precision in lifecycle pagination.
Rejected submillisecond candidates must not repeat the same page and
starve later notices.
- Document producer coverage and alternate-writer dispositions in
`doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`.

## Verification

Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting
master after #17592 merged at
`7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates
the actual landed parent. The entire 77-path milestone patch is
binary-identical to the locally verified preparation across that
ancestry reconciliation; both unrelated master paths are preserved. The
four earlier producer/test integration conflicts retain scoped milestone
and voice notices, current onboarding completion admission, and expanded
identity query/read-scope coverage.

- **330 distinct selected cases pass on the verified unchanged patch:**
305 across 14 complete suites (all five milestone worker suites, run
identity, board claim, Cloud provisioning, auth session routes, Cloud
actor authentication, stale-lock recovery, GitHub credentials, project
Git authorization, and app admission wiring), plus 25 selected real chat
producer cases. The remaining 1,590 chat integration cases were skipped,
not passed.
- The real producer cases cover saved milestone discovery, GitHub and
Slack identity changes, account-link rollback/reconfirmation, ordinary
ingress reactivation, post-close action/delivery receipt settlement, all
ten authority-restoration paths including Cloud synchronization, and
changed native review metadata.
- Worker coverage verifies outer commit, rollback, uncertain-write
recovery, restart, bounded batching/fairness, timestamp precision,
retries, drain/resume, and settled SQL quiet. Authorization and existing
publication deduplication remain with their owning services.
- Server production typechecking, build, and module-boundary checks
pass. The tests above were executed separately. Production typechecking
excludes `src/__tests__` and does not replace those runs. Each database
owner uses its own newly created and app-migrated PostgreSQL fixture.
- [Current-parent acceptance and preserved failure
ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28)
records commands, counts, producer scope, conflict dispositions, and
limits. Earlier combined-stack counts and failures keep their original
revision scope; local acceptance does not erase old CI failures.
- No new migration, queue table, dependency, workflow, or shared
scheduler contract is introduced. Parent migration 0397, bounded consent
recovery, voice cleanup/retention repairs, and runner checkpoint
restoration remain present.
- **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No
Ponytail review is present, which is an explicitly authorized
nonblocker; no failing review is waived. Older reviews and parent checks
are not current passes. Actual failing Ponytail findings must be fixed;
an absent/stopped/paused turn without a verdict is an authorized
nonblocker. Active unfinished Ponytail uses its original request/start
timer, with a 25-minute monitor cutoff respecting the user's 30-minute
maximum.
- No complete local repository matrix, live provider tenant, external
alarm, physical host/database suspension, or fleet savings claim is
made. Native progress remains a separate following slice.

## Risks

- A future eligibility writer must notify the existing projection
boundary. Independent SQL writers or another active server process are
outside this process-local signal contract.
- Authority and lineage remain sensitive. The worker uses the existing
selectors and publication policy; it does not grant execution or
disclosure authority.
- Native progress and other publication maintenance still have recurring
work. This PR removes one reason, not the entire chat maintenance timer.
- Worker quiet does not prove database suspension, host sleep, or
external alarm coverage. Cloud sleep blockers remain unchanged.
- Merge eligibility requires passing current-head CI and Greptile 5/5
after this actual-master reconciliation. Subsequent stack slices retain
their own gates.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window size are not exposed. Used repository inspection,
reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub
tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available in this session)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run the documented focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 11:07:54 -05:00
DottaandPaperclip e6dcc6b095 Connect personal Muse through secure native execution and Vault OAuth (#16448)
Add company-scoped Muse native control, OAuth, wake receiver and recovery.

Recheck current authority after waits and before effects or receipt replay. Preserve task-read scope, assignment expiry, independent holds, nonblocking reassignment and truthful cleanup provenance.

Retain source trust on progress comments and verify current Muse V8 authority through real signed-JWT HTTP requests. Preserve the upstream trimmed ordinary authentication lookup.

Refs #16448.

Preserve captured canonical queue recovery and concurrent environment preparation while retaining the scoped Muse V8 exclusion. Normal migration 0354 and existing Muse SQL remain unchanged. Fresh current-head Greptile review is 5/5 with the project-write finding addressed. Required CI and independent technical review gate the merge; maintainer override is reserved for formal approval rules.

Recheck current Muse project-write authority after all receipt/task/run/admission waits, before creation or replay. Real PostgreSQL regressions reproduce four late effects/replays on the old route; the repaired three-file treatment passes 354 cases with no skips, and the scoped server typecheck passes. Ordinary/Dot behavior is preserved.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-17 22:44:36 -05:00
DottaandPaperclip 84fe89906d fix: complete native agent review handoffs (#13581)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native execution uses durable runs, issue locks, wake requests, and
typed tool authority
> - A child can finish with a native agent review request while its
original assignee stays responsible for the work
> - The reviewer then needs a bounded execution path that can inspect
the child, record one decision, and finish safely
> - Before this change, assignee-only gates rejected the reviewer or
left the parent waiting after the child review ended
> - This pull request adds typed reviewer admission, scoped reviewer
tools, durable wake and recovery handling, and parent continuation
evidence
> - The benefit is that native review handoffs complete without changing
child ownership or granting broad mutation access

## Linked Issues or Issue Description

Refs: #13314
Refs: #13574

**What happened?**

A native child run could report `needs_review` for an agent reviewer.
The reviewer wake then failed assignee and execution-lock checks. The
child remained in review and the parent remained waiting.

**Expected behavior**

The named reviewer should receive one durable wake. The reviewer should
inspect the child and resolve the exact review card. The child assignee
should stay unchanged. The parent should receive the recorded review
outcome after the child reaches its terminal state.

**Steps to reproduce**

1. Run a native task with a different named agent reviewer.
2. Keep the child assigned to its original worker.
3. Let the worker finish with a native completion review request.
4. Start the durable reviewer wake.
5. Resolve the review and finish the reviewer run.
6. Observe the child and parent state.

**Paperclip version or commit**

Base: `e926b1301`. PR head: `b31ad9ab8`. Live reviewer verification
source: `eea171aae`.

**Deployment mode**

Built from source.

**Installation method**

Built from source (pnpm build).

**Agent adapter(s) involved**

Not adapter-specific (core bug).

**Access context**

Both.

**Database mode**

Embedded PostgreSQL in the isolated live test fixtures.

## What Changed

- Add server-validated native review assignment facts.
- Admit only the exact company, issue, source run, decision, revision,
addressee, and resolver policy.
- Give reviewer runs a narrow set of Paperclip read and resolve tools.
File and shell access follow the configured agent and environment
policy, so reviewers can run tests.
- Separate server-owned reviewer instructions from untrusted persisted
review data. Escape the data boundary; retain server-enforced
authorization.
- Keep the child assignee unchanged. Atomically claim the reviewer run,
wake request, and issue execution lock. A competing lock prevents
provider startup.
- Require the exact running reviewer session and current issue lock to
resolve its assigned card. Reject missing, unrelated, or terminal
reviewer runs.
- Add durable reviewer wake, lock, stale-card, and abandoned-run
recovery handling.
- Prevent duplicate native wake dispatches during deferred admission and
recovery.
- Carry accepted or rejected child review outcomes into parent task
context and continuation evidence.
- Add focused server, runner, and native protocol coverage.
- Preserve upstream continuation rules. Add child review decisions as
separate evidence, while keeping real human answers in their own field.
- Return actionable completion validation feedback to both providers.
Permit a corrected completion after rejection. Keep strict terminal
acknowledgment validation.
- Apply exclusive shared-workspace locks to sandbox environments. Local
and SSH folders can run concurrently, including when old settings
request serialization.
- Repair test timing, native event parsing, and the review artifact
assertion. Allow a valid reject, correct, and accept review sequence.
Check the accepted card against its reviewer run and decision. Keep
polling within the existing deadline when review acceptance precedes the
parent wake projection; report a specific missing-continuation error at
timeout.
- Apply the ACPX pending-call limit to reserved finish/block calls, with
capacity-release and cancellation tests.

## Verification

- `pnpm build`: passed on `eea171aae`.
- `pnpm -r typecheck`: passed on `eea171aae`.
- `pnpm test:e2e:runner:unit`: 359 tests passed in 30 files on
`b31ad9ab8`; runner E2E typecheck also passed.
- `pnpm check:token-gates`: passed.
- Focused DB review, reviewer authority, and prompt-boundary checks: 31
tests passed. They cover invalid reviewer runs, competing locks, atomic
admission, duplicate claims, and valid resolution.
- Heartbeat, workspace, and recovery checks: 30 tests passed.
- ACPX sidecar suite: 27 tests passed. Moving the capacity guard back
below reserved handling makes both new regression cases fail.
- Four focused live continuation checks passed on their first attempt at
`f15f55e0a`: answer updates scope (6/6 each on Codex and Claude) and
question tool guidance (12/12 each). These cases do not use the reviewer
prompt path changed afterward.
- Fresh Codex and Claude review-handoff checks passed all 29 native
checks each on their first attempt at `eea171aae`. Both runs received
the expected fixed prompt and completed cleanup. Only the six selected
live flows were tested; no full paid provider catalog run.
- The final commit only extracts the existing test-harness timeout
diagnostic into a shared helper and adds positive and negative coverage.
Removing the accepted-review guard makes two regression assertions fail;
restoring it passes all six timeout tests. Production runtime code,
prompts, deadlines, and grading criteria are unchanged by this final
commit.
- Deadline regressions: a valid continuation delayed 20 seconds succeeds
within its 30-second unit-test deadline; an absent wake returns a
specific candidate-failure diagnostic at that same deadline. Both
assertions failed before the fix. Production E2E deadlines remain
unchanged.
- Historical native failures remain recorded: Docker availability
failures; a valid reject/correct/accept sequence that the first-card
grader misread; and a test that rejected the gap between accepted child
review and parent wake projection. No failed result was regraded. The
latest tests use a protected reference to the pinned Docker image and
the unchanged artifact oracle and time limits.
- Full repository verification runs in GitHub CI. Local verification
uses the focused suites above, full build, and full typecheck. An
unchanged Codex shutdown timing test failed once in CI, passed in
isolation, and its full shard passed on the final commit without changes
to that test or its causal code path. The original failure is retained
in the verification record. Greptile reviewed `b31ad9ab8` at 5/5 with no
outstanding actionable findings. All review threads are resolved. All
current-head CI gates passed, including the isolated native runner
Docker build (55 successful checks; two skipped by the workflow).

## Risks

- Reviewer admission depends on exact persisted decision and interaction
bindings. A stale or changed card is rejected.
- Paperclip control-plane tools are limited to inspection and review
resolution. This is not a filesystem permission boundary; provider file
and shell access retain the configured policy.
- Deferred wake recovery changes dispatch receipt coalescing. A
scheduler regression could delay a continuation if the receipt state is
wrong.
- Parent review outcomes are evidence for the model. They do not grant
tool authority or change issue ownership.
- This change does not address legacy lease-hold handoff behavior.

> Roadmap review: native execution, review gates, and durable recovery
are existing roadmap capabilities. This PR completes a narrow
reliability path for those capabilities.

## Model Used

OpenAI `gpt-6-astra` with reasoning, tool use, and code execution.
OpenAI `gpt-5.6-luna` assisted with bounded implementation, review, and
journal work. Context window size is not exposed by this session. Live
test subjects use `gpt-5.6-sol` and `claude-sonnet-5`; they are not the
PR authors.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-17 15:52:19 -05:00