Files
PaperClipAI/server
DottaandClaude Opus 5.5 db8848f559 fix(heartbeat): bound the I7 queued-run check with a resumable cursor (#21022)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - #20986 added a reconciler check that reports a queued run whose
dispatch signal was lost (I7 `queued_run_not_dispatched`), and a worker
that re-drives such runs.
> - The Ponytail review of #20986 (4/5 on `fbab23cce6`) found three
issues. #20986 merged at that head before the fix commit was pushed.
> - The new I7 check read every queued run with no cap, while the report
still said that each I7 check reads at most 2,000 records.
> - This pull request adds that fix commit to master. The check now
reads a bounded slice per pass and continues from a cursor. The page
cursor no longer depends on the session date style. An agent with a
scheduled retry now counts as busy.
> - The benefit is that the reconciler pass stays bounded when many runs
are held on purpose, and the coverage statement is true again.

## Linked Issues or Issue Description

Refs #20986

Follow-up to the Ponytail review of #20986:
https://github.com/paperclipai/paperclip/pull/20986#pullrequestreview-5868879754
(review of `fbab23cce6`, score 4/5).

**What is wrong?**
1. `checkQueuedRunsDispatched`
(`server/src/services/recovery/ownership-invariant-report.ts`) paged
through every undispatched queued run with no cap.
`COVERAGE_GAPS.record_cap` still said each I7 check reads at most 2,000
records per pass.
2. The page cursor cast `created_at::text` back with `::timestamptz`.
That works only with the default `DateStyle`.
3. A queued run on an agent with a `scheduled_retry` run was reported
and re-driven, but `checkVisibleNextAction` already treats
`scheduled_retry` as an active run.

**Expected behavior**
The I7 queued-run check reads a bounded slice per pass and reaches every
run over later passes. The cursor format does not depend on session
settings. A scheduled retry counts as the agent's next action.

## What Changed

- `server/src/services/queued-run-dispatch-liveness.ts`: new
`readUndispatchedQueuedRuns`. It reads from an optional cursor until the
set ends or `maxRuns` rows were read, and returns the next cursor.
`listUndispatchedQueuedRuns` uses it with no limit. The cursor text is
`to_char(created_at at time zone 'UTC',
'YYYY-MM-DD"T"HH24:MI:SS.US"Z"')`, which keeps microsecond precision and
does not depend on `DateStyle`. The live-run exclusion now also covers
`status = 'scheduled_retry'`.
- `server/src/services/recovery/ownership-invariant-report.ts`: the I7
queued-run check reads at most `MAX_RECORDS_PER_KIND` (2,000) per pass.
It keeps a keyset cursor in `OwnershipReportState.queuedRunCursor`, and
the next pass continues from that cursor. A pass that stops at the cap
keeps a follow-up pass due (`retryNeeded`), like the I3 and I4 backlogs.
`COVERAGE_GAPS.record_cap` now describes this check.
- `server/src/__tests__/ownership-reconciliation-report.test.ts`: new
test. Bounded slices resume from the cursor and end with `next: null`. A
`scheduled_retry` run on the agent removes its queued runs from the set.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/ownership-reconciliation-report.test.ts
src/services/queued-run-dispatch-liveness.test.ts
src/services/heartbeat/dispatch-work.test.ts
src/__tests__/heartbeat-held-comment-wake.test.ts`: 75 tests pass on
this branch (fresh master + this commit).
- Server `tsc --noEmit` passes.
- The restart integration test from #20986 ("starts a continuation that
queued behind its predecessor's sandbox release after a graceful
restart") passed with this change on the #20986 branch, in both cases
(signalled and unsignalled).
- Restart-chaos CR1 360 s, Codex W2, `--daytona-warm
--ephemeral-server`, seed 9512 (`w-20261112T040314-s9512`, on the #20986
change set before this commit): the fault crossed the task and every
rule passed (R1–R6, T, U1–U5).

## Risks

- Low risk. The report reads less per pass. A very large held queue is
now covered over several passes instead of one.
- A queued run behind a `scheduled_retry` is no longer re-driven by the
backstop. The scheduled retry's own promotion signals dispatch.
- No schema change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 05:19:37 -06:00
..