Files
PaperClipAI/server/src/__tests__
DottaandPaperclip 8252974141 fix(server): schedule stopped instruction collection (#17674)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Runs can stop while their private instruction edits still need
collection.
> - Existing copy rows and stop evidence describe that unfinished work.
> - The orphan reaper still queries for it on every maintenance pass.
> - This PR activates collection from committed copy and run
transitions.
> - It uses saved retry deadlines when stop proof or collection remains
pending.
> - Active copies and completed history no longer cause collection
polling.

## Linked Issues or Issue Description

Refs #17671 and #17670. Both prerequisites are merged into master.
Captured saves and post-save release remain separate reasons with their
own workers.

**What needs to be improved?**

`recoverStopped` queries instruction copies on every orphan-reaper pass,
even when no terminal run owes collection. A run can become terminal
without changing its copy row, so copy-only notifications would miss
real work.

**Why is this improvement important?**

Repeated empty queries prevent idle tenant instances from becoming
quiet. Removing them must preserve edits, stop proof, current ownership,
and recovery after restart.

**Proposed solution**

Use the existing transaction signals, scoped deadline worker, and
scheduler. Reconstruct eligible copies at startup, signal supported copy
and terminal-run writers, and retain saved retry deadlines for
unresolved work. Preserve the existing collection and directory owners.

Related PR search found the warm agent-file ownership work (#14695),
verified stop boundary (#14311), and task-workspace cleanup (#16464).
This PR does not replace those contracts. ROADMAP.md has no duplicate
migration.

## What Changed

- Add scoped stopped-instruction collection with keyset reconstruction
and fair batches of 20.
- Signal before copy preparation, mutation and adoption writes, and
before terminal run writes in their existing transactions.
- Cover heartbeat, native finalization/restart/session recovery,
execution-control, queue and dispatch producers.
- Signal settings changes and company/agent deletion. Preserve drain,
standby, restart and shutdown ownership.
- Recheck terminal-run eligibility in the due-time claim. Keep terminal
status separate from process-stop proof.
- Retain existing stop checks, attempt limits, saved candidate bytes and
remote-resource fencing.
- Skip only this lane in periodic orphan recovery. Preserve
manual/startup recovery.
- Add concurrent, replay-safe partial index 0408 and retain the five
newest snapshots. Add no table.
- Record the owner and producer audit in
`doc/plans/2026-10-25-stopped-instruction-collection-sleep-audit.md`.

## Verification

- Source `cb2818a8f46ef961f744977a258f0e9042e31a42`: all 1469 cases
across 14 complete files pass without skips in 141.69 seconds. This
includes all 968 native-session cases, filesystem saves, startup,
terminal-run producers, dispatch, queue, execution-control, finalization
and lifecycle.
- Eleven separate fresh PostgreSQL 17 databases migrated through 0408.
The embedded-only lifecycle fixture temporarily used its own external
database on this IPC-exhausted host and was restored byte for byte.
Assertions and timing limits stayed unchanged. This qualifies
external-database behavior; it does not claim an unmodified embedded CI
run.
- Database build, server typecheck/build, module boundaries and
exact-current migration order pass. The original concurrent index SQL is
unchanged. All prior executable migration history, journal entries and
retained snapshot bytes are preserved; the five newest snapshots are
0404 through 0408.
- A populated 0407-to0408 upgrade preserves all 406 prior migration
receipts and 38 companies. It appends exactly one receipt. Replay adds
none. The collection index is valid and ready on fresh and upgraded
databases.
- The normal merge of actual parent
`267842fc3c6e6542deed8a1bcc2c78bec8df7862` preserves both complete
patches, all independent sidebar changes from #18117, and the full
accepted server/package/test trees. The complete sidebar test file
passes all 36 cases on the combined tree. React act warnings remain.
- Prior failures and fixture qualifications remain in the audit.
Published head `d176d731f083c7edd2ede688cb901d1e66d8f1d8` has Greptile
5/5, all 57 checks passing or skipped, and no unresolved review threads.
No Ponytail review is present. These local tests do not establish
deployed host/database suspension or Cloud alarm delivery.

## Risks

- Process-local signals cover instrumented writers. Independent SQL
writers and additional replicas still need integration.
- Terminal-run signals can cause one scoped inspection even if no
instruction copy exists. They do not create a recurring scan.
- Reconstruction and memory scale with actual pending candidates; active
candidates are filtered by the run join.
- Admitted file/provider operations can outlive a later drain request.
Shutdown joins them. Uncertain ownership remains pending rather than
being discarded.
- The partial index adds storage and write cost. Its concurrent
migration uses the existing interrupted-index repair path.
- Cloud alarms, sleep eligibility and provider suspension are unchanged.
Worker quiet is not a claim of deployed host/database sleep or measured
savings.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, shell tools, and
PostgreSQL/filesystem tests. The runtime does not expose a more precise
model ID or context-window size for this turn.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 22:13:57 -05:00
..
…

Server Tests

Server tests that need a real PostgreSQL process must use ./helpers/embedded-postgres.ts instead of constructing embedded-postgres directly.

The shared helper creates a throwaway data directory and a reserved-safe loopback port for each test database. This protects the live Paperclip control-plane Postgres from server vitest runs; see PAP-2033 for the incident that introduced this guard.