mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 20:29:18 +01:00
ae2afd525b52aa73d329c4ef454a7e4f2dc1362a
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
9824277f65 |
perf(run-events): append events whose only wake is chat progress in one statement (#21559)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Most runner events that are not text deltas (`tool.execution.started/progressed/completed`, `item.completed`) wake the chat progress projection > - That wake is registered inside the append's transaction, so each such event takes 4 statements (BEGIN, the run lock read, the append, COMMIT), where an event without a wake takes 1 (#21511) > - The chat progress projection rebuilds itself from the stored events; the wake only tells it when to look > - This pull request writes an event whose only wake is chat progress with the one-statement append, and sends the wake after that statement commits > - The benefit is 1 statement instead of 4 for most non-delta runner events, with response, reassessment, retention and recovery wakes unchanged ## Linked Issues or Issue Description Refs #21511 (one-statement append). Refs #21528 (lock read returns the transaction id). Refs #21549 (load model at recorded rates). **What existing behavior does this improve?** `appendHeartbeatRunEvent` for native source events whose only wake is chat progress. **Subsystem affected** Server run event ingest (`heartbeat-run-events.ts`), the chat progress notification helper, and two additive exports in `packages/db/src/work-signals.ts`. **Current behavior** A progress event opens a transaction, reads the run lock with `runtime_mode`, the bound task and the transaction id, probes for a replay, registers the chat-progress wake as a tracked intent, appends, and commits. **Proposed behavior** When chat progress is the event's only wake, the append takes the one-statement path of #21511. That statement also returns the run's `runtime_mode` and bound task id from the row it locks, and, where commit notifications are on, calls `pg_notify` inside the same statement for runs that are native and bound to a task, so other server processes get the wake at commit. After the statement returns, this process publishes the wake as settled to its own listeners. A replay, a gap, a changed run or any other fallback takes the existing transaction, which registers the wake as before. **Reason and benefit** At recorded rates these events are about 61% of the non-delta events (#21549), so this cuts about 3 statements per such event. **Breaking changes** None. No schema change. ## What Changed - `appendHeartbeatRunEvent`: `progressOnly` (progress set, every other wake clear) also qualifies for the one-statement path. - `APPEND_NEW_NATIVE_SOURCE_EVENTS_WITH_PROGRESS_SQL`: the one-statement append, plus `runtime_mode` and the bound task id (`boundRunIssueIdSql`, rendered once) from the locked row, plus a conditional `pg_notify`. - `chat-native-progress-notifications.ts`: `chatNativeProgressCommitNotification` and `publishChatNativeProgressSettled`. - `work-signals.ts` (additive): `databaseWorkCommitNotification(owner, topic)` returns the cross-process payload when notifications are on; `publishDatabaseWorkSettled(owner, topic, hint)` publishes a settled wake to this process's listeners. ## Verification - Red/green on embedded PostgreSQL. Each new test failed against `origin/master` first: - `heartbeat-run-event-atomic-append.test.ts` "appends an event whose only wake is chat progress in one statement and wakes progress after it commits": 1 statement (before: a transaction), the wake is published once as settled with no intent, and a replay publishes nothing. - "keeps the transaction for the last events of a turn and a run, and one statement for progress-only events": `tool.execution.started/progressed/completed` and a non-final `item.completed` take one statement; a final `item.completed`, `run.result.accepted`, `turn.completed`, `turn.failed`, `turn.cancelled` and `turn.interrupted` keep the transaction. - "sends the progress wake to other server processes with the one statement's commit": with commit notifications on, another connection that listens on the work channel receives exactly one chat-progress notification, and a replay sends none. - `chat-native-progress-work.test.ts` "wakes the projection from a one-statement progress event after the statement commits": the real chat progress worker publishes the progress for an event appended in one statement. - Mutation check: without the settled publish after the statement, that test fails. - Lost-wake repair (condition b): an event row stored without any wake models a lost hint. - "repairs a lost progress wake with the run's next progress event": nothing is published for the stored event; the run's next progress event publishes the progress. - "repairs a lost progress wake with the startup reconcile": a worker that starts after the stored event publishes it. - The existing response-wake test now uses a final reply and still shows the intent before the write, and the race test of #21528 uses a final reply so it still covers the transaction path. - Other focused suites, one file at a time: the port suite with conformance (105), `native-observer-replay` (1), `paperclip-semantic-conformance` (1), `native-interaction-bridge` (7), `heartbeat-run-event-sequencing` (5), `native-event-boundary-golden` (1), `heartbeat-native-restart-suspension` (115), `live-recovery-work` (51), `work-signals` (25), atomic-append (31), `chat-native-progress-work` (38). `pnpm --filter @paperclipai/server run typecheck` and `pnpm check:module-boundaries` pass. - CPU per operation (a local script, not committed: 800 `tool.execution.progressed` appends to one native task run on embedded PostgreSQL, one connection, three rounds, `origin/master` code against this branch): main-thread CPU per event 0.28–0.50 ms before and 0.10 ms after in the warm rounds; wall time per event 0.84–1.04 ms before and 0.52–0.53 ms after. - Inside a caller's transaction: new test "keeps a progress-only event inside a caller's transaction on the locked path, and settles its wake only when that transaction ends". `publishDatabaseWorkSettled` publishes at once only outside a transaction; inside one it waits for the outermost commit and drops the wake on rollback (new `work-signals.test.ts` test, which failed with the unguarded publish). The one-statement path is only taken when the owner is not a transaction (`!is(db, PgTransaction)`, #21511), so the two guards agree. ## Risks - **Lost wake.** If the statement's outcome is unknown (the connection drops after the server ran it), this process sends no wake for that event. In a chat channel (Slack, Telegram) the progress line for that task can then stay one step behind (for example it does not move from "working" to the latest tool step) until the run's next runner event, a run milestone, or a server restart. A turn normally ends with an event that keeps the transaction (`turn.completed/failed/cancelled/interrupted`), and the run's completion notifies chat run milestones, which include chat progress (`notifyChatRunMilestones` in `run-completion.ts` and `native-run-finalizer.ts`). So progress cannot stay stale after a run ends; in the worst case it lags one step for the rest of a turn. The projection rebuilds from stored events, so nothing is lost or duplicated. - The chat-progress wake no longer publishes an intent before the write. Intents let the coordinator fence a pass against an in-flight write; the settled wake after the commit makes the projection run again after the row is visible. - `packages/db/src/work-signals.ts` gets two additive exports and no change to existing behavior. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1ed0af4186 |
fix(server): wake native chat progress from committed changes (#17639)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat users receive coarse progress updates while native runs execute. > - A recurring chat pass scans saved run events even when no progress changes. > - This traffic prevents an idle database from staying idle. > - Events, lineage changes, and publication state already identify when projection can change. > - This PR gives native progress its own commit-driven worker and schedules only unfinished work. > - Idle progress stops scanning while restart and recovery preserve delivery. ## Linked Issues or Issue Description Refs #17606. This is slice 29 of the recurring-server-work stack, based on `codex/sleep-28-chat-run-milestones`. It owns native chat progress only. Related historical optimization: #16845. The parent and this PR must land in stack order. **What happened?** Native chat progress is still called by the one-second chat reconciliation loop. It repeatedly selects run events with no new work. Its cursor also truncates PostgreSQL timestamp precision, which can repeat a rejected full page forever. **Expected behavior** Committed events and eligibility changes should wake the projection owner. Existing rows should reconstruct work after restart. A settled worker should have no recurring database activity. Full batches, lock contention, and errors must retain recovery. **Steps to reproduce** Start an otherwise idle instance with chat history. Observe the native progress lane in the recurring chat coordinator. For the cursor defect, create a full page of suppressed candidates sharing a microsecond timestamp and one eligible later candidate. The old cursor fails to advance. ## What Changed - Extract native progress from lifecycle milestones into its owning service. Keep the public combined API and budget compatible. - Register a scoped worker with the existing delivery coordinator and scheduler. Remove native progress from the recurring chat coordinator. - Notify before allowlisted event writes in both persistence owners. Preserve outer commit, rollback, duplicate detection, and the ordinary non-progress log fast path. - Extend the existing lineage/authority notification boundary once. Failed-publication replay and operator retry carry their issue scope. First working publications signal progress so commit ordering cannot strand an already-inspected event. - Limit notification payloads to company/task IDs so wider caller inputs never serialize database handles or private fields. Load task-binding fields only for progress event kinds; other native/recovery events retain their narrow lock query. - Preserve current safe phrases, cadence, phase deduplication, provider policy, authorization, and final-output precedence. - Retry concrete issue/run lock contention. Disarm deleted or terminal candidates. Preserve scoped dirty work across batches, failures, drain and shutdown. - Preserve PostgreSQL timestamp precision in pagination. Add a bounded regression for the former infinite rejected-page scan. - Document the producer audit, lifecycle contract, failures, and remaining limits in `doc/plans/2026-10-25-native-chat-progress-sleep-audit.md`. ## Verification Published head: `b7ee820e49fd06c780b36b79d9b1c0883557fb73`, targeting master after #17606 merged at `5f0da562dbfdf3229f3cd4bb3dc742ab541986b4`. A normal ancestry merge preserves the complete 18-path native-progress patch byte-for-byte relative to its verified parent, with all six unrelated current master paths intact. No new production change was introduced during final reconciliation. - **163 distinct selected cases pass on the verified unchanged patch:** 137 across eight complete suites (native progress, atomic event append, all five milestone worker suites, and app admission wiring), plus 26 real chat integration cases selected by `native progress`. The other 1,595 chat integration cases were skipped, not passed. - The full worker suites cover restart, committed events, trace reprojection, duplicate/no-progress events, first-working publication ordering, concrete lock contention, issue scope, backlog, drain/resume, failures and shutdown. Settled history leaves no recurring SQL. The real chat cases verify operator retry and replay, microsecond pagination, authority, cadence, phase deduplication, final/question precedence and redaction. - The sole earlier parent conflict preserves both this slice's issue-scoped retry wake and the exact voice conversation scope. The current parent's milestone producers, onboarding completion guard, identity query/read-scope coverage, migration 0397 and voice cleanup/retention repairs remain present. - Server production typechecking, build and module-boundary checks pass. The selected server tests live under the excluded `src/__tests__` path and were executed separately. Each database owner uses a new isolated app-migrated PostgreSQL fixture. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/b7ee820e49fd06c780b36b79d9b1c0883557fb73/doc/plans/2026-10-25-native-chat-progress-sleep-audit.md#current-milestone-parent-acceptance-october-28) records current counts and scope. Older combined checks and failures retain their original revision scope and are not added to 163. No new table, dependency, workflow, migration, or scheduler contract is introduced. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing verdict is waived. Older head/parent reviews are not current passes. Actual failing Ponytail findings must be fixed. Missing/stopped/paused turns without a verdict are authorized nonblockers; active unfinished reviews retain the original per-head timer with a 25-minute monitor cutoff respecting the 30-minute maximum. - No complete local repository matrix, live provider tenant, external wake, physical host/database suspension or savings claim is made. Following stack duties remain separate. ## Risks - Signals are process-local under the existing single active writer contract. Independent SQL writers or another active process need an explicit wake integration. - Allowlisted direct log writes now use a transaction. Native bound runs register transaction intent; unrelated direct event types retain their atomic-statement path. - The shared eligibility boundary wakes both lifecycle and native projection. It does not decide provider policy or copy event payloads. - The existing 24-hour selection window, event-time cadence and authority checks remain. A suppressed event is not retried just because wall-clock time passes. - This establishes worker quiet, not whole-instance suspension. Cloud alarm coverage and conservative sleep gates do not change. Other chat maintenance remains on its existing cadence. - This actual-master reconciliation still requires fresh exact-head review and CI before protected merge. Subsequent slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code execution and browser-test tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |