PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-29 21:07:40 +01:00

Author	SHA1	Message	Date
DottaandPaperclip	65fa1cd617	fix(server): schedule GitHub review checks from committed changes (#17662 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/213/changes/paperclip/213 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review checks report the state of admitted agent work. > - The chat maintenance loop searches for check work even when none remains. > - Existing action rows already record unfinished publication and retry deadlines. > - Committed changes can wake those rows through the shared coordinator. > - An unresolved check must still detect a remote head change when a webhook is missed. > - This pull request schedules only outstanding checks and leaves settled history quiet. ## Linked Issues or Issue Description What happened? The GitHub chat maintenance callback repeatedly queries review-check actions. Empty queues and completed history still incur database reads. Local state changes wait for the maintenance callback. Expected behavior Committed check creation, review binding, run changes, and publication-lease release wake the worker. Existing retry and remote-head deadlines apply only to unfinished checks. A settled worker makes no recurring database or provider calls. Steps to reproduce Configure a GitHub bot with no outstanding checks, or finish all of its checks. Leave the instance idle. The old maintenance callback continues to call the check service. Refs #17661, the merged terminal-response parent. Related #17656 migrates response acknowledgements. Related #16385 changes branch-policy admission; this PR preserves that separate policy boundary. Late stale-head projection repair remains a separate follow-up slice. ## What Changed - Move the existing check-state reader into the GitHub provider owner. Preserve current authority, run selection, assessment ownership, and check replacement behavior. - Register a scoped worker with the existing coordinator and scheduler. Existing received and retryable-failed actions remain durable truth. No new table is added. - Signal check enqueue, task review receipts, manual review insertion/binding, run lifecycle changes, and publication-lease release through existing transaction ownership. - Preserve the saved failure backoff, eight-attempt policy, and competing lease expiry. Keep five-second remote-head reconciliation only while a received check remains unresolved. - Check admission before each HTTP request. Shutdown leaves durable work recoverable without spending a provider failure attempt. - Remove check dispatch from the periodic GitHub callback. Keep the separate late-projection repair and the manual combined recovery method compatible. - Add an idempotent concurrent partial index for unfinished check selection. Retain the newest five generated snapshots. - Add eleven real-database worker cases and startup wiring coverage. Document producer coverage and the external reconciliation limit. ## Verification Current reviewed source: `2bba08403b195bf646eda48992b46d6f5cebf756`. - Normal merges preserve the complete review-check patch, the landed terminal-response parent, and current master changes. No landed SQL migration or journal entry changes. - At `6995f84b240a14a7b8660e1cf095b3aa1e1e3168`, 331 distinct tests pass: 264 complete GitHub workflow cases, 30 completion cases, eight issue-tree producer cases, 16 login-cleanup cases, and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. Login cleanup uses its real embedded PostgreSQL fixture. - Combined-master acceptance: 277 distinct tests pass at `9c538aed1251fd7f7cecb829a0ecbce6110fb1ed`: 264 complete GitHub workflow cases and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. A new PostgreSQL database receives all migrations through 0402.. - Migration `0402_ancient_malice.sql` preserves the original partial index definition and concurrent idempotent creation. All 400 prior migration receipts stay unchanged during upgrade. One new receipt is added. Replay adds no receipt. The index is ready and valid on fresh and upgraded PostgreSQL databases. The five newest snapshots are retained. - A rollback-only query probe adds 10,000 processed actions and one pending action. The actual parameterized candidate predicate selects the one pending row through the partial index, with two shared buffers and 0.007 ms execution time. This measures candidate discovery, not all lease or provenance reads. - Database build, server typecheck and build, module boundaries, migration safety and migration order pass. Audit: [GitHub review-check scheduling](https://github.com/paperclipai/paperclip/blob/2bba08403b195bf646eda48992b46d6f5cebf756/doc/plans/2026-10-25-github-review-check-sleep-audit.md). - Fresh checks and Greptile review are required for this head. Prior scores do not count. These checks do not prove deployed host suspension or fleet savings. ## Risks - Unresolved queued/running checks still make remote-head requests at the existing five-second deadline. GitHub webhook receipt does not prove future pushes cannot be missed. This PR does not claim active reviews become network-idle. - Run and configuration hints rebuild outstanding checks for the affected company. Enqueue, review, and lease hints narrow to the affected endpoint/repository/PR. Startup and scope reconstruction read existing pending rows; they are not constant-cost operations. - Delivery admission failures remain discoverable through the unresolved check's external reconciliation deadline. Not every shared delivery mutation gains a new immediate signal. - Notifications use the existing process-local database ownership contract. Independent SQL writers and other replicas need their own supported integration. - Provider markers, publication leases, and saved actions preserve recovery after uncertainty. This change does not expand Cloud alarm coverage or prove deployed host/database suspension. - Each remaining sleep slice needs its own review and CI checks before merge. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context-window size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 17:08:27 -05:00
DottaandPaperclip	f8226b0515	fix(server): wake GitHub terminal responses from committed changes (#17661 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/210/changes/paperclip/210 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip coordinates agent work and presents results in chat. > - GitHub acknowledgements already persist the response action and provider receipt. > - A terminal run can still owe a final response after acknowledgement. > - Periodic maintenance searches those rows even when no relevant state changed. > - Committed run, issue, presentation, lease and publication changes identify when recovery can advance. > - The owner should recover from those events and real deadlines, then become quiet. ## Linked Issues or Issue Description Refs #17656, which is merged. This existing terminal-response slice targets master. Problem: A saved GitHub response can remain nonfinal while its run, native presentation, publication or source is unresolved. Removing maintenance without covering those producers loses recovery. Polling settled history keeps the tenant active. Expected behavior: Committed supported changes wake the terminal owner. It reconstructs existing actions, preserves provider authority and pending publication ownership, waits at saved deadlines where needed, and becomes quiet after settlement. ## What Changed - Register terminal GitHub response work with the shared scoped deadline coordinator, using existing action/delivery/publication/run rows. - Notify response availability from supported run milestones, native identity, recovery resolution, issue updates and tree/workspace restoration writes. Preserve outer transaction commit/rollback. - Keep GitHub response interpretation in its platform owner. Use the bounded run projection for both direct authorization and scheduling, including native presentation guards and deliberate passive outcomes. - Retain the existing publication lease, markers, source/principal checks and final receipt precedence. A publication owner or uncertain outcome keeps its existing authority; the worker does not replay it. - Remove only recurring terminal-response recovery, preserving the manual path and neighbouring duties. Add no table or scheduler service. - Regenerate the original sparse terminal-work index as `0401_yummy_ultimates.sql`, concurrently and idempotently through the existing top-level migration runner. Preserve all prior SQL/journal entries and the five newest snapshots. ## Verification Published head: `88a209e28b969b5229e9a621aeddfd3cbae9b0cd`, integrated with landed #17656 (`ba5bba86dcdde32b14d4d7dec1b51898e5f6f22d`) and master `52b86c2032a189b0ea259c5b0cb7158b07ef788c`. - 74 distinct current cases pass: 17 terminal-response lifecycle/producer cases (1,721 other chat cases skipped), all six queued-comment consumption regressions for the new upstream query optimization, all 30 heartbeat completion cases, all eight real/unit issue-tree control cases and all 13 startup cases. Database build, server typecheck/build, module boundaries and four fresh migrations through 0401 pass. - The earlier 304-case prepared-parent matrix passes at `aa72f32641b2e2c680342cd49477272349aca9db`: all 253 GitHub workflow cases, 30 completion cases, eight issue-tree cases and 13 startup cases. It retains that revision scope. Repeated cases are not added to the current 74. - The terminal matrix covers finalized-history quiet, deleted Board sources, pending presentation/passive outcomes, outer rollback/commit, paused/new runs, restart/drain/standby, committed native identity, lease waits, full-page unrelated history, uncertain publication ownership and tree/workspace/exhausted-recovery producers. - A populated 0400 database upgrades to 0401 with all 399 prior migration receipts identical, exactly one added, and none added on replay. The terminal index is valid/ready on fresh and upgraded databases. Its SQL matches the original draft after whitespace normalization; generation keeps snapshots 0397–0401. - Normal integration `3c8339a541bf68bc9957b9d5a8df80d3a7c3452d` preserves the complete 24-path owned binary patch and all five new master paths. The landed parent tree also equals the previously published parent tree byte for byte. Both normal merge parents and the exact composition tree are checked. No force push or rebase occurs. - [The owner audit](https://github.com/paperclipai/paperclip/blob/88a209e28b969b5229e9a621aeddfd3cbae9b0cd/doc/plans/2026-10-25-github-terminal-response-sleep-audit.md#acceptance-on-the-landed-github-parent-and-current-master) preserves the original producer, quietness, failure-isolation, skipped-suite and presentation-wait results with their scopes. Current passes do not erase earlier failures or turn skips into passes. - The audit-only follow-up leaves all owned source/test/migration hashes unchanged. Fresh exact-head CI and Greptile 5/5 with no actionable findings remain required. No full local repository suite, live GitHub tenant or deployed sleep result is claimed. ## Risks - Process-local notifications cover supported application writers and shared pools. Independent SQL writers and other replicas gain no new wake guarantee. - Native terminal status alone does not authorize fallback publication. Pending presentation and existing publication ownership remain separate obligations. - Stop can leave an in-flight provider request settling; admission fences stop a subsequent request. Unknown outcomes retain their existing recovery rules. - Index creation can take time on large histories. Concurrent/idempotent migration and existing replay recovery remain in charge. - Worker quiet does not establish host/database suspension. Cloud alarm coverage and sleep admission remain unchanged. ## Model Used OpenAI Codex, using reasoning, repository inspection, code execution and real PostgreSQL tests. The exact model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 16:51:57 -05:00
DottaandPaperclip	ba5bba86dc	fix(server): wake GitHub acknowledgements from committed changes (#17656 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/206/changes/paperclip/206 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip coordinates agent work and presents its progress in chat. > - GitHub deliveries save acknowledgement actions and provider receipts. > - Periodic maintenance currently searches for unfinished acknowledgements even when idle. > - Committed delivery, lease, response and presentation changes identify when work can advance. > - The existing owner should wake on those changes and recover only at saved deadlines. > - Completed history should become quiet without losing unfinished provider effects. ## Linked Issues or Issue Description Refs #17653, which is merged. This existing GitHub acknowledgement slice targets master. Problem: A saved GitHub acknowledgement can wait for a publication lease, fast-response result or native presentation decision. Polling empty or unchanged work keeps the tenant active. Removing maintenance without covering these producers can strand the acknowledgement. Expected behavior: Committed changes wake the scoped owner. It reconstructs existing actions on startup, retries at saved deadlines, parks unchanged presentation waits and becomes quiet when settled. ## What Changed - Register GitHub acknowledgement work with the shared scoped deadline coordinator. Reuse existing action, delivery, lease and fast-response rows. - Add transaction-aware response-availability notifications at supported writes. Keep GitHub protocol details in its platform owner and shared notifications limited to IDs. - Share the bounded latest-request/run projection between scheduling and direct response authorization. An absent native presentation decision parks the action until its real writer commits. - Retain publication leases, source/principal authority, version fencing, response markers and final receipt precedence. Check admission before each new provider request; let in-flight work settle. Stop does not consume the provider retry budget. - Remove only recurring acknowledgement recovery and keep its manual path. Add no work table or scheduler service. - Regenerate the three original indexes as `0400_absent_phantom_reporter.sql`: pending acknowledgement IDs, fast-response delivery lookup and publication comment/destination lookup. Each uses concurrent/idempotent creation and the existing top-level migration runner. Preserve all landed SQL/journal entries and the five newest snapshots. ## Verification Published head: `09ca00b9f029838434ec042cda691345d67b493e`, on landed continuation parent `063cfd43813cabf18417605fb9eefb9aacf95c8d`. - 87 distinct current cases pass: 18 real PostgreSQL acknowledgement lifecycle/presentation cases, all 26 fast-response service cases, all 30 heartbeat completion cases and all 13 startup cases. Database build, server typechecking/build, module boundaries and three fresh migrations through 0400 pass. The current Runner bundle rebuild also passes after upstream provider-event changes. - The lifecycle matrix covers empty/history quiet, lease release, restart, drain/standby, fast-response failure/expiry/publication/unknown outcome, streaming quarantine, outer rollback/commit, saved provider retry, stop between requests and receipt state. The real presentation writer proves quiet while pending, rollback, committed reactivation and final quiet. - The earlier 305-case matrix (236 full GitHub workflow cases, 26 fast-response service cases, 30 completion cases and 13 startup cases) passes at `337f272853d4f8a51b5b9300900a46eb4e13f698`. It retains its prepared-parent scope. Repeated cases are not added to current 87. - A populated 0399 database upgrades to 0400 with all 398 prior migration receipts unchanged, one new receipt and no additional receipt on replay. All three indexes are valid/ready on fresh and upgraded databases. Their SQL definitions match the original draft after whitespace normalization; migration history and retained snapshot bytes are preserved. - Normal integration `9b3607e159153e00ccdfe538d52b591bd38d6499` preserves the full 28-path owned binary patch and all 20 unrelated master paths, including native retry, provider-event and evaluation workflow changes. Both normal merge parents and the exact tree are checked. No force push or rebase occurs. - [The owner audit](https://github.com/paperclipai/paperclip/blob/09ca00b9f029838434ec042cda691345d67b493e/doc/plans/2026-10-25-github-chat-maintenance-sleep-audit.md#acceptance-on-the-actual-landed-continuation-parent) retains the original missing-notification, JSON precedence, receipt lifecycle, external-database fixture and migration-registry failures. Earlier 69/228/broad counts keep their original scope. Skips and failed revisions are not relabelled as current passes. - The final audit-only edit leaves all owned source/test/migration hashes unchanged. Fresh exact-head CI and Greptile 5/5 with no actionable findings remain required. No full local repository, live GitHub tenant or deployed sleep result is claimed. ## Risks - Process-local transaction hints cover supported application writers and shared pools. Independent SQL writers and other replicas do not gain a new wake guarantee. - Provider effects remain guarded by the existing publication lease and current source/principal authority. Unknown outcomes retain their existing recovery semantics. - Presentation waits depend on their supported milestone/transaction producers. A missing producer can defer recovery until another supported change or restart. - Index creation can take time on large histories and runs concurrently outside a transaction. Existing migration recovery handles replay; prior SQL is unchanged. - Worker quiet is distinct from host/database suspension. Cloud wake coverage and sleep admission remain unchanged; later maintenance duties are separate slices. ## Model Used OpenAI Codex, using reasoning, repository inspection, code execution and real PostgreSQL tests. The exact model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 16:06:42 -05:00
DottaandPaperclip	063cfd4381	fix(server): wake interaction continuations from committed responses (#17653 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/204/changes/paperclip/204 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip coordinates agent work and human approvals. > - Human decisions already persist the continuation action that resumes an agent. > - A response can commit before its callback or publication finishes. > - Periodic maintenance repeatedly discovers those rows, including when idle. > - Committed writes should wake the existing owner; saved retries and claims supply real deadlines. > - Startup reconstructs durable work, and settled history becomes quiet. ## Linked Issues or Issue Description Refs #17651, which is merged. This existing continuation slice targets master and removes its duty from periodic chat maintenance. Problem: A saved `interaction_wakeup` action can survive a committed human response whose callback crashes. Without producer coverage, removing maintenance strands the continuation. Repeated empty maintenance also keeps the database active. Expected behavior: The outer transaction's settlement wakes the continuation owner. It handles ready conversation publications, dispatches under existing authority, preserves interrupted/retry work, and becomes quiet after settlement. ## What Changed - Register continuation recovery with the shared scoped deadline coordinator. Reconstruct existing issued/processing actions and retain the existing retry and stale-claim deadlines. - Notify from both enqueue producers and owner-local claim/settlement writes. Preserve outer commit/rollback and avoid moving provider or heartbeat calls into database transactions. - Give the existing publication queue an optional company/endpoint/conversation scope. The continuation owner gives ready cards in that conversation a bounded delivery opportunity before claiming the action; FIFO, capacity, ambiguity and provider authority remain with the queue. - Carry admission/cancellation through publication preparation and check before every new send. Join in-flight requests and retain pending actions/publications through drain, standby, shutdown and resume. - Remove only recurring continuation recovery; keep the manual API. Add no table. A concurrent partial index on existing pending continuation actions and matching company/id keyset paging prevent retained history from being scanned. Parent #17651's indexed confirmation discovery and current master are retained. ## Verification Published head: `a1d16f84b3fae361d5d648670a5964486370c3a3`, on actual landed terminal-confirmation parent `ee67d2ece9bda157dfad9bdf25175b1d004ed341`. - 75 distinct current cases pass: 32 selected real PostgreSQL chat cases (14 continuation lifecycle/producer/stop cases, one new continuation query-cost case, and 17 parent confirmation/query-cost cases; 1,671 skipped), and all 43 cases in complete interaction-publication and app/startup suites. Database build, server typecheck/build, module boundaries and fresh migrations through 0399 pass. The original landed-parent matrix passed 74 cases before the additional query-cost repair; those are not added again. - The new real-SQL regression seeds 20,000 completed/unrelated actions and one pending processing claim. Before repair it examines 20,345 rows and hits 20,421 buffers. The final startup and company-scoped queries must examine at most two action rows and fewer than 20 action buffers; empty scoped recovery must also stay below 20 buffers. Planner settings are unchanged. Migration `0399_ambitious_onslaught.sql` is concurrent/idempotent and uses the existing top-level migration runner; prior SQL/journal entries and the five newest snapshots are preserved. - The matrix covers callback recovery, concurrent responses, malformed prefixes, more than one page, saved retries/stale claims, outer rollback/commit, deferred/future/paused publication states, restart, drain and stop before each new provider request. Quiet assertions inspect real selects/transactions across virtual idle time. - Normal integration `db5a84c0f8b73a0ce863f4d2fbaee0a115cf044f` preserves the complete 12-path owned binary patch and all 12 unrelated current-master paths, including managed Cloud sandbox changes. A verified semantic base resolves squash-ancestry conflicts; both normal parents and the exact tree are checked. The subsequent index/keyset repair is a deliberate additional change, tested separately; the final patch is not claimed identical to the original 12-file implementation. No force push or rebase occurs. - The same 74 pre-index cases passed on the earlier prepared and actual landed parents. The new query-cost regression failed before its repair, then passes on the repaired source. Earlier 57-case and broad acceptance retain their original revision scopes; repeated cases are not added to the current count. The audit-only follow-up leaves all owned server source/test hashes unchanged. - [The owner audit](https://github.com/paperclipai/paperclip/blob/a1d16f84b3fae361d5d648670a5964486370c3a3/doc/plans/2026-10-25-interaction-continuation-sleep-audit.md#acceptance-on-the-actual-landed-terminal-confirmation-parent) retains the original missing-signal regression, the skipped standalone publication suite, and the review's stop-before-send failures with their corrections. Skips and earlier failing revisions are not relabelled as current passes. - Fresh exact-head CI and Greptile 5/5 without unresolved actionable findings remain required. The former remote head's reviews are stale. No full local repository suite, live provider or deployed sleep result is claimed. ## Risks - Process-local transaction hints cover supported application writers and shared pools. Independent SQL writers and other replicas do not gain a new wake guarantee. - The publication queue's existing FIFO, capacity, retry/ambiguity and authorization contracts still govern effects. In-flight provider requests can settle after Stop; the fence prevents a subsequent request from starting. - Missing a supported producer can defer recovery until another change or startup. Producer and rollback tests cover this slice's supported paths. - Worker quiet is separate from host/database suspension. External alarm coverage and Cloud sleep gates are unchanged. ## Model Used OpenAI Codex, using repository inspection, code execution and real PostgreSQL tests. The exact model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 15:45:18 -05:00
DottaandPaperclip	ee67d2ece9	fix(server): wake terminal confirmation recovery from committed changes (#17651 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/202/changes/paperclip/202 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip persists confirmation clicks and interaction decisions before dispatch completes. > - A crash after a decision commits can leave a processing confirmation action. > - Periodic publication maintenance repeatedly scans those actions even when idle. > - Saved decisions and the supported board/admin writers already identify when settlement can change. > - This owner reconstructs durable actions, reconciles real changes and then becomes quiet. ## Linked Issues or Issue Description This existing terminal-confirmation slice follows merged #17649. It removes only terminal-confirmation recovery from periodic chat maintenance while retaining the explicit manual API and existing durable action/interaction/publication rows. Problem: An accepted response can commit before its dispatch settles. Removing periodic recovery without covering board decisions, expiry and ownership changes would strand processing actions. Pending human decisions must remain recoverable without polling indefinitely. Expected behavior: Committed supported changes wake the scoped owner. It reconciles terminal responses once, parks unresolved human decisions, preserves concurrent commits/restart/drain and becomes quiet when settled. ## What Changed - Register terminal-confirmation work with the existing delivery coordinator and scoped lifecycle. Reconstruct processing action IDs, bound selection pages, preserve concurrent wakes and keep current authorization in the existing interaction owner. - Add the existing transaction-aware confirmation notification topic at real resolution writes and callback settlement. Reuse milestone signals instead of adding per-caller recovery wrappers. - Cover ordinary and checkbox confirmations, suggestions and connection intents through their owning service. Cover board accept/reject, cancellation/withdrawal, opt-in comment supersession, expiry, reassignment and already-satisfied connection resolution. - Preserve task/actor authority, one-use callback tokens and immutable provider publication semantics. Explicitly fail malformed missing bindings instead of silently retaining unrecoverable processing actions. - Remove only this duty from recurring publication maintenance. Add no table or scheduler infrastructure. A concurrent partial index on existing action rows prevents recovery discovery from scanning completed history. ## Verification Published repair head: `a35e78d076c1b618bc643391b7520fa6952fc996`, targeting master after normal integration of `644937c93150c423f12674992c7ea1e8b5dbc98c`. [Detailed acceptance and preserved failures](https://github.com/paperclipai/paperclip/blob/a35e78d076c1b618bc643391b7520fa6952fc996/doc/plans/2026-10-25-terminal-confirmation-sleep-audit.md#bound-retained-history-reads--october-28-review-repair). - New query-cost regression: Capture the actual SQL emitted by startup and company-scoped confirmation discovery. Seed 20,000 completed/unrelated action rows and one pending human decision. Before the index, the regression fails because recovery examines 20,009 action rows. After the partial index, startup and scoped discovery must examine at most two action rows and fewer than 20 table/index buffers; the no-pending scoped path must also stay below 20 buffers. Planner settings are unchanged, and returned IDs remain correct. - Match startup keyset ordering to `(company_id, id)`, the partial index order. The index alone still let PostgreSQL choose the primary key and scan 20,206 rows for `ORDER BY id`; the final query fixes this without forcing a planner setting or adding a second index. Fixed kind/status predicates stay literal for prepared-plan index eligibility. - Add the generated `0398_yielding_random.sql` migration, using `CREATE INDEX CONCURRENTLY IF NOT EXISTS` and the existing top-level migration runner. Keep the five newest generated snapshots and preserve every prior SQL/journal entry. No work table is introduced. - 110 distinct cases pass: 17 final confirmation lifecycle/query-cost cases (1,671 skipped), seven real Runner descriptor variants (295 skipped), and all 86 app/startup/bundled-plugin/plugin-database cases. Server/database typechecking and builds, migration safety/numbering, module boundaries and a fresh PostgreSQL migration through 0398 pass. The final test-only vacuum precondition is followed by another 17-case confirmation pass and server typecheck. Counts are not added across repeated runs. Normal integration of master `644937c93150c423f12674992c7ea1e8b5dbc98c` preserves the complete owned patch and all 18 unrelated master paths. - Previous landed-parent acceptance: 29 distinct cases (16 confirmation lifecycle/producer cases with 1,671 skipped, plus all 13 startup cases) passed on `32e64a7e9ea5b457e54ca7b268be90e1e155b451`; the earlier 80-case matrix remains scoped to `c1e8542b588a4fa59b62f9829541d3daea4f1ddc`. Counts are not added across repeated runs. - Actual producer cases prove parking/quiet, caller-owned outer transaction rollback, commit-driven automatic settlement, restart/drain and concurrent commits. Checkbox, suggestion, connection-intent and Telegram/Slack callback paths retain existing authority and immutable publication semantics. - Fresh exact-head CI and Greptile 5/5 with no unresolved actionable findings are required. The preceding head's Runner CI failed its timeout-attachment durable-suspension case (1 failed/1,955 passed/4 skipped); that failure is retained and not counted as a pass. The new head must pass this job. No full local repository, live provider or physical suspension claim is made. The owner audit retains the earlier missing-board-producer regressions and their corrections, including the invalid supersession fixture that assumed a new checkbox expires an older one. Existing behavior was preserved by using the supported opt-in human-comment path. Prior scheduler/backoff, route and plugin test failures remain at their original revisions; no old pass is substituted for current remote gates. ## Risks - Process-local notifications cover supported application writers and their shared pools, not independent SQL writers or other replicas. - Processing actions waiting on a human response are deliberately parked. Supported interaction resolution/expiry writers must use the notification boundary; a missing producer delays settlement until restart. - Invalid bindings have explicit failure evidence. Source, actor, task and publication authority remain checked by their existing owners. - Following continuation/GitHub duties remain separate. Cloud sleep gates and external alarms are unchanged; local worker quiet does not establish physical host/database suspension or fleet savings. ## Model Used OpenAI GPT-6 / Codex, with reasoning, repository inspection, code execution, and PostgreSQL tests. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 15:00:31 -05:00
DottaandPaperclip	f4f02880ca	fix(server): wake failed retry notices from committed changes (#17649 ) ## Thinking Path > - Paperclip keeps exact failed chat retries in durable action and wake receipts. > - Deferred and declined retries owe safe notices to their original conversation. > - Periodic publication maintenance repeatedly inspects those receipts when idle. > - Committing receipt, authority, lifecycle and lease changes already identifies when eligibility can change. > - A scoped owner reconstructs durable obligations, drains bounded candidates, retains contention, then becomes quiet. ## Linked Issues or Issue Description This existing failed-retry notice slice follows #17647. It removes the failed-retry notice duty from periodic publication maintenance while preserving the explicit manual API. Existing action, receipt, publication and source rows remain authoritative; no new table or scheduler is added. Problem: Removing the scan alone loses notices after real admission, cancellation, early refusal, requester or agent restoration, and relevant lease settlement. Millisecond cursor truncation can also repeat denied history. Expected behavior: Committed supported writes wake the scoped notice owner, which uses current authorization and precise cursors. Real retries survive failure, contention, restart and drain; settled work has no recurring SQL. ## What Changed - Extract failed-retry projection under the existing authorization owner. Preserve current receipt/source authority and provider transport authorization. - Register bounded scoped work using the existing delivery coordinator/scheduler. Preserve exact PostgreSQL cursor precision, contention and a wake arriving during reconciliation. - Extend the shared transaction-aware notification boundary to failed-retry notices. Cover admission, conditional denial, early refusal, requester changes, and agent lifecycle transitions to an invokable status. Two shared receipt hooks and their budget fixture correction already landed in the preceding inbound slice; preserve them once. - Notify when relevant native lease cleanup completes or a reusable lease is retired, preserving claim fencing and caller-owned transactions. - Avoid the discovery scan identified in #17647: known task scopes require zero discovery SQL; company/startup discovery uses a single DISTINCT projection from the same candidate predicate. Bounded current authorization still runs afterward. - Remove only this duty from periodic publication maintenance, retain manual behavior and startup reconstruction, and add real producer/lifecycle/query-budget regressions with an audit. ## Verification Published head: `fa95a1f2fb9d546aa62dbd3b43aa714f5c88bd35`, targeting master after #17647 merged at `1aa3a21ac41f602977638cf0be875b01a243e69f`. - 42 distinct current combined-master cases pass: 21 retry-notice/native-lease/restored-input producer cases (1,651 skipped), plus all 21 app/startup and task-drain admission-release cases. Server typechecking, build and module boundaries pass. The final follow-up adds acceptance documentation only; all 17 owned server source/test hashes are unchanged. - The prior 404-case matrix passes at prepared revision `c2c53b839497be6f5e28aa2cd5ba50cfd1d76d4a`: 259 across six complete consumer/lifecycle/environment suites, 20 notice/lease cases, 53 ordinary-notice and 72 Retry/budget cases. Those chat selections skip 1,651/1,618/1,599 cases. A subsequent 21-case combined producer check passes after the parent restoration repair. These earlier runs retain their revision scope. - The normal two-parent integration `43d0f84ea56845b8fc4946a7533ecbf7f07b679a` preserves the complete 19-path owned binary patch and all 23 unrelated master paths/deletions. The semantic merge base is the verified landed parent head `41f7e2c9e351efa6cfa5ef699c0e69df7425012b`; squash ancestry otherwise causes historical conflicts. Both parent commits and the exact resolved tree are verified; no rebase or force push occurs. - Three scope-discovery regressions fail before the fix. The first SQL projection fails with PostgreSQL 42702 (ambiguous company column), then passes after using a distinct company alias. Known task hints have zero discovery SQL; company/startup discovery has one query returning only distinct task scopes. Existing bounded projection/current authorization remain intact. - Current producer tests cover commit/rollback, parked authority restoration, real agent resume, early refusal, requester changes, native lease cleanup/replacement, drain/restart/shutdown, precise cursors and contention. Parent shared admission/denial/restored-input notifications are preserved once. No polling backstop or storage is added. - [The owner audit](https://github.com/paperclipai/paperclip/blob/fa95a1f2fb9d546aa62dbd3b43aa714f5c88bd35/doc/plans/2026-10-25-failed-retry-notices-sleep-audit.md#final-landed-parent-acceptance--october-28) records current commands, fixtures, failure history and limits. Fresh exact-head remote checks and Greptile 5/5 are pending; old-head reviews do not count. The owner audit preserves prior failures and their original revision scope. The old remote head's Runner receipt timeout, cancelled gate, and review error are not current passes; fresh exact-head checks and Greptile 5/5 are required. No full local repository suite, live provider tenant, external wake alarm, physical suspension or savings claim is made. ### Final remote gates Exact published head `fa95a1f2fb9d546aa62dbd3b43aa714f5c88bd35` has Greptile 5/5 with no actionable findings, successful current main CI and ready-triggered security scan. The final dependency/quality review workflow `40747725096` passed after waiting for a GitHub-hosted runner. No Ponytail verdict is present. Two automatic main CI requests and two identical-head quality-review requests were created one second apart. The older main run was automatically cancelled; only the proven duplicate, unstarted quality request was cancelled manually. Their same-head, same-workflow replacements both finish successfully. The merge guard verifies these replacements, all current checks and current required-status integration rules before excluding the superseded cancellation contexts. No actual failing review or current check is waived and no protection bypass is used. ## Risks - Process-local signals cover supported writers using this boundary, not independent SQL or other replicas. - Lease completion now reads/locks the exact row before updating. Reusable lease retirement wakes company scope because its previous issue can differ. Claims and transaction ownership remain enforced. - Broad discovery can inspect history inside PostgreSQL; DISTINCT reduces returned rows and round trips, without claiming constant database cost. Task hints are wake requests, so bounded reconciliation handles already exhausted scopes. - Native filesystem activation belongs to execution authorization and is not an eligibility input for this notice projection. Other notice/maintenance duties and Cloud sleep gates remain separate. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, local code inspection, shell execution, and PostgreSQL testing. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 14:14:28 -05:00
DottaandPaperclip	1aa3a21ac4	fix(server): wake inbound queue notices from committed changes (#17647 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat users receive a notice when accepted input waits for execution. > - Periodic publication maintenance scans those receipts even when nothing changes. > - Admission, cancellation, source removal, and authority changes already define when a notice can change. > - Existing action, receipt, publication, and provider-link rows preserve that work across restart. > - This PR gives ordinary inbound queue notices a commit-driven owner using the shared scheduler. > - Settled notices stop querying while unfinished batches and contention retain recovery. ## Linked Issues or Issue Description This ordinary inbound-notice slice follows merged #17895 and now targets master. Related extraction: #16187. Failed-run retry notice projection remains a separate following slice (#17649); its two shared receipt-writer notifications move forward here because ordinary input can attach to those receipt owners. What happened? Publication maintenance scans accepted input to project queued, not-started, and removed-source notices. Empty scans keep touching the database. Its cursor also loses PostgreSQL timestamp precision, so a full page of suppressed candidates can repeat. Expected behavior Committed changes should wake the notice owner. The worker should reconstruct saved candidates after restart and stop scanning when settled. It must retain a wake across drain, failed selection, contention, and a commit during reconciliation. Steps to reproduce Start an otherwise idle instance with chat history. Observe the ordinary inbound notice scan in periodic publication maintenance. For the cursor defect, retain a full page of denied candidates with microsecond timestamps before an eligible notice. ## What Changed - Register an ordinary inbound-notice owner with the existing delivery coordinator and scheduler. Bound each pass to 100 candidates, preserve precise continuation cursors and earlier-page contention, and reset scoped discovery after committed changes. - Notify from real admission, failed action settlement, source removal, task/tree/agent/budget/company cancellation, signed Slack Stop, and deferred queue cancellation/failure. Preserve pending-status and absent-run guards so concurrent promotion wins. - Cover task privacy/grants, project membership/visibility and current authority through their persistence owners. Keep receipt ownership, current authorization, provider policy and closed notice text in the existing chat service. - Cover shared retry-owned receipts through the existing durable authorization callback and a notifying transaction around conditional denial retirement. These small producer fixes already exist in #17649; no per-caller recovery wrappers are added. Retain its matching budget-refusal fixture correction. - Remove only ordinary notices from periodic maintenance. Preserve the separate failed-retry pass at 50 and keep the explicit combined manual API. - Preserve timestamp precision, fairness, restart, drain/resume, rollback and quiet behavior. Add two real producer transaction regressions and update the audit. Add no storage or scheduler infrastructure. ## Verification Current publication head: `41f7e2c9e351efa6cfa5ef699c0e69df7425012b`, after normal merge of master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd` (#17960). The integration preserves both complete added/removed patches, all non-overlap upstream blobs/deletions, and task-before-comment lock order before the notice source lock/intent. Full binary identity is not claimed for the overlapping files. The final follow-up changes acceptance documentation only. - 202 distinct combined-master cases pass on `1720b685d9d5fb9cb4377223fdaa0a9fae2aff9c`: 148 across the real PostgreSQL receipt concurrency and two comment route suites, plus 54 notice/FIFO/producer cases (1,598 skipped). Current server typechecking/build and module boundaries pass; all 21 owned source/test hashes are unchanged after the documentation follow-up. - The earlier CI voice-reconstruction quiet assertion failed once after 105 fetches. All 15 voice lifecycle cases pass unchanged locally; no assertions or production code were relaxed and no CI retry was used. The original failure remains in the audit, and current-head full CI must pass. - Earlier restoration repair (head `28c898e93504318056d7a6ab2bbc5eafa6b639ae`): 126 distinct selected cases pass (54 ordinary-notice/producer cases with 1,598 skipped, plus 72 separate Retry/budget cases with 1,580 skipped). Fix the blocking [restored input stays silent](https://github.com/paperclipai/paperclip/pull/17647#discussion_r4404007181) finding. A failed Retry's earlier receipt wake can drain before `releaseUnadmittedTaskRetryInput` restores ordinary receipts in a later transaction. The restoring transaction now awaits the existing company/task notification before its guarded write. - The new causal regression builds a durable unadmitted failed-Retry owner, uses the real input-claim owner, consumes an earlier committed wake, and proves two minutes of worker quiet with no notice. The real release transaction then restores the ordinary row, emits intent/commit settlement, and reactivates the coordinator to create its queued notice. It fails before the hook with the restored row visible but no intent. This is real PostgreSQL producer/consumer boundary proof with seeded Retry-refusal state; the separate compatibility selection exercises existing supported GitHub Retry flows. It is not a claimed complete public-request failure walkthrough. - Earlier discovery-repair acceptance (revision `4a713edfa06b9d3dcc0e81e900006fbdb973b2d9`): 267 distinct selected cases pass: 142 complete cancellation/app/admission consumer cases, 53 ordinary-notice cases, and 72 separate retry/budget cases. The selections skip 1,598 and 1,579 other cases. The final master integration leaves this complete patch byte-identical; the new master paths concern provider workspace/session reuse and E2E reports. - Fix the valid P2 [discovery rereads denied history](https://github.com/paperclipai/paperclip/pull/17647#discussion_r4386889825), despite the previous head's 5/5 summary. Task hints now need zero discovery SQL; company/startup discovery returns DISTINCT company/task identifiers with one query. Bounded projection still checks current authority and candidates. Three real database regressions with 250 denied actions and a second company fail before the fix and pass afterward, checking actual query and returned-row counts. Broad SQL may still inspect candidate history; no constant database-cost claim is made. - Both new producer tests fail before their fixes and pass afterward. One observes intent before a callback-owned receipt write and settlement after outer commit. The other uses real heartbeat admission/recovery and a PostgreSQL row lock to prove exact company/task intent exists before conditional denial retirement; the outside connection still sees deferred status until commit. - The reader can select a retry-owned deferred receipt for later ordinary input, and receipt resolution follows its owner. Disabling coalescing on the retry's own request does not exclude that later path. The two transferred notifications close the actual writer gaps; current authorization remains enforced by existing consumers. This is producer/transaction proof, not a claimed complete mixed-ingress walkthrough. - The moved callback exposes an old test mock that authorized against the root DB. The existing #17649 fixture correction authorizes and writes in one transaction; original requester, comment, skipped-status and budget assertions remain. Both the initial invalid lock fixture and the corrected missing-intent failures are preserved in the audit. - Updated shared dependency build, server typechecking/build and module-boundary checks pass. Because master also changed agent configuration, upgrade admission and runner contracts, the same 264 cases were rerun on combined revision `875a6ee8d211cba7f0499aace7fcea7f6880c4a2` with new isolated databases. Repeated checks are not 528 different tests. Tests under `src/__tests__` were executed separately from production typechecking. - [Current acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/41f7e2c9e351efa6cfa5ef699c0e69df7425012b/doc/plans/2026-10-25-inbound-wake-notices-sleep-audit.md#shared-retry-receipt-producer-gap-resolved-october-28) records current scope, exact commands, actual fixtures, source hashes and remaining limits. Earlier baseline/full-suite counts retain their original revision scope. - Server typechecking/build and module boundaries pass after the current restoration hook. The earlier combined-master compilation at `661f2f6da8c6c22cdcc25edccd57502ee6fa8c4b` retains its original scope. - Current exact-head CI is green and Greptile is 5/5. The old head's review error is not a pass. No full local repository matrix, live provider tenant, external alarm, physical suspension or savings claim is made. Following notice duties retain their separate owners. ### Final merge verification - Reviewed PR head remains `41f7e2c9e351efa6cfa5ef699c0e69df7425012b`: exact-head Greptile 5/5, all 55 check contexts clean, no unresolved review threads, and no Ponytail verdict present. - One Runner shard initially failed descriptor suspension and a five-second OpenCode fixture deadline. The same package passed on the preceding head. The first local attempt was invalid because required Rust fixture binaries were absent; after building them, all eight relevant cases pass with unchanged timeouts. One failed-job retry then passes 1,956 cases (four skipped). No source/assertion/time-budget change or second CI retry was used. - Current master `0e365768f032a472807b5720da8431f92d2f07ce` adds manual monitor and task-drain fixes. Its routes file overlaps but merges cleanly. Local normal-merge validation commit `2720b79c769cb609ad4d392512331e67b910ff3f` preserves both complete added/removed patches and every unrelated master blob/deletion. This is not raw binary identity for overlapping files. - 156 distinct cases pass on that exact combined tree: all 102 monitor/drain/policy cases plus 54 notice cases (1,598 skipped). Types and build pass. An initial competing-cold-check run failed setup hooks (38 passed, two hook failures, 62 skipped); the same four suites pass alone with all original hook/test timeouts and assertions intact. The failed run remains recorded, not counted as full acceptance. - This merge proof changes no published source or review head. The protected pinned-head merge guard recomputes both patches and requires the exact tested master/tree/head, successful validation receipts and unchanged live merge gates. Previous acceptance counts retain their original revision scope; no local full-repository or deployed-sleep claim is made. Master then advanced to `75754ffb7af2b7ee7dce1e915a145faa82a8fb2a` with #17976 (five UI/e2e/map files only). The entire `server` and `packages` trees are identical to the validated composition. Both complete patches and all newly landed UI files are preserved in the final clean merge tree; the protected gate checks these exact source-root hashes and changed paths. No additional UI-test result is claimed. ## Risks - Missing a supported producer could delay a notice until restart. Tests cover real service and API producers; durable rows remain authoritative. Notifications cover supported application writers in the owning process, not arbitrary external SQL writers. - Privacy invalidation can affect descendants, so those changes use company scope. Ordinary admission and publication changes use task scope. Settled history causes no recurring scan. - Provider output remains subject to current authority and receipt ownership. A cancelled request without a visible queued notice does not acquire a new failure message. - This establishes worker quiet, not host or database suspension. Cloud alarms and conservative sleep gates do not change. Other maintenance remains in place. - Remaining stacked slices still need their own integration, review, and CI. This slice merges only under the user’s explicit campaign authorization after current-head gates pass. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection and code execution tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 13:34:38 -05:00
DottaandPaperclip	1ed0af4186	fix(server): wake native chat progress from committed changes (#17639 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat users receive coarse progress updates while native runs execute. > - A recurring chat pass scans saved run events even when no progress changes. > - This traffic prevents an idle database from staying idle. > - Events, lineage changes, and publication state already identify when projection can change. > - This PR gives native progress its own commit-driven worker and schedules only unfinished work. > - Idle progress stops scanning while restart and recovery preserve delivery. ## Linked Issues or Issue Description Refs #17606. This is slice 29 of the recurring-server-work stack, based on `codex/sleep-28-chat-run-milestones`. It owns native chat progress only. Related historical optimization: #16845. The parent and this PR must land in stack order. What happened? Native chat progress is still called by the one-second chat reconciliation loop. It repeatedly selects run events with no new work. Its cursor also truncates PostgreSQL timestamp precision, which can repeat a rejected full page forever. Expected behavior Committed events and eligibility changes should wake the projection owner. Existing rows should reconstruct work after restart. A settled worker should have no recurring database activity. Full batches, lock contention, and errors must retain recovery. Steps to reproduce Start an otherwise idle instance with chat history. Observe the native progress lane in the recurring chat coordinator. For the cursor defect, create a full page of suppressed candidates sharing a microsecond timestamp and one eligible later candidate. The old cursor fails to advance. ## What Changed - Extract native progress from lifecycle milestones into its owning service. Keep the public combined API and budget compatible. - Register a scoped worker with the existing delivery coordinator and scheduler. Remove native progress from the recurring chat coordinator. - Notify before allowlisted event writes in both persistence owners. Preserve outer commit, rollback, duplicate detection, and the ordinary non-progress log fast path. - Extend the existing lineage/authority notification boundary once. Failed-publication replay and operator retry carry their issue scope. First working publications signal progress so commit ordering cannot strand an already-inspected event. - Limit notification payloads to company/task IDs so wider caller inputs never serialize database handles or private fields. Load task-binding fields only for progress event kinds; other native/recovery events retain their narrow lock query. - Preserve current safe phrases, cadence, phase deduplication, provider policy, authorization, and final-output precedence. - Retry concrete issue/run lock contention. Disarm deleted or terminal candidates. Preserve scoped dirty work across batches, failures, drain and shutdown. - Preserve PostgreSQL timestamp precision in pagination. Add a bounded regression for the former infinite rejected-page scan. - Document the producer audit, lifecycle contract, failures, and remaining limits in `doc/plans/2026-10-25-native-chat-progress-sleep-audit.md`. ## Verification Published head: `b7ee820e49fd06c780b36b79d9b1c0883557fb73`, targeting master after #17606 merged at `5f0da562dbfdf3229f3cd4bb3dc742ab541986b4`. A normal ancestry merge preserves the complete 18-path native-progress patch byte-for-byte relative to its verified parent, with all six unrelated current master paths intact. No new production change was introduced during final reconciliation. - 163 distinct selected cases pass on the verified unchanged patch: 137 across eight complete suites (native progress, atomic event append, all five milestone worker suites, and app admission wiring), plus 26 real chat integration cases selected by `native progress`. The other 1,595 chat integration cases were skipped, not passed. - The full worker suites cover restart, committed events, trace reprojection, duplicate/no-progress events, first-working publication ordering, concrete lock contention, issue scope, backlog, drain/resume, failures and shutdown. Settled history leaves no recurring SQL. The real chat cases verify operator retry and replay, microsecond pagination, authority, cadence, phase deduplication, final/question precedence and redaction. - The sole earlier parent conflict preserves both this slice's issue-scoped retry wake and the exact voice conversation scope. The current parent's milestone producers, onboarding completion guard, identity query/read-scope coverage, migration 0397 and voice cleanup/retention repairs remain present. - Server production typechecking, build and module-boundary checks pass. The selected server tests live under the excluded `src/__tests__` path and were executed separately. Each database owner uses a new isolated app-migrated PostgreSQL fixture. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/b7ee820e49fd06c780b36b79d9b1c0883557fb73/doc/plans/2026-10-25-native-chat-progress-sleep-audit.md#current-milestone-parent-acceptance-october-28) records current counts and scope. Older combined checks and failures retain their original revision scope and are not added to 163. No new table, dependency, workflow, migration, or scheduler contract is introduced. - Exact-head Greptile is 5/5 and all 55 current checks are clean. No Ponytail review is present, which is an explicitly authorized nonblocker; no failing verdict is waived. Older head/parent reviews are not current passes. Actual failing Ponytail findings must be fixed. Missing/stopped/paused turns without a verdict are authorized nonblockers; active unfinished reviews retain the original per-head timer with a 25-minute monitor cutoff respecting the 30-minute maximum. - No complete local repository matrix, live provider tenant, external wake, physical host/database suspension or savings claim is made. Following stack duties remain separate. ## Risks - Signals are process-local under the existing single active writer contract. Independent SQL writers or another active process need an explicit wake integration. - Allowlisted direct log writes now use a transaction. Native bound runs register transaction intent; unrelated direct event types retain their atomic-statement path. - The shared eligibility boundary wakes both lifecycle and native projection. It does not decide provider policy or copy event payloads. - The existing 24-hour selection window, event-time cadence and authority checks remain. A suppressed event is not retried just because wall-clock time passes. - This establishes worker quiet, not whole-instance suspension. Cloud alarm coverage and conservative sleep gates do not change. Other chat maintenance remains on its existing cadence. - This actual-master reconciliation still requires fresh exact-head review and CI before protected merge. Subsequent slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code execution and browser-test tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 11:22:29 -05:00
DottaandPaperclip	5f0da562db	fix(server): wake chat lifecycle milestones from committed changes (#17606 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description What happened? The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. What did you expect? Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. Steps to reproduce Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - 330 distinct selected cases pass on the verified unchanged patch: 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - Exact-head Greptile is 5/5 and all 55 current checks are clean. No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 11:07:54 -05:00
DottaandPaperclip	7d75ae0569	fix(server): recover Teams attempts from saved claim deadlines (#17592 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves a claim before each provider operation. > - A crashed operation can leave that claim unfinished. > - Periodic maintenance scans for abandoned claims even when none exist. > - This PR wakes recovery from committed changes and the saved claim deadline. > - Empty work becomes quiet while unknown effects remain safe to review. ## Linked Issues or Issue Description What existing behavior does this improve? Teams abandoned file-transfer recovery. This is slice 27, stacked on consent expiry #17590. Current behavior The remaining Teams maintenance callback scans every 30 seconds for expired transport claims. It runs from shared publication maintenance. Proposed behavior The Teams owner reconstructs existing claim deadlines at startup and reloads a transfer after its committed update. At expiry it clears the claim and preserves the existing unknown-outcome or conflict state. Normal receipt settlement removes the deadline. Reason and benefit Remove the last periodic Teams maintenance scan without resending a possibly delivered card or file. Breaking changes No schema or public API change. Explicit diagnostic recovery methods remain compatible. Consent expiry stays with its separate parent worker. ## What Changed - Add a Teams-owned worker for consent_sending, uploading, file_info_sending and conflict rows with a saved attempt expiry. - Notify from the existing versioned update before writes. Preserve transaction ownership and shared uncertain-write reconciliation. - Expose a narrow per-transfer recovery method that uses the existing lock order, integrity checks and atomic projection. - Remove the periodic Teams maintenance callback and its publication-maintenance invocation. - Add real claim, receipt, conflict, backlog, restart, precision, failure, drain and shutdown tests. Wire the owner through existing app admission gates. - Retain the existing signoff cancelled-run selection test and GitHub inline retry-window race assertion. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-attempt-recovery.md` and its linked audit. ## Verification Published head: `24c54399909105984e2a10f5869064984f69c472`, targeting master after consent #17590 merged at `4a89456bb3f9aef87cfe0e922e8b9d1a777a6c7e`. A normal ancestry merge reconciles the landed squash with the previously verified child. The complete 13-path binary patch is identical before and after reconciliation, and all 30 unrelated master paths are preserved. This adds no new worker behavior; the 239-case local evidence below belongs to the verified unchanged patch. All 58 current-head checks are now clean, including the complete required CI run. Greptile is 5/5 at this exact head. The current Ponytail turn is paused without a verdict, which is an explicitly authorized nonblocker; no failing review is waived. The complete 13-path slice preserves the reviewed attempt implementation and inherits the parent's report migration 0397, bounded consent worker and retention CI test repair. No new schema, queue table, shared scheduler contract, workflow or lockfile change is added by this duty. - 239 distinct selected cases pass: 221 across five complete transfer/publication/consent/runtime/app-wiring suites, 16 selected real Board/Teams-authority/GitHub-retry integration cases, and two delegated Slack policy cases. Other chat integration cases (1,579) and connection-management cases (81) were skipped, not passed. Each database owner uses a new isolated PostgreSQL database. - Real Board publication saves an active attempt deadline; its confirmed receipt retires recovery, and SQL stays quiet afterward. The inherited consent producer and authority cases still pass. Lifecycle coverage exercises claim stages, unknown-outcome quarantine without repeating provider I/O, conflicts, rollback, restart, precision, backlog/fairness, failure recovery, drain and shutdown. The changed GitHub metadata/retry race passes without duplicate assessments. - The old server10 CI failure, run `40557802358` job `122390238707`, was a delegated Slack invitation rate-window assertion (2,122 passed/one failed/15 skipped in that shard). The verified parent already contains its Date-only fixture fix. Both linked and invitation cases now pass their full caller/target/shared-policy scenarios on a fresh server fixture; no duplicate policy change or new owned file was added. That old failure is preserved rather than called a pass. - Server production typecheck, build and module-boundary checks pass. Test files were executed; production typechecking does not statically check them. The existing test-only signoff cancelled-run selection fix is retained but was not executed locally in this acceptance; fresh remote E2E gates remain required. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-25-teams-attempt-recovery-sleep-audit.md#current-consent-parent-preparation-and-acceptance-october-28) records current scope, commands, counts and limits. Earlier 235-case and combined-stack counts belong to their stated revisions and are not added to 239. The [stack ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository suite, live Teams/Slack tenant, external wake, physical suspension or savings claim is made. - Exact-head Greptile 5/5 and all 58 checks are clean. Actual failing Ponytail findings must be repaired; a missing/stopped/paused turn without a verdict is an authorized nonblocker. An active no-verdict turn waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older reviews and parent checks are not treated as current passes. ## Risks - Recovery must not infer that an expired claim means a provider effect did not happen. It retains unknown outcomes and does not resend. - Late receipts must still match current claim identity. Current-state rechecks and existing protocol tests preserve this fence. - Supported deployment uses one active writer process through its owning DB client. Independent SQL and additional processes need a separate wake contract. - Cloud external alarm coverage and sleep eligibility are unchanged. Other chat duties still run periodically until their own migrations. - Voice-report parent review repairs are propagated: explicit single-active-writer scope and an indexed report window. Thirty report-worker/startup tests pass on the combined child. Fresh remote checks, older stack findings and final master reconciliation remain required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 10:51:16 -05:00
DottaandPaperclip	4a89456bb3	fix(server): schedule Teams consent expiry from saved deadlines (#17590 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves consent and transfer state in the database. > - Unused consent must expire even when no new message arrives. > - The maintenance loop currently checks this state every 30 seconds. > - This PR schedules each existing consent deadline and wakes on committed changes. > - Empty or settled consent work stops querying while expiry remains recoverable. ## Linked Issues or Issue Description What existing behavior does this improve? Teams unused file-consent expiry. This is slice 26 of the recurring-server migration stack, following merged #17586 and targeting master. Current behavior The Teams maintenance callback scans for expired unused consent and abandoned transport attempts together, even when consent work is empty. Proposed behavior The Teams owner reconstructs unused consent at startup and reloads one transfer after a committed write. The shared scheduler runs its saved expiry. Abandoned attempts keep their current recovery until a separate slice migrates them. Reason and benefit Remove unnecessary consent queries without losing expiry, atomic publication projection, or recovery after restart. Breaking changes No schema or API change. The explicit combined recovery method remains available. Cloud sleep eligibility and external alarms remain unchanged. ## What Changed - Add a Teams-owned scoped deadline worker for the three unused consent phases. Registered reconstruction yields after one 100-row page per scope refresh and resets cursors on registration; the explicit diagnostic inventory still returns every eligible row. - Signal before insertion and the existing versioned update inside the owning transaction. Nested writes wait for outer settlement. - Extract the existing locked per-transfer recovery operation. Preserve authority checks, atomic projection, bounded lock waits and unknown-outcome quarantine. - Remove unused consent from periodic maintenance selection and execution. Keep abandoned attempts as a separate obligation. - Register the worker through existing app lifecycle gates. Add lifecycle, producer, production wiring and precision regressions. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-consent-expiry.md` and its linked audit. ## Verification Published head: `9e84cfab1f795279c8cec413a946631f1d781bd2`. This follow-up changes only the inherited voice-retention test and its CI failure ledger; all production behavior is unchanged from `36f43922`. It normally merges actual landed master/report parent `8049fbf51d0701ca6b242e4b1198e5449f7661b2`. The verified 12-path Teams patch was byte-identical across the actual-parent ancestry integration. This test-only CI follow-up adds one existing retention test path, bringing the final scope to 13 paths. Every unrelated master change, report documentation fix, and migration 0397 is preserved. No database, shared-worker, workflow or lockfile change belongs to this slice. - 223 distinct selected cases pass: 209 across five complete transfer/publication/consent/runtime/app-wiring suites, plus 14 selected real Board-producer and Teams-authority integration cases. The other 1,580 integration cases were skipped, not passed. The protocol and production consumers use separate new PostgreSQL databases migrated through the application migrator. - A real 105-transfer regression failed against the original worker because one registered startup pass selected 105 rows instead of yielding at 100. The corrected worker uses the existing paged shared contract. Restarting the same service midway through discovery selects the first 100 again, restores all 105 saved deadlines and expires them at the original deadline. SQL is quiet before the deadline and after settlement, with no consent-card or upload requests. The one-shot inventory still returns all 105 rows. - A guarded mutation removing only cursor reset fails at five rows instead of 100 after restart. The original production source was restored before the complete suites. The initial regression fixture incorrectly faked network timers before its first SQL connection and timed out; that setup failure is retained separately from the corrected 105-versus-100 regression. No assertion or timeout was weakened. - The real Board producer registers the worker before publication, commits the transfer wake, preserves the saved expiry, remains SQL-quiet until due and atomically projects expiry to a cancelled publication without another provider request. Existing authority and consent transitions remain covered. - Complete lifecycle coverage retains all three consent phases, precision rounding, rollback and held outer commit, exact-row hints, expiry recheck, failed projection retry, corrupt-row fairness, drain/resume, shutdown and separation from abandoned-attempt maintenance. Production server typecheck, build and module boundaries pass. Server production typechecking does not statically check test files; those files were executed. - [Current bounded reconstruction and failure ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#bounded-reconstruction-and-current-parent-acceptance-october-28) records exact scope, counts, guarded mutation, corrected timer setup, earlier fixture failures and limitations. Earlier 222-case and combined-stack results belong to their stated historical heads; they are not added to 223 or claimed rerun here. The [stack ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository-suite, live Teams tenant, Cloud wake, host/database suspension or savings claim is made. - Fresh exact-head Greptile 5/5 and all CI are required. Actual failing Ponytail findings must be repaired. A missing/stopped/paused turn without a verdict is an authorized nonblocker; an active no-verdict review waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older scores and parent checks are not accepted as current passes. - Preserved inherited CI failure: prior head `36f43922` had exact-head Greptile/Ponytail 5/5 but CI run `40708566195` server6 job `122887395220` failed the retention fairness assertion (2,291 passed/one failed/one skipped in that shard). A fixed final UUID reproduces the wrong expectation that an undiscovered page-two row is already deleted. The corrected test controls time, expects that row to remain before discovery and requires its deletion in the next fair batch, then all remaining history and idle SQL. The existing five-second wait budget and global timeout are unchanged. All 13 retention lifecycle cases pass; bypassing company rotation fails the corrected assertion, and the original shared helper is restored byte-for-byte. Earlier 223-case Teams acceptance remains valid for unchanged Teams source and is not relabeled as a new complete repository run. [Full CI repair ledger](https://github.com/paperclipai/paperclip/blob/9e84cfab1f795279c8cec413a946631f1d781bd2/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#inherited-retention-ci-fairness-repair-october-28). Fresh CI and exact-head reviews are required for this test-only follow-up; old scores are stale. - Current merge gates: all 58 checks are now clean for this exact head and Greptile is 5/5 with no actionable findings. The current Ponytail turn is paused without a verdict, an authorized nonblocking state. Workflow `40712464720` attempt 1 independently hit npm `ECONNRESET` during the serialized8 job's tool self-installer, before its tests ran. Exactly one targeted job retry was requested after the original workflow finished; attempt 2 completed successfully on the unchanged head. No code, dependencies, workflow, timeout or review timer changed for that retry. The original setup failure is retained in its [job log](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122901243926); the successful replacement is [job 122910350074](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122910350074). ## Risks - Missed supported producer signals could delay expiry until restart. Real insertion, cancellation, consent transitions, caller-owned transactions and Board publication are covered. - Corrupt or locked rows must retain retry without starving healthy work. Paging, fairness, atomic rollback, redaction, drain/resume and shutdown tests cover these paths. - Independent SQL writers and other processes need their own notification contract. No persistent DB listener is added. - Attempt recovery still scans periodically until the next slice. A local deadline is not a durable Cloud alarm. This PR does not claim whole-instance sleep. - This slice is reconciled with its landed parent; current-head remote checks/review and live-master composition proof still gate merge. Later stack duties remain separate PRs. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 10:32:30 -05:00
DottaandPaperclip	71b448b9cb	fix(server): schedule inbound voice cleanup from durable obligations (#17580 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls need approval, expiry, and safe provider cleanup. > - The remaining voice maintenance tick scans admissions every second, including idle endpoints. > - Admission status does not prove that the provider call has ended. > - Existing admission rows can retain the cleanup owner, confirmation, and retry deadline. > - This pull request replaces the cleanup scan with committed changes and actual deadlines. > - Idle cleanup stops querying while unfinished calls keep a recoverable path to termination. ## Linked Issues or Issue Description What existing behavior does this improve? Inbound voice admission cleanup. Its reply-delivery parent #17578 has landed; this PR targets master. Live-session recovery is in #17576. It preserves separate ownership for those services. Current behavior The voice maintenance tick scans expired admissions even when no cleanup is due. A session can commit before the separate approved-admission write. Expiry can then terminate that authorized call. A rejected guest call can be labeled ended before provider termination succeeds. Seven-day retention can delete that unfinished cleanup. Proposed behavior Wake cleanup from committed admission and session changes. Schedule the original expiry or saved retry. Claim current state under the admission lock before provider I/O. Commit approval with session reservation. Keep unconfirmed termination durable and transfer bound calls to live-session recovery. Reason and benefit Configured idle endpoints and confirmed history cause no recurring admission cleanup SQL or provider requests. Approval, retries, crash recovery, and cleanup remain supported. History retention and call reports stay separate migration items. ## What Changed - Recover terminal and rejected legacy admissions whose session binding was lost. Copy known termination or bind a proven matching live session for cleanup without granting approval. - Register inbound cleanup with the shared coordinator and scoped deadline worker. Yield after each 100-row scope page and reset cursors on registration. Remove only cleanup from periodic voice maintenance. Keep complete manual reconciliation. - Add commit-aware notifications at admission, approval, rejection, guest binding, pending-tool expiry, lifecycle confirmation, and session termination writes. - Commit authorized session binding and admission approval together. Recheck cleanup ownership under the same admission lock. Recover older interrupted bindings only from matching session evidence. - Add nullable cleanup confirmation, next-attempt, and provider-account fields to the existing admission table. No new queue table is added. Generated migration 0396 backfills only provable historical termination in committed 1,000-row keyset batches and adds a concurrent partial index. - Save the retry before provider contact. An unconfirmed end receipt retains cleanup. Bound calls remain owned by live-session recovery. - Verify the signed provider account before using a rotated key for unbound cleanup. A different account cannot inspect or end the saved call. Legacy admissions require their original credential fingerprint. - Prevent history pruning and company deletion from erasing unfinished cleanup. Company deletion locks endpoints before the voice owner's check, matching ingress serialization. - Document producer coverage, ownership, deadlines, verification, and remaining sleep limits. ## Verification Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply parent #17578 has landed. A normal merge integrates actual master `e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated changes, and regenerates this slice’s migration after landed migration 0395. Fresh exact-head reviews and CI are required. - The previous Ponytail 3/5 finding is repaired: legacy terminal admissions no longer enter approval-only repair, and rejected admissions with valid matching live sessions retain their rejection and persist the binding needed for session-scoped cleanup confirmation. Five regressions failed with ownership conflicts before the repair. Five conflicting-session regressions prevent an unrelated leg from being claimed. No cleanup operation grants approval or starts a provider call. - 24 complete cleanup lifecycle cases and 122 complete voice-store integration cases pass on the repair. After actual-master integration and migration regeneration, the complete 24-case cleanup suite passes again on a new empty PostgreSQL 17 database, and the database migration integration test passes separately. The cleanup owner, voice-store test and admission schema are byte-identical to that tested repair; the cleanup fixture only changes the migration filename. Migration numbering/safety, database build, production server typecheck and build pass. - Migration `0396_tan_paladin.sql` was generated with `pnpm db:generate`. Every landed SQL migration and the complete parent journal prefix are unchanged. Generated snapshots retain exactly 0392–0396, pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and committed 1,000-row historical backfill are retained byte for byte under the new generated filename, registered for top-level execution. The real application migrator boots a fresh database, repairs an interrupted concurrent index, serializes replays and verifies eligible-history backfill across actual transactions. - The preceding published head `48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but failed CI server shard 9/12: one existing connected MCP rate-limit test expected two real-clock calls to share a fixed 60-second bucket. A controlled window rollover reproduced its exact failure. That existing fixture now freezes only `Date` around the same-window 429 assertion, verifies acceptance after advancing one window, and restores the real clock in `finally`. All 92 gateway acceptance cases pass. Production rate limiting, timeouts and assertions are unchanged. This adds one existing CI fixture to the slice; the original failed run remains recorded. - The first regression invocation skipped the suite because it used the wrong database variable; that is not a pass. The corrected invocation reproduced five ownership conflicts. The first fixed cleanup run passed 22 and failed two new low-level deletion test calls on a retained-session foreign key; the regressions now check the cleanup deletion guard at its owning boundary, with production deletion ordering unchanged. The final full suite passes all 24. - The first complete voice-store invocation passed 121 and timed out on one transcript case at the unchanged 15-second limit. That case passed in isolation on a fresh database. One subsequent complete invocation, with builds finished, passed all 122 without source, assertion or timeout changes. The underlying timeout cause is unproven; the original failure remains recorded. A new PostgreSQL startup also failed from exhausted host semaphore capacity; fresh databases on the already-owned server were used without removing shared IPC resources. - Prior source `b8b12199b9` passed 266 distinct cases and carried three guarded mutation proofs; those older results are not added to this repair’s case count. Its CI run 40641168596 failed E2E shard 3 when legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub initially rejected a retry while the run was active, then accepted one targeted retry after completion; attempt 2 passed on that same old head. The lock holder and underlying cause remain unproven. The unchanged Node policy checker’s older failure on ignored generated Copilot manifests and passing tracked-input mirror are also retained in the audit; no workflow, checker, vendor manifest or ignore rule was modified. - [Current repair and failure record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28). Current-head CI and Greptile 5/5 gate merge. A completed failing Ponytail verdict must be fixed; missing or interrupted reviews do not block, and active pending review waits are capped at 30 minutes under the maintainer’s explicit instruction. No new full local repository suite, real provider, Cloud alarm, host/database suspension or savings claim is made. ## Risks - The migration distinguishes admission status from physical termination. Legacy ended guest rejections without proof remain cleanup obligations. Missing original credentials can keep those old obligations unresolved. - Future writers must use the existing notification boundary. Independent SQL writers and multiple processes need their own wake contract. - Pending inbound cleanup now blocks company deletion instead of allowing cascade deletion to lose the provider obligation. Existing voice-history deletion restrictions remain. This deliberately includes an active anonymous inbound ring and an old call whose termination cannot be verified. An operator must stop ingress and settle the call before deletion; this slice adds no abandonment override and never treats missing credentials as proof of termination. This is the retained cleanup ownership policy, not a claim that every deletion attempt can complete immediately. - History retention and report processing still run in the remaining maintenance lane. This PR does not extend Cloud wake coverage or claim measured host/database suspension. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 08:30:15 -05:00
DottaandPaperclip	e93732c3d4	fix(server): deliver voice replies from commits and saved deadlines (#17578 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Phone calls receive approved answers from the existing chat publication queue. > - The chat maintenance timer checks for voice replies every second, including when no reply is ready. > - A reply can be saved before its publication commits. An early wake alone can miss that final transition. > - Existing session, reply, publication, and action rows already record delivery and recovery work. > - This pull request uses committed changes and saved deadlines to drive phone reply delivery. > - Empty reply queues become quiet while authorization, ordering, and uncertain-send protection remain in place. ## Linked Issues or Issue Description Stacked on #17576. Related delivery infrastructure: #15625. Related voice connection work: #11418. What existing behavior does this improve? Background phone reply delivery and recovery of abandoned dispatch intents. Current behavior Chat maintenance calls voice reply delivery every second. It also scans old dispatch intents. Publication transport starts a direct push before the final publication commit. The timer masks that timing gap. Proposed behavior Committed publication, reply, session, and configuration changes wake the shared coordinator. Ready replies drain in bounded passes. Saved rate-limit and dispatch-expiry deadlines retain recovery. An empty queue or a permanent delivery barrier has no recurring reply scan. Reason and benefit Remove recurring SQL from idle phone reply delivery without dropping replies, replaying uncertain sends, or adding a queue table. Breaking changes No API or schema change. Explicit manual recovery remains available. Inbound admission cleanup, history retention, and reports remain on their current paths for separate migrations. ## What Changed - Extract ordered reply state and multipart policy within the voice owner. Retain current authorization and atomic dispatch claims. - Register a scoped reply worker with startup reconstruction, fair work batches, failure backoff, drain/resume handling and shutdown ownership. - Extend the existing shared dispatcher with normalized partial-page results to preserve the bounded parent recovery contract. Existing complete-array consumers retain their calls. - Signal before relevant writes in their existing transactions. Both normal publication completion and operator resolution wake the actual conversation after commit. - Bound reply inventory to 100 total headers per refresh. Preserve partial scans, concurrent changes and newer denied-call restoration obligations across reconstruction. - Track abandoned dispatch actions independently of live calls. Quarantine expired intents without resending them. - Use existing active-call authorization checks to reactivate replies that were previously denied. Healthy call inspections do not rescan empty reply queues. - Remove the periodic reply scan and the early direct publication push. Keep provider requests outside database transactions. ## Verification Published head: `1435cab10be3f95c38ab6672d2a890ae49e9d300`. Voice recovery #17576 has landed at master `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. A normal merge reconciles squash ancestry while preserving the complete reviewed 31-file reply patch byte for byte and all unrelated master changes. The PR now targets master; fresh exact-head CI and Greptile are required. The prior 449-case/typecheck/build results below cover the identical owned source, not a new full-suite run. - 449 distinct targeted cases pass: 111 complete voice-store/reply; 19 complete parent recovery; and 319 complete tool-gateway, secret-proposal, shared dispatcher/coordinator/batching/startup/hostname cases. Production server typecheck and build pass. The policy checker and its Node tests pass. Later edits record evidence only. - Two real normal-publication/operator-resolution tests seed 105 unrelated calls and 105 future dispatches. Five publications selected 1,065/1,060 inventory rows before repair; both paths now pass the same 125-row total budget and deliver every answer once. This counts selected SQL results, not physical tuple visits or host savings. - Publication commits now signal their actual conversation. The voice reader returns at most 100 total session/action headers per refresh, preserves phase/cursor state and retries, and resets on worker registration. Current-state checks and dispatch claims still own provider effects. The 105-action case measures the page cap, restarts the same owner during reconstruction, waits at saved future deadlines and quarantines every abandoned action without resending. - A real 403 fixture proves that an older broad scan cannot erase a denied call first observed behind its cursor. Its owner callback still emits the committed restoration wake after scan completion. The existing actual-service authority-restoration and final-publication timing tests remain in the full suite. They are not replaced by direct helper calls. - Guarded removal of conversation scoping, cursor reset or the denial revision fence fails the intended budget, 100-versus-5 restart or missing committed-wake assertion; source is restored. An initial shadowed fixture name and an inadequate due-action reset probe are retained as failed attempts in the [current audit](https://github.com/paperclipai/paperclip/blob/d2c0a7de8c3e9cac8382b1268f3f5d883a2067b3/doc/plans/2026-10-25-voice-reply-delivery-sleep-audit.md#publication-inventory-and-paging-acceptance-october-28). - Parent recovery still passes paging, restart, behind-cursor insertion, feature changes, actual disablement, quiet health observations, current authority, exact-scope failure fences and wildcard-failure isolation through the shared worker. Four existing array-based worker consumers retain their API and pass their full suites. - The two voice fixtures self-migrate separate empty databases. Legacy tool-review/secrets suites use separate migrated clones. Earlier full-local attempts remain limited by embedded PostgreSQL setup and are not passing full-matrix claims. Older counts apply only to their stated revisions. Fresh exact-head Greptile, Ponytail if present, security and all CI remain merge gates. No live-provider, multi-writer, external-wake, deployed-sleep or savings claim is made. ## Risks - Missing a supported producer can strand work. Tests cover real enqueue, retrieval, normal publication, and operator publication paths. Startup reconstructs existing records. - Signals cover the process-local database owner. Independent SQL writers or other replicas need their own wake contract. - Authority restoration relies on the existing live-call recovery check. This dependency must be preserved if that owner changes later. - A provider acceptance receipt does not prove speech. Unknown outcomes remain blocked from automatic resend. - This change does not extend Cloud alarms, allow sleep during live calls, or prove deployed suspension or cost savings. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 07:50:27 -05:00
DottaandPaperclip	e1ba77f067	fix(server): schedule live voice recovery from session changes and deadlines (#17576 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain durable session state and provider identities. > - The chat maintenance timer scans live voice sessions even when no call exists. > - Session writes, saved expiry and active-call inspection deadlines identify real work. > - This pull request moves live-session recovery to the shared coordinator. > - Idle session recovery becomes quiet without losing exact-call cleanup or current authorization checks. ## Linked Issues or Issue Description Refs #17572. This PR is stacked on receiver discovery. It owns only live voice session recovery. Reply delivery, inbound admission cleanup, inbound history retention and call reports remain separate migration slices. What happened? The one-second chat maintenance lane reads voice settings and scans live sessions even when an instance has no calls or only terminal history. What did you expect? Reconstruct live calls on startup. Wake on committed session or configuration changes. Inspect active calls and retry cleanup at actual deadlines. Stop querying when no live obligation remains. Steps to reproduce Start an instance with no live voice sessions. Let startup settle. The old maintenance method continues to read settings and session inventory every chat maintenance pass. ## What Changed - Add a voice-owned bounded deadline reader and scoped worker on the existing coordinator. Retain concurrent writes across pages and reset partial scans on restart. No tables or migrations are added. - Separate generic connection health observations from configuration changes. Existing receiver and command consumers retain their notifications. Voice eligibility and provider operations remain in the voice owner. - Signal existing session transactions and configuration/feature changes. Keep provider calls outside database transactions. Standalone session writes use one small transaction helper. - Preserve saved expiry, the two-minute unknown-creation grace, active-call authorization and missed-hangup inspection, and exact-provider cleanup. Recovery never creates a provider call. - Retry transient authority lookup failures instead of treating them as confirmed revocation. - Fence expiration against a provider identity bound after discovery. A late known call remains eligible for cleanup. - Preserve work through rollback, drain, restart, concurrent refresh, backoff and shutdown. Page inventory and rotate bounded batches across companies. - Leave the four other voice maintenance duties in the periodic lane. - Document producer coverage and the supported writer contract in `doc/idle-voice-session-recovery.md` and the dated audit. ## Verification Published head: `f0b2e3cc22a18e7d08b10d4fc02aae4a19a52f30`. Receiver #17572 has landed in master at `dce004259394486f6361d0c0dd5a33a5272acc0a`. This normal merge reconciles squash ancestry; the complete 22-file voice recovery patch is byte-for-byte identical to the reviewed and tested `ad79c7e805` patch, while all unrelated master changes are preserved. The PR now targets master. Fresh exact-head CI and Greptile are running; interrupted Ponytail turns are not completed verdicts under the maintainer’s clarified merge policy, and an actual failing verdict still blocks. - 210 distinct targeted cases pass on the current repair: 19 complete recovery lifecycle cases and 191 complete voice-store, coordinator, batching, startup and hostname cases. Production server typecheck and build pass. Later edits record evidence only. - CI `40633667141` on prior head `1b6323c164` failed the feature-change assertion with `34050 < 25000`; its verification aggregate also failed. The test now disables automatically advancing fake time, drives 20 controlled scheduler turns and requires all 205 calls to close before an unrelated inspection deadline can mask a lost wake. Removing generation replay fails with missing closures. The repair does not benchmark database/host speed or raise the elapsed-time bound. - A real committed exact-call regression reproduces wildcard-backoff starvation. The worker keeps the broad retry but processes independent exact hints and cached due calls. Each check reloads current state and authority. Failed exact scopes remain fenced. Removing wildcard isolation fails the intended assertion; a successful broad retry then leaves the worker SQL-quiet. Full-row and lean queries now share the state/scope/cursor predicate. - Earlier acceptance on `712f3862b5` passed 725 distinct targeted cases. Its measurements remain useful history: five real metadata writes and one health check over 205 future calls selected 1,230 rows before the inventory repair and zero afterward, without changing the saved deadline. Those older test counts are not added to the current 210-case count. - Both external voice fixtures self-migrate; the current full voice-store run uses a separate initially empty database. Earlier fixture/method/observer failures, source mutations and partial edit attempts remain in the [current audit](https://github.com/paperclipai/paperclip/blob/ad79c7e805367fc7e2df3ea53c942ca6c7d27365/doc/plans/2026-10-25-voice-maintenance-sleep-audit.md#wildcard-failure-and-deterministic-generation-acceptance-october-28). Source is restored after guarded mutations. - Older CI `40632057899` failed because its trusted policy script was absent from the checkout, while build separately received a runner shutdown signal. E2E shards succeeded but their aggregate failed on policy. The current checkout includes the landed policy tooling and its checker/tests pass locally. The shutdown cause is unproven; neither old failed run is counted as passing. Earlier full-local attempts stopped on embedded PostgreSQL setup failures and did not pass. - Fresh exact-head Greptile, security and all CI must pass before merge. A completed failing Ponytail verdict blocks; an interrupted turn without a verdict does not. No full current local repository matrix, live provider acceptance, multi-writer support, Cloud wake, host suspension or savings is claimed. ## Risks - Process-local hints cover the existing active writer and shared clients. Independent SQL writers or another replica need explicit wake integration. - Membership, task binding, phone preferences and other authority inputs remain checked on the existing 30-second active-call inspection deadline and on use. This is not an instant cleanup guarantee for every authority mutation. - Unknown creation retains its existing maximum-duration grace. Cleanup failures retain retries. A provider request is never replayed merely because its creation response is unknown. - Genuine configuration changes retain bounded reconstruction. Metadata writes can probe parent eligibility, while health-only observations leave voice recovery quiet. Paging state is process-local and reconstructs from durable rows after restart. - Active calls, the other voice maintenance owners and connector transports remain real obligations. This does not expand Cloud alarm coverage, relax sleep eligibility or prove deployed suspension. - The receiver and Discord parents have landed. Current-master compatibility and exact-head CI and Greptile remain merge gates. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 07:38:48 -05:00
DottaandPaperclip	dce0042593	fix(server): wake chat receiver discovery from configuration and lease deadlines (#17572 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat receivers own persistent provider connections and fenced database leases. > - The chat maintenance timer repeatedly queries all eligible receivers, including when none exist. > - Configuration changes and known lease deadlines already identify when discovery is needed. > - This pull request moves receiver discovery to the shared delivery coordinator. > - Empty or settled discovery stops querying, while ownership and recovery remain explicit. ## Linked Issues or Issue Description Refs #17557. This is the next stacked slice after Discord command registration. The base is `codex/sleep-19-discord-command-registration`. What happened? The recurring chat maintenance lane scans receiver inventory and reconciles Discord and Photon runtimes even when configuration has not changed. Expected behavior Reconstruct receiver work at startup. React to committed configuration changes and local listener retirement. Retry foreign ownership at the saved lease expiry. Stop discovery when there is no remaining obligation. Steps to reproduce Start an instance with no chat endpoints. Let startup finish. Observe the recurring receiver inventory query from chat maintenance. With a configured receiver, remove its ownership token while listener shutdown is delayed and signal a configuration refresh: the old implementation can replace the listener before shutdown completes. ## What Changed - Preserve healthy receiver progress after a slow failed batch while retaining per-receiver retry backoff. - Register a receiver discovery worker with scoped refreshes, paged startup inventory, fair batches, and per-receiver or scope failure backoff. - Keep owned receiver identities cached without a discovery deadline, so deletion still stops their listeners. - Use existing transaction signals for configuration, credentials, health, access revocation, company deletion, and provider lifecycle changes. Rename the generic command-only helper to describe its broader configuration contract. - Keep lease-key selection with the receiver owner. Shared scheduling contains no provider protocol. - Retain stopping ownership until listener shutdown and lease release complete. Concurrent acquisition joins the existing stop promise. - Remove receiver discovery from the periodic chat lane. Preserve voice and the other unmigrated duties. - Record producer coverage, failure evidence, limits, and remaining work in `doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md`. ## Verification Published head: `ab39569250f4b699a0ab7a829e66dbe27c15020b`. Discord #17557 has landed at master `5bf0142d45`. A normal merge reconciles its squash ancestry by applying only the recorded 27-file receiver patch from parent `547fdf2139`. Every other path matches that master. Every server and package production/test file is byte-identical to the published `6e88ca0201` source that passed the 131-case run, typecheck and build. Unrelated upstream UI/eval/documentation changes are preserved. This PR targets master. Fresh exact-head reviews and CI are required. - 131 distinct targeted cases pass. Five complete receiver, coordinator, batching, startup, and GitHub health error-projection suites pass 99 cases. Twelve real Discord automatic-registration/receiver and Slack held-transaction cases pass separately, with 1,581 other chat integration cases deselected. The complete Photon channel suite passes 20 cases. - The standalone receiver and Photon fixtures each reproduced missing-table failures on separate initially empty PostgreSQL 17 databases. Both now apply pending migrations before client use; their complete 20-case files pass on those same databases. The large chat producer suite uses a separate migrated clone. - The old review concerns already have source fixes: discovery stops suppress their own completion wake at `1604a9fcba`; slow failed batches supply `nextRunAtOnError` at `ab56393acc`. Guarded mutations prove the existing regressions: reenabling the stop wake yields two inventory reads instead of one, and removing the error deadline fails the 1,100 ms provider-delay next-wake assertion. Both original source files are restored byte for byte. Normal full worker and real producer suites pass with those fixes. - All 131 distinct targeted cases, production server typecheck and build pass on source `6e88ca0201`. Exact source comparison carries that evidence to this normal merge; the unchanged suites were not rerun just for unrelated upstream changes. Receiver and Photon fixtures self-migrate separate initially empty databases; the large producer fixture uses its own migrated clone. Source is unchanged after these checks. Recorded listener-count warnings and controlled provider failure logs remain; no assertion was weakened. An initial whole-file identity check rejected expected parent queue additions; precise patch comparison confirms those declarations are correctly preserved. - Old server shard 12 failed because GitHub reports that its self-hosted runner lost communication. The grouped-test step is cancelled, and the retained log has no failing assertion or completed summary. The underlying host/resource/network cause is unproven. That interrupted job is not a pass; the updated parent and fixture source require a fresh full CI run. - [Current acceptance and failure record](https://github.com/paperclipai/paperclip/blob/ab39569250f4b699a0ab7a829e66dbe27c15020b/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#compatibility-after-discord-squash-merge-october-27). Prior 218-case and full-workspace results describe earlier revisions. No full local repository-suite, multi-writer deployment, Cloud wake, host suspension, or fleet cost claim is made. ## Risks - Process-local transaction hints cover the active database owner and its shared clients. Arbitrary direct SQL or independent configuration writers need explicit wake integration. Startup/resume reconstruction and known foreign-lease takeover remain supported; this does not add general multi-writer discovery. - Active receiver sockets and lease renewals continue. This change does not prove host suspension or extend Cloud alarm coverage. - Configuration signals can arrive during drain, initialization, or shutdown. The worker preserves dirty scopes and the owner joins listener retirement before replacement. - Voice, Teams, milestone projection, publication maintenance, GitHub recovery, and the remaining stack inventory are separate work. - The Discord parent has landed and the receiver child now targets master. Fresh exact-head reviews and all CI must pass before merge. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, code execution, and test tools. The runtime does not expose a more specific deployment identifier or context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 07:25:20 -05:00
DottaandPaperclip	a91f78e7e0	perf(chat): reduce inbound queue discovery cost (#17971 ) ## Thinking Path > - Paperclip helps people manage companies of AI agents. > - External chat messages enter a durable delivery queue. > - Queue reads must preserve pending wake ownership and conversation order. > - Retained processed history increases the estimated cost of a correlated pending-action test. > - PostgreSQL 17 enables JIT for these small selectors and reads extra JSON in deadline discovery. > - This pull request uses a scoped pending-action set for open-work discovery. > - The benefit is less database work with the same selected deliveries and deadlines. ## Linked Issues or Issue Description What existing behavior does this improve? Inbound delivery deadline discovery and conversation-head selection. Subsystem affected Server chat queue discovery. Current behavior A correlated `EXISTS` inside an `OR` has a high cost estimate on retained completed history. PostgreSQL 17 already hashes its pending-action read, but the estimate still enables JIT. Deadline discovery also reads JSON from completed deliveries that have no pending wake. Proposed behavior Use `(company_id, endpoint_id, delivery_id)` membership in the pending-action set for the two open-work selectors. Keep ready-only evaluation on its original scoped `EXISTS`. Keep retry, lease, ordering, lifecycle and endpoint rules. Reason and benefit The local retained-history fixture reduces deadline selection time. The small control and the final head selection are neutral. Ready-only selection remains an unchanged control. Breaking changes None. No API, schema, migration, stored history, worker, timer or notification changes. Related work: #17647 and #15927 change inbound wake-notice scheduling and selection. This PR changes the separate conversation queue ownership predicate. ## What Changed - Use a fully scoped pending-action set for open-work discovery. Exclude null delivery IDs to preserve exact boolean results. Keep duplicate actions from multiplying deliveries. - Preserve the original ready-only query. - Add five real PostgreSQL tests and a reproducible baseline/treatment benchmark. Capture the complete real service SQL, parameters, positional rows and execution plans. - Document the query cause, rejected hypotheses, test results and limits in `doc/plans/2026-10-27-inbound-queue-query-proof.md`. ## Verification Pinned pre-fix baseline: `09ad9adce2cf4742944353a3707a2f908e9bb35e`. Local fixture: Linux ARM64 PostgreSQL 17.11, Node 26.4.0. It has 1500 completed deliveries and actions, two open deliveries, and 32 KiB of synthetic incompressible JSON per historical delivery. A separate control has 20 completed rows. The normal settings use JIT on with a 100000 cost threshold, 128 MiB shared buffers and 4 MiB work_mem. The final quiet retained-history pair measured these local SQL medians. Each arm has two warmups and five alternating samples. The small control is neutral. \| Actual selector \| Baseline \| Treatment \| Complete positional output \| \| --- \| ---: \| ---: \| --- \| \| Next inbound deadline \| 18.78 ms \| 1.58 ms \| Equal \| \| Conversation head \| 5.71 ms \| 6.18 ms \| Equal \| \| Ordinary ready control \| 9.23 ms \| 8.33 ms \| Equal ; unchanged \| The deadline plan uses fewer shared buffers. Head physical reads remain unchanged. Deadline discovery avoids JIT on this fixture. The pristine head estimate is below the JIT threshold in both arms, and its timing is neutral. With JIT disabled, deadline discovery still improves; head and ready timings overlap. - Red: the unchanged baseline fails the explicit local fixture work budget for JIT and deadline buffer reads. Green: the treatment passes. There is no timing threshold in ordinary CI. - Five new PostgreSQL tests and six existing assembled chat tests pass. They cover scoped/null/duplicate pending actions, boolean results, payloads, head barriers, reactions, paused/attention/AgentMail exclusions, retry/processing deadlines, provider ordering, lease loss, lifecycle retry and edit/admission races. - Actual old/new ready selectors have equal error outcomes for unrelated malformed retry timestamps. An initial AND-only oracle comparison was inconclusive and is not defect evidence. - `pnpm -r typecheck` passes. - `pnpm build` passes. - Final head `f3cac298d8a8a5377127b0db7057c52019bbe426` passes the complete [CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40625611887). Greptile is 5/5 on that head with no unresolved review threads. The earlier end-to-end cleanup lock-timeout failure passed on this revision. - The complete local suite was not run. The host's known native PostgreSQL startup failure exhausts SysV semaphores. Focused integration tests use a disposable Docker PostgreSQL constructor through the existing test seam. Complete final-head CI is required. Reproduce the real SQL comparison: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-inbound-queue-discovery.ts --baseline-ref=09ad9adce2cf4742944353a3707a2f908e9bb35e --require-work-budget --retained-only ``` Use `--jit-off` for the diagnostic control. The default command uses the repository's bundled PostgreSQL version and records it in output. The reported 17.11 run used the existing test constructor seam with a disposable `postgres:17` container. Raw samples, complete queries, parameters and plans are emitted as JSON. The benchmark uses disposable synthetic data and does not read a live database URL. ## Risks - Query plans depend on data and PostgreSQL settings. These local measurements do not prove deployed speed or request latency. - Head selection still reads historical JSON. Action history remains stored and scanned. This is not a constant-time queue implementation. - The changed false branch has no retry timestamp cast. Ready-only parsing, claims and retries stay on their existing query. - Background SQL totals can overlap. Do not add these savings into an HTTP latency claim. The largest possible saving for an affected selection is bounded by its observed SQL time, about 49–53 ms on average in discovery. ## Model Used OpenAI Codex, GPT-6 family, with tool use and code execution. The runtime does not expose the exact model ID or context-window size. Three parallel agents own separate performance fixes; this PR contains only the inbound queue fix. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 07:19:22 -05:00
DottaandPaperclip	5bf0142d45	fix(server): schedule Discord command registration from saved deadlines (#17557 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connections register and verify provider commands in the background. > - Discord command recovery ran inside the recurring Gateway runtime scan. > - Its existing durable records already store verification and retry deadlines. > - This pull request gives command registration its own shared-coordinator worker and scoped commit notifications. > - Idle command work stops querying between real deadlines. Existing command authority and recovery remain intact. ## Linked Issues or Issue Description Refs #17554 and #17561. Both foundations have merged. This slice targets master. Runtime discovery and the other chat maintenance duties remain separate work. What happened? The chat runtime maintenance loop called Discord command registration on every reconciliation. It repeatedly read durable command state even when the next remote verification or retry was minutes away. Expected behavior Use committed producer changes and saved deadlines. Keep the existing five-minute verification policy, Retry-After, uncertain-write quarantine, namespace ownership, and credential fences. Do not add a database table for the timer. Steps to reproduce Configure a Discord connection and let command registration settle. In the parent revision, repeated runtime reconciliation still invokes command recovery. This revision reconstructs command deadlines at startup and removes that call from runtime reconciliation. ## What Changed - Expose normalized deadlines from the existing registration store. Park indefinite retries and invalid or foreign-owned records. Signal before store-owned claims, receipts, and retry-result writes. - Discover eligible endpoints in bounded pages. Refresh only affected endpoint, connection, or company scopes after commits. Dispatch at most 25 due commands per pass with fair company rotation. - Wire configuration, credentials, provider lifecycle/failure paths, inbound activation, generic connection mutation/removal/health, access revocation, and company deletion. - Register the worker through the chat service and shared coordinator. Preserve direct foreground setup calls. Refresh an existing local Gateway through its current owner after command work. - Preserve dirty work during drain and concurrent changes, back off unresolved attempts, and join admitted provider work during shutdown. - Document producer coverage, retained deadlines, supported writers, verification failures, and remaining runtime/Cloud obligations in `doc/idle-discord-command-registration.md` and its dated audit. ## Verification Published head: `547fdf2139144e1e6484198278bf4bb0eade8794`. It normally integrates master at `f8563d25077b17681ab6ef5949f7b07415a58ede`, including landed health #17554 and company deletion #17561. Squash ancestry caused merge conflicts; resolution applied this slice's exact 26-file owned patch from its recorded parent to pinned master. Every other path matches master. Migration 0394 and the current lockfile are preserved. - 141 distinct targeted cases pass on reconciled production source: 78 complete command-store, batching, coordinator and startup-timing cases; 8 real automatic Discord and held-transaction Slack cases; and 55 complete startup/export and company-deletion cases. The 1,581 other chat integration cases were deselected. - Server typecheck and build pass. The company-deletion fixture self-migrates an initially empty isolated database. The two Discord fixtures use separate migrated clones; this is not a fresh-schema claim for those fixtures. Signal-listener warnings remain recorded without assertion failures. - The parent integration preserves health and command notifications at shared producer sites. It changes no provider behavior, schema or coordinator contract. Prior producer, backlog, drain/resume, malformed-hint, transaction, namespace, shutdown and retry regressions remain in the complete command suite. - [Landing compatibility acceptance](https://github.com/paperclipai/paperclip/blob/547fdf2139144e1e6484198278bf4bb0eade8794/doc/idle-discord-command-registration.md#acceptance-after-the-company-deletion-parent-landed-october-27) records current scope and the earlier failures. Older full-matrix or combined-stack runs apply only to their stated revisions. - Fresh exact-head Greptile, Ponytail if present and all CI are required. No full local repository suite, live provider, Cloud alarm, host suspension or savings claim is made. ## Risks - Discord still has real five-minute verification and failure-retry obligations. This does not prove host sleep or external alarm coverage. - Runtime discovery, receipt-to-runtime projection, Gateway lease renewal and persistent sockets remain separate. The remaining runtime poll can still read command receipts; it no longer invokes the command protocol. - Process-local signals cover integrated writers. Independent SQL writers and other replicas require an explicit wake contract. Existing authorization and durable ownership checks remain authoritative. - Refreshes coalesce with a 30-second minimum spacing. Saved work is never discarded because a call returned without progress. - No schema migration, new table, workflow or lockfile change is introduced by this slice. Landed parent repairs are retained. The receiver child must receive this integration before landing. ## Model Used OpenAI Codex, GPT-6 family, with repository tools, code execution, and reasoning. The exact serving version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 23:32:24 -05:00
DottaandPaperclip	f8563d2507	fix(server): remove chat endpoints before company agents (#17561 ) ## Thinking Path > - Paperclip manages agent companies and their connected chat channels. > - A company deletion must remove dependent data in one transaction. > - Chat endpoints reference assigned agents with a restrictive foreign key. > - The current deletion order purges agents before those endpoints. > - This pull request removes company endpoint rows and chat history before issue and agent deletion. > - The existing lifecycle checks, company isolation, and rollback remain intact. ## Linked Issues or Issue Description What happened? Deleting a company with a chat endpoint fails on the endpoint's assigned-agent foreign key. The failure was found while verifying the company-deletion wake in #17557. Expected behavior Company deletion removes endpoint dependencies before their referenced issues and agents. It preserves other companies and global Discord ownership tombstones. A later failure rolls back the cleanup. Steps to reproduce Create a company, an agent, and a Discord endpoint assigned to that agent. Complete agent termination, then delete the company. The old order fails with the restrictive endpoint-to-agent foreign key. Paperclip version or commit This prerequisite includes merged #17554 at `94a780286f` and supports #17557. Only four cleanup-owned paths differ from that landed parent. Related #16465 covers the deletion feature flag, not dependency order. ## What Changed - Add a chat-owned company-deletion participant using the existing transaction contract. - Remove only the requested company's endpoints before issue deletion and the agent lifecycle participant. - Preserve namespace tombstones and existing restrictive foreign keys. - Add real PostgreSQL regressions for ordering, isolation, and rollback. - Migrate a validated disposable external database before fixture use, use the shared PostgreSQL bootstrap budget for setup, and remove fixture-owned Discord tombstones during test teardown. ## Verification Published head: `9766cc61c42fe0f73e241bbe99f9d9f145f4b11c`. Parent #17554 is merged. This branch reconciles the actual landed health squash `94a780286f` and targets master. Fresh reviews and CI are required for this exact head. - The latest review requested the repository's shared EMBEDDED_POSTGRES_TEST_TIMEOUT_MS (90 seconds) for bootstrap. This replaces the earlier suggested 30-second limit. All six existing cases pass on another initially empty database with that constant; no test assertion or production code changes. The documented contention budget is repository evidence, not a locally reproduced 40-second bootstrap failure. - All six complete cleanup-removal service cases pass on an initially empty dedicated PostgreSQL 17 database with the updated health parent. - The pre-fix real endpoint-deletion case failed with missing-table errors because external fixture setup skipped migrations. Applying pending migrations before client use fixes that failure; no assertion is removed or weakened. - An independent query after the complete suite finds zero fixture ownership tombstones. Production company deletion still preserves those records. - Cases cover endpoint removal before issue and agent references, other-company isolation, namespace ownership preservation, and rollback after a later failure. Both product regressions failed before the original ordering fix. - Server typecheck and build pass with the updated parent and fixture repair at `21be75d2b0`. The landed-parent reconciliation changes only the lockfile; all source and test bytes remain identical. All six cases pass again with the new installed dependency patch. The initial scope-proof command raced a shared `origin/master` update; a new proof pins the actual landed parent and confirms the four-file scope. A later native-checkpoint merge `09ad9adce2` has no overlap with these four files. - [Owning acceptance document](https://github.com/paperclipai/paperclip/blob/9766cc61c42fe0f73e241bbe99f9d9f145f4b11c/doc/company-chat-deletion.md#current-parent-acceptance-october-27) retains the failing baseline, parent scope, and limitations. No full local repository-suite or deployed suspension claim is made. Older inherited gateway/combined-stack evidence is separate from this four-file fix. ## Risks - The endpoint cleanup shares the caller's transaction. A later deletion failure must restore the endpoint; the regression verifies that boundary. - Existing retained voice-history restrictions and external provider teardown policy remain unchanged. This fix does not bypass restrictive foreign keys. - No migrations, lockfile changes, or workflow edits. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code edits, and tool-assisted tests. The exact runtime variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 23:06:31 -05:00
DottaandPaperclip	94a780286f	fix(server): schedule tool health from committed changes and deadlines (#17554 ) ## Thinking Path > - Paperclip manages AI-agent work and its connected tools. > - Idle hosted instances need their database and network activity to stop. > - The maintenance tick queries tool connections even when no health check is due. > - Existing connection rows already contain eligibility and the last health observation. > - This pull request reconstructs health deadlines and refreshes them after committed changes. > - The shared coordinator preserves retries, drain, and shutdown without another private timer. ## Linked Issues or Issue Description What happened? The aggregate maintenance tick scans connection health every 30 seconds. Empty instances and connections with fresh health observations still cause SQL activity. Expected behavior No eligible connection means no recurring health SQL. Eligible connections retain the existing 15-minute proactive check. Configuration changes and health observations update the schedule after commit. Steps to reproduce Start a server with heartbeat scheduling enabled and no tool connections. Let startup finish and observe two maintenance ticks. The old code repeats the health inventory query. The new startup regression verifies that those ticks do not invoke the health inventory. Paperclip version or commit This branch includes merged parent #17518 and master `96727b3cfefb61f85ffe1fbc29a9b6dd6837efe8`. The 47 changed paths belong to the single tool-connection health duty. No database file differs from master. Related: #14638 covers runtime-auth exclusions. #15448, #15325, and #11910 concern MCP failure classification. This PR preserves those provider policies and does not replace their work. ## What Changed - Keep the GitHub error-projection unit fixture at the health-write boundary and verify the real transaction path in the PostgreSQL health-worker suite. - Add a health schedule owned by tool access. Read lean eligibility fields and saved observation timestamps. - Register health work with the existing coordinator. Refresh only dirty connection/application/company scopes using bounded inventory pages, check due connections in fair company batches, continue concrete due backlog after one second, and retain failures with backoff. - Notify before supported eligibility and observation writes in their owning transactions. Cover API/gallery setup, OAuth, catalog/gateway observations, email, voice, access removal, deletion, and settings. - Commit connection creation with grant, binding, and runtime-slot SQL before its wake. Keep provider calls outside the new transactions. - Keep provider-to-transport mapping in the chat catalog. SDK-only chat changes do not wake health work. - Replace aggregate health scans, retain the public manual sweep, and document the producer audit and remaining sleep limits. - Keep metadata-only example and Smoke Lab installs scoped. Lock and re-read application type before publishing a real eligibility change. - Keep failed inventory reads on their own retry deadline while healthy due batches continue. - Apply fixture migrations before using external disposable databases in all six producer suites. ## Verification Published head: `8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62`. Fresh CI and reviews are required for this exact head. - 623 distinct targeted cases pass. The final four complete health, fair-batching, GitHub health recovery, and startup suites pass 80 cases after the retry fix. Five complete producer suites pass 543 cases: tool access 469, access 18, email 45, GitHub events four, and Smoke Lab seven. The earlier 26 health cases are part of this total, not additional cases. - The six producer suites first passed against separate, initially empty PostgreSQL 17 databases at `acbf262843`. They apply their own migrations. The subsequent repair changes only the health worker and its regression; its complete final consumer suites pass on that integrated parent. Server typecheck and build pass with the final repair. - Installer amplification: five real installs with 205 unrelated connections reproduced 1,035 example rows and 1,045 Smoke Lab rows. Repeated metadata installs now load only changed fixture connections. The same cases restore an excluded application through each installer and require all 205 unrelated IDs to re-enter the schedule. - Concurrent eligibility: a real PostgreSQL lock-wait case requires the installer to notify after a concurrent type change. A mutation that retains the lock but uses the old lookup fails the settled-hint assertion; original source was restored byte for byte. - Retry isolation: the failing baseline read one unavailable company three times while another company's 60 due checks drained. Final tests require one initial read, at least 30 seconds between failed reads, prompt healthy progress, eventual recovery, and SQL quiet after settlement. - Real-service producer coverage includes AgentMail activation/pause, GitHub installation changes, and member access removal. Lifecycle coverage includes outer commit/rollback, startup reconstruction, drain/resume, failure, and shutdown. - Failure history and exact evidence scope: [health acceptance](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/idle-tool-connection-health.md#acceptance-follow-up-after-the-plugin-parent-merged-october-27), [producer audit](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/plans/2026-10-24-tool-connection-health-sleep-audit.md). Prior remote failures and earlier 706-case evidence remain historical. Startup tests emitted listener-count warnings; all assertions passed. No full local repository-suite, deployed sleep, fleet savings, or live provider qualification claim is made. ## Risks - Supported live writers must use the transaction signals. Independent SQL writers and other replicas need their own wake integration. - Configured eligible connections still have a real proactive health obligation. This does not prove Cloud alarm coverage or host/database suspension. - Existing manual probes and catalog operations can overlap. This migration does not introduce a credential-version CAS or change provider failure policy. - Voice remains eligible under the previous health policy. Its generic probe can still report an unsupported transport; changing that behavior is separate work. - No database migration, new table, workflow, or lockfile change. This PR includes its merged parent and current inspected master; later stack PRs and all current-head acceptance gates remain separate. ## Model Used OpenAI Codex, GPT-6 family, with repository tools, code execution, and reasoning. The exact serving version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 21:37:50 -05:00
DottaandPaperclip	8406047b4e	fix(server): schedule join request retention from durable deadlines (#17508 ) ## Thinking Path > - Paperclip manages agent companies and their access requests. > - Pending join requests appear in the attention feed and can age into automatic archival. > - The maintenance tick reads those requests even when no archive is due. > - Existing request rows already store the activity timestamp needed for retention. > - Committed changes can wake a worker that registers the next actual deadline. > - This PR gives join-request retention its own bounded worker and removes its recurring feed scan. > - Idle and resolved history stop producing retention traffic without changing access approval. ## Linked Issues or Issue Description What happened? Every retention maintenance pass reads pending join requests through the attention feed. Empty, resolved, kept, and future-due states still participate in recurring discovery. Expected behavior Committed request and retention changes wake the owner. It waits until the next actual archive deadline and stops querying when no eligible request remains. Existing approval and visibility behavior is preserved. Steps to reproduce Start an idle company with no pending join requests, or with requests younger than the 90-day archive threshold. Observe the maintenance feed repeatedly select join_requests. No state change is needed to cause those reads. This PR targets master after #17505 merged. Approval retention is #17497. Other attention source kinds remain separate migration duties. No duplicate open join-retention PR was found. ## What Changed - Add a join-request retention owner that preserves pending status, active-company visibility, matching invite ownership, and keep/archive controls. Invite expiry or revocation retains the existing feed policy. - Reuse one `createDecisionRetentionSweep` for approval and join paging, atomic retention and finite scan cycles that revisit failures despite continuing arrivals. Eligibility and source-row locks stay with each owner. - Register real saved deadlines, rounded upward in SQL to millisecond precision, with the existing coordinator. Preserve 100-row pages and thirty-second notification coalescing without idle polling. - Materialize unseen join state inside the existing authorized keep/archive/revive transaction, preserving archive choices and source locking. - Notify before request inserts and updates in the existing acceptance and Slack transactions. Use a small SQL-only helper for replay and approval/rejection updates. Company status, budget pause/resume, deletion, and retention controls notify this owner too. - Preserve the approved-only API-key consumption path without a new wrapper; it cannot produce pending retention work. - Exclude join requests from maintenance feed collection while retaining normal UI reads. - Add an idempotent concurrent partial pending index and prune old schema snapshots through the existing generator. - Document producer coverage, boundaries, failure evidence, and remaining work. ## Verification Test revision: `df1815fbba`. Normal integration of landed workspace precision `d23dd19750` has an identical complete source tree. This remains only the join-retention duty and its shared approval-retention mechanics, targeting master. - 55 worker cases pass, including two new PostgreSQL regressions that failed before repair: continuing overdue arrivals no longer postpone a recovered failure indefinitely, while an entire failed page still allows later healthy rows to progress. - The nine complete suites passed 341 cases but failed an attention cleanup hook after its 16,400-run case. After vacuum/analyze of only that disposable fixture, the full 31-case attention suite passes unchanged. Together with the original eight passing suites (311 cases), all 342 distinct cases pass across complete-suite executions. Recovery is established; the original cleanup slowdown is not claimed explained. Server typecheck and production build pass. - Earlier CI at `df1d4d0` failed the unchanged decision-expiry restart deadline assertion (a thirty-second retry appeared before the saved expiry). All twenty unchanged expiry cases pass locally, but the CI timing cause remains unproven. No retry, timeout relaxation or assertion change is used to dismiss it; fresh final-head CI remains required. - Five regressions failed before repair: three unseen join controls returned 404, and both owners advertised a microsecond deadline one millisecond early. All pass. Two old real-producer expectations failed after the correct rounding change; they now derive exact expected milliseconds independently from database integer microseconds instead of truncating a JavaScript Date. - The initial timing-only concurrency test still passed a missing-lock mutation. It now observes the actual PostgreSQL wait before releasing the writer. Production passes; removing the lock fails with zero waiters instead of one. Source is restored before commit. The archive-first committed-order case confirms that later activity stays archived until explicit revival; no broader race guarantee is claimed. - Generated concurrent/idempotent `0393_open_grandmaster.sql` adds only the pending join index. All 391 parent journal entries and historical SQL files remain unchanged. Five generated snapshots remain (0389–0393). Migration safety/numbering, schema drift and actual apply/replay on six isolated PostgreSQL fixtures pass. - Existing lifecycle cases retain real HTTP invite acceptance/approval/rejection, Slack identity linking, rollback/outer commit, source visibility, failure rotation, drain/resume and shutdown coverage. Earlier stack setup and CI failures remain in the owning audit and stack ledger; they are historical evidence, not this head's acceptance. Commands: server Vitest for `join-request-retention-work`, `approval-retention-work`, `approvals-service`, `attention-service`, `decision-retention-service`, `budgets-service`, `tool-gateway-service`, `server-startup-feedback-export`, `scoped-deadline-work`; server typecheck/build; DB migration checks, snapshot drift and actual apply/replay. Fresh exact-head CI, security and Greptile/Ponytail 5/5 remain required. No full local repository-suite, deployed suspension or cost claim is made. ## Risks - A future supported producer must signal its transaction. Independent SQL writers and another active server process are outside the process-local notification contract. - A writer that wins the source lock first is rechecked before archival. If archival commits first, later activity preserves the archived state until explicit revival. Existing invite replay expires before the retention cutoff and Slack replay deduplicates. Future producers needing renewed visibility must request revival. Cross-table visibility retains the existing contract. - The worker registers only under the existing maintenance enablement switch. The internal deadline does not add Cloud alarm coverage or relax conservative sleep blockers. - New eligibility can wait within the thirty-second aggregate notification window; known deadlines preempt it. - Archival preserves pending approval status. Other attention sources and aggregate maintenance remain separate work. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact serving model identifier and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 20:03:16 -05:00
DottaandPaperclip	6e5b9794d6	perf(github): avoid duplicate webhook body reads (#17894 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub webhooks enter a durable action queue before the worker processes them. > - A worker must check retry eligibility and win an atomic claim before it replays a delivery. > - The current eligibility read returns the complete webhook body, and the successful claim returns it again. > - This pull request reads small eligibility fields first and returns the body only to the successful claimant. > - The worker keeps its retry, ownership, runtime, credential, and recovery checks. > - The benefit is less database transfer without removing durable replay data. ## Linked Issues or Issue Description Refs: #17666, #15981. Those changes own the existing recovery scheduler and ingress module. This PR changes the result projections and validation order within that module. It has no dependency on another open PR. A public issue and PR search found no duplicate change. What existing behavior does this improve? The durable GitHub webhook worker reads action eligibility and replays claimed deliveries. Current behavior The first read returns all action columns and the complete webhook body. A successful claim returns them again. An active claim or future retry also downloads a body that this worker cannot process. Proposed behavior Read only ID, status, result, and update time for eligibility. Return ID, endpoint ID, and payload from the successful atomic claim. Validate that claimed payload before replay. Reason and benefit A synthetic 131,105-byte webhook body produces about 263 KB of response values before this change. The changed service returns about 132 KB for successful processing and fewer than 150 bytes for skipped work. This removes duplicate body transfer. Breaking changes There is no public API or schema change. Malformed active or delayed rows now remain under their current claim or retry deadline. The eligible claimant validates and fails them later. A worker that does not own the claim can no longer clear their payload. ## What Changed - Use a small projection for initial action eligibility. - Return the replay body only from a successful claim. Keep the existing status, stale-time, and previous-attempt guards. - Validate the row returned by the claim. Fence invalid-payload settlement by kind, processing status, and the claimed attempt. - Add 19 real PostgreSQL regressions for complete replay bodies, skipped work, retries, stale-claim recovery, contention, corrupt bodies, claim loss, replacement bodies, and payload retention. - Wait for the saved chat authentication link in one existing async UI test. Keep its exact destination and all other assertions. This repairs a second independent CI test failure without changing product UI. - Accept the two precise invalid-repository diagnostics from Git 2.55 and 2.56 in one existing test. This repairs an independent CI compatibility failure. It changes no Git expiration service behavior. - Add a reproducible benchmark that loads the real baseline service from an explicit commit. It captures generated SQL and parameters, query plans, response-value estimates, timings, and complete valid-work behavior comparisons. ## Verification - Baseline: `663fe667d66d3863d0919a8724f1d99500feef0b`. - PostgreSQL 17.11, Node 24.22.1. The fixture uses synthetic random body content. No hosted database or provider was changed. - `pnpm exec vitest run server/src/services/chat-channels/github/webhook-ingress.test.ts`: 19 passed with a dedicated disposable PostgreSQL database. - Existing assembled GitHub tests: recovery from pre-ingress failure, scheduler retry and restart, exact terminal redelivery, runtime pause/resume, recovery tombstones, and credential rotation passed. - `node scripts/check-module-boundaries.mjs` and `git diff --check`: passed. - Benchmark: `node --import ./server/node_modules/tsx/dist/loader.mjs scripts/benchmark-github-webhook-ingress.ts --baseline-ref=663fe667d66d3863d0919a8724f1d99500feef0b`. The default uses disposable embedded PostgreSQL. `PAPERCLIP_TEST_DATABASE_URL` can select a dedicated migrated test database with a name that ends in `_test`. - Ten timed samples after two warmups, with alternating baseline/treatment order. The main acceptance criterion is response size, rather than a timing threshold. The 0.51 ratio bound is deterministic for this fixed synthetic fixture; it is not a promise for arbitrary webhook bodies or result metadata sizes. Focused tests measure the body transfer reduction directly. Actual wire savings require post-rollout measurement. \| Scenario \| Baseline response-value bytes \| Changed response-value bytes \| \| --- \| ---: \| ---: \| \| Received \| 263,276 \| 131,547 \| \| Due retry \| 263,342 \| 131,613 \| \| Stale processing \| 263,291 \| 131,563 \| \| Future retry \| 131,692 \| 136 \| \| Active processing \| 131,642 \| 86 \| \| Processed \| 131,659 \| 103 \| The benchmark compares response status/body, all stable durable action fields, and the replay body, signature, delivery ID, and provider routing. All six valid scenarios match. It excludes generated row IDs and timestamps. The final measurement records treatment commit `f1ad8ce838700237b56f895cc61a9c6ed0850cd6`, an empty initial worktree status, and the SHA-256 of the actual source. It ran from the server directory to verify that Git paths resolve from the repository root. Local timing ranges overlap and vary with host load. An earlier loaded repeat had a due-retry median of 19.61 ms before and 22.58 ms after, so no consistent latency gain is claimed. The byte reduction is the verified result. Response-value estimates exclude PostgreSQL protocol/TLS overhead and JSONB whitespace. These are local measurements, not a claim about deployed latency or a monthly bill. `pnpm -r typecheck` and `pnpm build` passed on production-change commit `6f4f36bff525494dadda30b49ea6a9a33976e1ab`. Greptile completed with 5/5 and no review threads on that commit. Commit `2abf9b1e50c568b632e7568398a5826a6122e456` changes only the Git test matcher. Commit `f1ad8ce838700237b56f895cc61a9c6ed0850cd6` fixes benchmark provenance after automated review. Commit `ee2a96dad549c72da3132d2b93fd11eb1c45bc31` repairs an existing async UI test. Final commit `994d398d12be65f595b305d1c2f8a61b9d587ff9` extends real PostgreSQL contention coverage after review. Full CI completed successfully on that final head: [workflow run 40537698349](https://github.com/paperclipai/paperclip/actions/runs/40537698349). All 55 successful checks, one neutral review, and two optional skipped checks are terminal; none are pending or failing. Fresh Greptile and Paperclip reviews are both 5/5, with zero unresolved review threads. The production source SHA-256 remains `402f9c66cd030c9ded9f39a3bc72d26c56627752c0950b102adcab15c4512df3`, so the production proof is unchanged. The full local `pnpm test:run` was not completed because this host exhausted embedded PostgreSQL semaphore resources; the focused real PostgreSQL tests used a private disposable container. Current-head remote CI passed the full suite. A direct server typecheck before runner preparation reported missing runner build outputs; supported preparation and the full typecheck then passed. The first complete build found a partial local Copilot closure from the dependency cache. The complete pinned closure was restored and verified, then the complete build passed. The first local test run also found a fixture teardown order error; the teardown was corrected, then all 17 tests passed. Frozen dependency installation failed on the existing master patch metadata. Independent cached dependency directories were cloned, their workspace links were checked, and no dependency manifest or lockfile was changed. The first CI server4 shard failed one existing Git expiration test: Git 2.56 reported `gitfile does not point to a valid repository`, while the test expected the Git 2.55 phrase `not a git repository`. The service returned the correct `inspection_failed` result and preserved the workspace. Separate test-only commit `2abf9b1e50c568b632e7568398a5826a9c6ed0850cd6` accepts either precise diagnostic. The affected assembled test passed locally with real PostgreSQL 17.11 and Git 2.55; the captured CI diagnostic also matches, and unrelated permission errors do not. This was a compatibility failure, not a transient failure or an unchanged rerun. Paperclip Review found that the benchmark recorded its own scratch directory as an uncommitted change. The final benchmark-only repair captures repository status before scratch creation, runs Git commands from the repository root, records the source hash before measurement, and rejects source edits during measurement. A full benchmark run from the server directory then reported clean provenance and passed all six byte/behavior comparisons. The review thread was marked addressed after this proof. The next full CI run found a separate unchanged UI test failure in `ConnectionIntentInteractionBody.test.tsx`: the saved authentication link was still absent after one fixed flush. The same failure reproduced locally: one failed and 50 passed. Separate test-only commit `ee2a96dad549c72da3132d2b93fd11eb1c45bc31` uses the existing bounded `waitForAssertion` helper around the identical destination assertion. The whole file then passed all 51 tests, and UI token gates passed. The no-extra-setup and no-automatic-completion assertions remain. No UI runtime or timeout changed. Paperclip Review also asked for parallel claims from stale-processing and due-failed rows. The contention test now covers those states and received rows. Each case holds the claims until all 12 workers have read the same eligible attempt, then uses real PostgreSQL to assert one replay and exactly one attempt increment. The whole focused suite passed all 19 tests. This coverage change does not modify production code or the measured queries. The review thread was marked addressed after this proof. ## Risks - Payload validation now follows claim commit. If the process stops there, the existing stale-claim recovery path retains the body and retries it. Tests cover fresh and stale processing rows. - A malformed future retry now fails when due instead of being cleared by an unrelated early worker. The claimed-attempt guard prevents invalid settlement from overwriting a later claim. Terminal invalid-payload settlement intentionally retains the baseline result shape (`code` and `retryable: false`, without `attempts`). Changing that result contract is outside this byte reduction; the non-retryable row cannot become due again. - Existing runtime-generation, credential, recovery-source, replay-barrier, redaction, and retry checks remain in place. Existing assembled tests cover these boundaries. - There is no migration, startup job, dependency change, or deployment in this PR. ## Model Used - OpenAI GPT-6 through Codex. The exact runtime model ID and context window are not exposed by this environment. Capabilities used: source analysis, code editing, shell tools, PostgreSQL experiments, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (The public contract is unchanged; benchmark usage and evidence are documented above.) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 12:05:30 -05:00
DottaandPaperclip	7a0c9c23cf	perf(github): reduce inherited review-wait transfer (#17893 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review tasks can retain a passive wait after a review is published. > - The wait check reads current requester, task, agent, repository, and publication bindings. > - It also transfers the complete assessment for one commit comparison. > - Large review summaries and findings add repeated database transfer. > - This pull request selects only the typed reviewed commit value. > - The wait keeps its existing checks and transfers less data. ## Linked Issues or Issue Description What existing behavior does this improve? The inherited GitHub review-wait check in the server. Current behavior The joined query selects the complete assessment JSON. The caller uses only `assessment.reviewedCommit`. Proposed behavior Select `assessment->'reviewedCommit'`. Keep its JSON type and the strict comparison with the head SHA. Keep all joins, filters, ordering, and publication checks. Reason and benefit The synthetic PostgreSQL 17 fixture reduces one result DataRow from 6,874 bytes to 423 bytes (93.85%). This measurement excludes response metadata, TLS, and TCP. It does not predict a monthly bill or deployed savings. Breaking changes None. No public API, stored data, schema, or execution policy changes. Additional context Related work: #17562 narrowed a different external-review candidate query. This PR changes the inherited published-review lookup. The duplicate search found no matching open change. ## What Changed - Select the commit as JSON, rather than text, to preserve strict comparison for legacy numeric, object, array, missing, and null values. - Add real PostgreSQL regressions for narrow responses, unchanged stored assessments, revocation, tenant and agent bindings, and a newer queued head. - Add a synthetic benchmark that captures the real baseline and treatment SQL, compares outputs, and measures raw DataRow bytes. - Document the read contract and benchmark command. ## Verification - Baseline: `663fe667d66d3863d0919a8724f1d99500feef0b`. - `vitest run server/src/__tests__/github-review-wait.test.ts`: 14 tests passed on disposable PostgreSQL 17.11. - Restoring the baseline query makes the response-size regression fail: 8,823 column-value bytes exceed the 1,000-byte bound. Restoring treatment passes. - Benchmark: `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-github-review-wait.ts --baseline-ref=663fe667d66d3863d0919a8724f1d99500feef0b`. - The benchmark compares every unchanged SQL column and full service output. It also compares 15 JSON cases and live connection revocation. - Two warmups and seven alternating samples had overlapping latency ranges. No latency improvement is claimed. - `pnpm -r typecheck` and `pnpm build` passed locally. - Twenty existing published-review consumer regressions passed on PostgreSQL 17.11. These cover continued review, a newer head, publication state, and a connection-disable race at final commit. - Final-head focused tests passed again after the benchmark-only review fix. - Final benchmark treatment: `d94410a6e6bb83e727afdf09cff450a91a89edd6`, with an empty source diff. DataRow bytes remained 6,874 to 423. Warm medians were 3.11 ms before and 3.43 ms after, with overlapping ranges. This proves byte reduction, not a latency improvement. - A provenance check with one staged and one unstaged source edit confirmed that the benchmark records both from the documented server working directory. Those temporary edits were removed. - All final-head CI checks passed for `d94410a6e6bb83e727afdf09cff450a91a89edd6`. Greptile scored this exact commit 5/5. There are zero unresolved review threads or recommendations. The independent Paperclip review also scored this commit 5/5. - The first local embedded PostgreSQL attempt failed before assertions because the host exhausted System V semaphores. A direct `initdb` reproduced `could not create semaphores: No space left on device`. The passing focused proof used an owned disposable Docker PostgreSQL 17 container. `pnpm test:run` was not completed locally because of that host limit. The full final-head CI suite passed. - The final CI aggregate briefly reported `in_progress` after all of its steps and the workflow had completed successfully. One rerun of only that aggregate produced a terminal success. The test matrix was not rerun. - The initial frozen install hit the existing upstream patch/lock configuration mismatch. A cached non-frozen local install succeeded. The generated lock change is excluded from this PR. ## Risks The selected commit must retain its JSON type. Using `->>` could turn numeric `1` into text and accept a malformed assessment against head `1`. The regression covers this case. The complete assessment remains available for actual review and publication. Existing fresh authority checks remain in place. No migration is required. ## Model Used OpenAI GPT-6, accessed through Codex. The exact runtime model ID and context-window size were not exposed in this session. Used repository inspection, code execution, and PostgreSQL validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 12:05:01 -05:00
DottaandPaperclip	585b4e6af8	fix(chat): preserve deferred Retry source batches (#17883 ) ## Thinking Path > - Paperclip manages agent work through tasks and governed execution. > - Connected conversations retain the source and authority for each request. > - Task Retry pins the full saved GitHub input batch before it waits for cleanup. > - Resuming that wait could append an independent board queue after authorization. > - Dispatch then rejected the changed batch before the provider started. > - This pull request keeps the authorized batch intact and restores certified undelivered input from historical failed attempts. > - Each board message retains a separate turn and its own admission checks. ## Linked Issues or Issue Description What happened? A deferred GitHub task Retry could consume a separate board message while promoting its saved receipt. The new run then failed with `chat_failed_run_retry_not_authorized`. A second Retry could also fail because the historical run retained the mixed batch. Expected behavior Retry must preserve the source batch it authorized. Separate board input must keep its own execution path. Historical input may be restored only when durable evidence proves that the rejected attempt never started a provider. Steps to reproduce 1. Retain ten authorized GitHub requests behind a failed native turn. 2. Start task Retry while the predecessor still lacks its exact termination receipt. 3. Save an independent board message. 4. Complete cleanup and resume the same Retry receipt. 5. Observe the mixed batch rejected during dispatch. Retry that failed attempt to reproduce the second refusal. Refs #17725 for the existing saved-input Retry contract. This change repairs the later queue merge and its retained failed state. Related #17878 covers snapshot continuation and later turns in recovered directories; its workspace repair can be reviewed independently. ## What Changed - Keep deferred chat Retry promotion scoped to its exact receipt. Ordinary board Retry coalescing retains its existing behavior. - Let a new explicit task Retry restore a historical mixed batch only after strict never-started, source, ownership, allocation, board-comment and prior-delivery checks. - Preserve failed-run history and original budget-skipped receipts. Record restored queue ownership in activity. - Add production queue and HTTP-route regressions. Update the execution contract and verification maps. ## Verification - Before the repair, the deferred ten-input batch failed before identity initialization. The same regression now reaches the real initializer with the exact GitHub batch. The independent board message then reaches its own initializer in a separate turn. - Before historical recovery, ordinary task Retry refused the persisted mixed failed batch. The repair passes that case without changing the failed run. HTTP duplicate requests retain one Retry owner. An additional compatibility probe restored the old queue candidate predicate and removed all historical mixed-state fixture writes: the real producer created the exact batch and the complete historical HTTP recovery case passed (1/1). The fixed-source historical fixture now uses the production serializers. - The affected Retry group passes 86 cases on commit `325c00158df5569f5b42731cda4ecc4584909d99`, including already-coalesced external receipts. Refusals cover edited sources, active controllers, unresolved allocations, changed receipts, foreign external receipt ownership and prior delivery of either board or GitHub input, including immutable budget-skipped receipts. - Full workspace `pnpm -r typecheck` and `pnpm build` pass with these source changes. Map, module-boundary and diff checks pass. - Full local `pnpm test:run` was started. The host cannot create additional PostgreSQL semaphores; database-dependent suites fail or skip at startup. The affected database group runs against a separate migrated PostgreSQL 17.10 test database with Node 24.22.0 on macOS arm64. - The regression deliberately stops after the real identity initializer to avoid provider allocation. It proves HTTP, queue, source authorization, identity and separate continuation behavior. It does not claim a live model result, GitHub publication, deployment or recovery of the original hosted task. - Current head `325c00158df5569f5b42731cda4ecc4584909d99`: all checks are terminal with no failures or pending checks; required `ci / verify` and `ci / e2e` pass. Conditional Storybook checks are skipped. Greptile and Paperclip each reviewed this exact commit at 5/5. Both earlier review threads are resolved. Paperclip retains one optional robustness suggestion for batch ordering; the verified historical producer order remains a strict recovery contract. ## Risks Restoring input could repeat work if the prior turn had executed. The existing strict bootstrap proof rejects provider output, native state, process or controller evidence, and allocations. Restoration also checks current external authority, unchanged board revisions and other run history. An uncertain executor remains held. No migration, provider fallback or relaxation of dispatch validation is included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code editing and test execution. The exact runtime model ID and context-window size are not exposed in this session. A separately shared regression supplied independent reproduction evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected database regressions; full local suite has the host limitation described above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 11:34:18 -05:00
DottaandPaperclip	13709b4c7a	feat(connections): let agents configure connections within delegated authority (#17737 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections provide governed tools, chat channels, model accounts, and gateways. > - Several management routes require a human board actor even when an agent has the required grant. > - Default agents must configure these resources within the responsible user's authority. > - This pull request adds a shared authorization path to the existing REST and native API surface. > - Human authentication, execution approvals, and local host boundaries remain enforced. > - The benefit is that a standard agent can complete authorized setup with a bounded human handoff. ## Linked Issues or Issue Description Subsystem affected Server authorization, connection services, shared contracts, native API discovery, database schema. Problem or motivation Connection administration is available in the web app but often unavailable to agents. Agent grants, responsible-user authority, credential ownership, and actor attribution need separate checks. Opening gateway-token creation also requires preserving the issuing agent's delegated authority during token use. Proposed solution Use the existing authorization evaluator for every connection family. Enforce current responsible-user authority, effective trust, grant scope, resource visibility, and target-agent access. Keep existing agent grants unchanged. Expose authorized configuration through default `search_api` and `call_api`. Save human authentication steps and let agents continue after completion. Roadmap alignment This change completes the API access path for the existing Connected Apps, Personal & Shared AI Accounts, and governed MCP Gateway capabilities in `ROADMAP.md`. It follows the maintainer's requested design. Related work: #11899 enforces connection execution access and creator/manager rules; this PR extends agent management across connection families. It does not replace that execution-policy work. ## What Changed - Add shared connection management context and mandatory responsible-user, trust, scope, and target checks. - Allow authorized agents to configure tool, chat/email, GitHub/Slack, AI, profile, and gateway resources through existing routes. - Preserve human credential ownership and record the actual agent, responsible user, and run or key in audit data. - Preserve saved browser authentication handoffs and callback session checks. - Bind agent-issued durable gateway tokens to their responsible user and recheck delegated authority at use. - Add explicit REST transport and management capability metadata, built-in guidance, and sensitive-response handling. - Intersect caller and endpoint-agent Slack test/invitation policies, including independent and shared rates. Require the addressed human’s current endpoint identity link before completing identity handoffs. - Add production-authenticated integration scenarios. Companion #17739 contains the saved setup UI (including the required human import/rotation step), product and permission documentation, catalog/Vercel credential and callback ownership, and operation maps to keep each review below 100 files. Both PRs must ship together; merge this implementation first, then its companion. ## Verification - Current implementation head: `2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master `ea727dc3c2`, 99 files. The latest follow-up fixes the two valid findings from the preceding f9 review. Exact-head Greptile has completed at5/5 with no findings and Superagent passed. Full exact-head Linux CI workflow40446606056 completed SUCCESS; only the queued external Paperclip Review remains pending; earlier Linux passes do not qualify this successor. - Latest preserved-stack verification passed 2,165 tests in fresh PostgreSQL: 70 actual-auth scenarios, 27 OpenAPI contracts, 2,022 native API tests and 46 saved GitHub wizard cases. 141 existing gateway/policy acceptance tests also passed. Workspace typecheck/build, maps and token gates passed. The first workspace typecheck was blocked by sandbox tsx IPC permissions; its authorized rerun passed. This is focused local evidence; the full Linux CI matrix is still required. - Issuer/target gateway policy regressions were red before the fix. Both decisions now intersect, including Block/Ask first/rates, same credential identity and current token/user/agent authority after approval. Invocations retain their token and actual requesting agent; deleted tokens fail closed. Thirteen actual-auth cases cover revocation, narrowed scope, approvals, shared rates and board compatibility. Intermediate missing-token attribution, review-context and implicit-session actor failures remain in the qualification record; all final cases pass. - Activity pagination now retains a bounded cursor across private-only windows. The actual-JWT regression reaches three older readable events after 27 private ones without exposing private details. A native device-login regression was red for plaintext receipts; local check and test-call results now use existing protected-response handling. The passing actual-auth case proves encrypted persistence, exact replay, redacted history and no response asset. - Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in the existing connection-intent continuation test: periodic recovery queued a repair at 01:21:47, the scripted journey completed the task at 01:21:48, and the extra queued run was cancelled at 01:21:49 because the task was terminal. The assertion selected that latest cancelled run. GitHub refused the first attempt while the workflow was nonterminal. After completion, the single evidence-supported failed-job rerun was accepted as attempt 2 on the same head (workflow 40431202466). The failed shard and final E2E aggregate passed on attempt 2, and the complete Linux workflow for that head is successful. The original failure remains part of qualification. No additional unchanged-code rerun will be used for this failure. - Companion CI also has an unchanged login-cleanup idleness timeout, retained in its qualification record. That same 16-case suite passed in this parent Linux run and a fresh local database. Both preceding workflows passed after their single failed-job reruns as attempt 2; the original failures remain recorded. No additional unchanged-code rerun was used. - Migration `0380_broken_aqueduct.sql` was regenerated through the repository workflow after master took 0379 for project Git authorization. It adds the responsible-user column with `IF NOT EXISTS`, preserves upstream migrations, and satisfies snapshot retention and migration safety checks. - Upstream published Copilot v62 and v63 identities remain byte-for-byte unchanged. The combined guidance is now v64, digest `sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`. The normal shared TypeScript/Rust identity check and 61 Copilot installation tests passed. The old source-identity assertion first failed against unchanged published v63; regenerating a new v64 preserves the published release. - Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI token gates passed on the final combined stack after the upstream merge. Node 24.22.1 / pnpm 9.15.4 were used. - Earlier 82 API/OpenAPI tests passed in fresh PostgreSQL 18: 55 assembled-server scenarios and 27 contracts. They use real persistent agent keys, run JWTs, human sessions, production defaults, and deterministic provider fixtures. No actor injection establishes access. - The new email regression was red before per-account scope filtering. The Arcade regression was red before human ownership and inventory-key fixes; it also verifies actual-agent audit identity and a human-session inventory read. The activity regression measured 200 authorization checks before the fix and one afterward for the same one-item page. - 82 GitHub UI tests passed, including all three new failed-replacement regressions that were red before their fix. The UI preserves an earlier copy only after confirming its committed generation. Changed or unreadable results block copying and reconnecting until reconciled. Human-only secret responses now include their nonsecret committed generation. - Private test status now rechecks the originating principal and linked task, including mandatory agent/responsible-user privacy. Cross-agent tests require the full caller tool access decision and the same credential identity, regardless of auth label; caller and target policies/approvals and rate limits apply before execution and after human approval. A profile-only precondition was found to reject valid allow-policy and scoped-grant access and has been removed. Both permitted-access cases and a no-auth-label identity regression were red before the correction; caller-specific Block still prevents provider execution. The private-status regression was also red before its fix. - The tool-activity regression measured 100 capability decisions and 102 run captures on a 100-event page before the fix. It now uses two task decisions and at most two captures while retaining the same 50 visible events, private-event filtering, and 403 after audit-grant revocation. The regression was red before request-context reuse and task-decision sharing; final server typecheck/build also pass. - Generic chat configure/reconnect now hands agent-supplied verification credentials to the addressed human before provider work or vault changes. Actual key/JWT regressions were red for Slack signing-key replacement and generic channel secret-reference rebinding. They now prove unchanged refs and vaulted signing value, human setup, permitted agent bot-token repair, and governed test messages. All 82 API/OpenAPI cases pass in fresh PostgreSQL. Eight native receipt protections and final server typecheck/build also pass. - The preceding review found that old healthy state could retire a credential-entry card before a human changed anything. The actual-JWT regression was red. Saved field generations now change only with protected values and commit with refs plus a verification fence; retries retain the baseline. Old health, bot-token repair and unchanged entry stay pending. New signing verification and current authority permit continuation. A separate red race proved that user revocation during provider authentication could return 403 after refs changed; authority is now checked before persistence and inside the credential transaction. The final 82 API/OpenAPI cases, four neighboring real-database Slack/GitHub lifecycle cases, and final server typecheck/build pass. - A preceding full run crossed a rate-window boundary. Database rows prove one consumed slot at 23:35 and another at 23:36 for the same policy/bucket. Only Date is frozen in the rate test cases; transport and asynchronous timers remain real. Production rate behavior is unchanged, and the final full run passes. An intermediate fixture failure that overwrote hidden generation metadata with an API projection is retained in the qualification record. - Linux CI on an earlier head found two rate-admission assertions that omitted the newly persisted bucket key. The failures were reproduced locally; both assertions now verify the exact actor/tool bucket and all 49 policy-service tests pass. The companion qualification record retains the initial and intermediate failures. Catalog creator attribution is preserved in the companion to keep this implementation review at 99 files. - The latest review reproduced three regressions before repair: complete human replacements failed on unavailable old signing secrets, fresh GitHub links incorrectly selected reconnect, and different credential-entry requests shared an idempotency key. Each old protected ref now has a narrow known-unavailable fallback; unexpected resolver failures and company/scope violations still abort. Fresh imports keep their credential form, configured endpoints keep reconnect, and canonical field sets/completion modes distinguish handoffs while matching retries retain their baseline. All 55 actual-auth scenarios and 27 separately run OpenAPI contracts pass, as do server typecheck/build. Companion #17739 also passes all 41 wizard tests including actual fresh-form submission, UI typecheck, maps and token gates. Earlier red results and a mistyped OpenAPI filename are retained in qualification. - The configured GitHub handoff regression was red in the browser-component tests and actual-auth link/persistence tests. Saved links now name requested fields and all/any completion mode; the companion form submits those supplied fields through reconnect and retains other saved credentials. Webhook/client-secret reconnect remains human-only; first-setup secret restrictions remain. All 46 wizard tests pass, including combined modes and legacy links. A duplicate review referenced the older companion bf760912; the current form is already fixed, linked in the reply, and its thread is resolved. - Linux CI on `bf760912` failed automatic Slack provisioning because vault persistence cleared an already verified manifest-update challenge. The same regression was reproduced locally. The provider-owned helper now preserves proof only for the current signing fingerprint and callback URL. Eight existing compatibility cases pass, including this case and initial GitHub secret rejection. The first uncommitted GitHub-field variant also failed that initial boundary and was narrowed to reconnect; both failures remain in qualification. - The upstream merge conflicted only in generated migration metadata. Upstream 0379 is preserved, normal generation/pruning produced 0380, and two repeated applications of the idempotent additive SQL passed. The final combined stack passed 128 tests (55 actual-auth, 27 OpenAPI and 46 wizard cases) in fresh PostgreSQL. Eight Google UI cases also pass; the two protocol cases could not initialize embedded PostgreSQL in this fresh local attempt. Their earlier passing evidence remains historical; fresh Linux CI must execute them. Vercel personal credential attribution is preserved in companion #17739 with its other ownership/consent follow-ups, keeping this PR at 99 files. - The suite covers fresh defaults, restricted users/agents, scopes, personal isolation, token revocation, human handoffs, callback replay, execution approvals, authorized self-targeted policy changes, self-approval denial, and current authority before commit. Reproduce API checks with `pnpm exec vitest run server/src/__tests__/connection-management.integration.test.ts server/src/__tests__/openapi-routes.test.ts`, using a fresh database through `PAPERCLIP_CONNECTION_MANAGEMENT_TEST_DATABASE_URL` or the default embedded test database. - Companion #17739 also records consent separately from token refresh so saved Google choices survive refresh. Its eight service-picker and two local-HTTP OAuth tests pass; both new UI cases were red before the fix. Normal migration generation confirms no SQL change is needed for the additive JSON metadata. - The integration module shares one assembled server, authentication factory, and provider lifecycle to preserve the HTTP-to-vault-to-task-continuation proof. A reused external database caused a Slack fixture identity conflict; the fresh final run passed. Earlier fixture/transport and contract-update failures remain in the companion qualification record. - The broad local `pnpm test:run` attempt remains blocked by macOS embedded PostgreSQL's host semaphore limit. It is not a clean pass. The fresh Linux CI matrix must complete. Provider fixtures do not prove live setup or an empirical LLM success rate. - A current-head completed Paperclip assessment is not yet recorded. Its earlier external recovery boundary has not been bypassed, and no incomplete assessment is reported as passing. The user authorized both merges when checks pass; merge this implementation first, then replay, retarget, and requalify companion #17739 against the squash-merged master. - After the master refresh, local workspace typecheck/build, 97 actual-auth/OpenAPI cases and 141 gateway/policy cases pass. The original fixed database port was occupied; Docker assigned a free localhost port. The first neighbor setup command could not resolve root `tsx`, leaving its schema uninitialized; the built database package migrated the separate fixture and all unchanged cases passed. Both setup failures are retained in qualification. Fresh Linux CI remains required; no full local test-sweep pass is claimed. - The f9 Greptile review scored 2/5 for two reproduced findings: Slack test sends skipped the caller’s tool policy, and healthy App credentials completed an unconfirmed identity handoff. Both are fixed in this head and have resolved evidence threads ([Slack](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927765), [identity](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927895)). Actual key/JWT cases cover twelve Slack caller/target/recipient combinations; the human-session GitHub OAuth callback alone remains pending until explicit confirmation. All 99 API/OpenAPI tests (72 actual-auth + 27 contracts), workspace typecheck and build pass. On the combined companion, all four focused cases (including the twelve Slack combinations), typecheck/build and maps also pass. The continuation suite’s 50 embedded-PG cases were skipped because its database could not start locally; fresh Linux CI must run them. Initial fixture credential-collision and audit-projection assertion failures are retained with the red reproductions in the companion qualification record. No full local sweep or live consent is claimed. ## Risks - The maintainer explicitly confirmed authorized policy administration, including self-targeted changes that loosen controls. These writes require management grants and the responsible user’s current authority. Agents cannot approve their own governed action requests or edit their own principal permission grants. The actual-auth regression exercises both permitted administration and those denials. - This is a broad authorization change. Incorrect scope or owner resolution could deny permitted setup or expose another user's resources. Integration cases cover both permitted and denied requests with actual authentication. - The additive nullable gateway-token column does not infer identities for legacy clients or backfill agent grants. Legacy agent tokens without a responsible user fail closed. - Provider fixtures test supported setup and callback contracts. They do not prove live provider availability. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The runtime does not expose a more specific deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — focused suites above; full local sweep remains blocked as documented - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green — exact `2ecd6451d0` Linux CI completed SUCCESS; external Paperclip Review is still queued - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — exact `2ecd6451d0` review completed; both f9 findings have resolved evidence replies - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 08:20:24 -05:00
DottaandPaperclip	208a093d19	fix(chat): preserve resumed review waits and retry newer saved input (#17725 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/36/changes/paperclip/36 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review agents publish findings and wait for an authorized reply or head change. > - Review tools could retain the requester after resume while completion lost that same wait proof. > - A yielded report could then queue its own continuation or repeat a missing-wait error. > - Ordinary Retry could also select an older failed turn while a newer saved request had no admission. > - This change shares the existing authority proofs and canonical admission owners. > - Current access, publication, execution and budget gates remain required. ## Linked Issues or Issue Description Refs #17318. Builds on the canonical saved-input retry owner in #17733. Related #16550 covers deferred legacy comments after ownership waits. The separate recovery repair in #17724 owns the cleanup-to-saved-input handoff; this PR does not add a second admission owner. A resumed review could publish an assessment, then turn its `response_wake` into an immediate `issue_status_changed` wake. A provider that omitted its continuation could instead repeat a completion error. A newer linked-human message saved during a budget hold could also remain unstarted because Retry selected the older failed source. ## What Changed - Resolve passive review waits through the accepted requester ledger used by GitHub tools. Require the latest reviewed head, exact company, task, agent, repository and processed publication under current access. - Fill omitted/null yielded continuation from that server proof. Permit an inherited normal tree resume only through its accepted parent identity. Permit an ordinary follow-up only after this run has a processed useful reply linked to its exact inbound message. Reject acknowledgements, partial assessments, pending or uncertain writes as that reply proof. - Recheck the same run/review/head/source proof during feedback, finalization and final status commit. Preserve explicit malformed-input rejection and Done policy. Add no monitor permission or publication grant. - Keep heartbeat and publication projections narrow. Preserve both published Copilot v62 identities and generate source-bound v63 from the pinned merged base. No v63 live qualification is claimed. - Certify the structured isolated-directory binding failure only at its pre-allocation boundary and only through canonical task Retry. Other workspace validation reasons and unknown effects remain denied. - Route Retry to eligible newer saved input through the existing canonical owner. Preserve the old result and skipped receipt. Require current principal, source, App, runtime fence, budget, native ownership and exact cleanup proof. - Recover absent historical App metadata only through an earlier trusted processed response receipt and an immutable non-null endpoint claim. Explicit malformed or conflicting metadata stays denied. - Revalidate execution through the exact claimed wake, coalesced task Retry and resolved user-continuation receipt after admission. A second normal Retry can recover the exact never-started authorization failure with unchanged saved input. Require empty execution evidence, exact diagnostics and no allocated or active executor. - Explain when a recorded result prevents exact replay. Ordinary board Retry directs a new task message to continue from saved work. Inherited requester identity alone never grants replay. - Keep accepted board-message preparation failures with the canonical task Retry owner. Certify the exact unchanged comment, original user wake, accepted same-user identity and absence of provider execution. Pin that source, recheck it at admission, and bind identity initialization to the coalesced intent and exact claimed successor receipt. Uncertifiable sources retain an actionable fresh-message refusal. - Return safe named authority categories for board Retry refusals. Update execution documentation and capability maps. Add no schema or polling loop. ## Verification - A fresh external review request exposed a separate isolated-directory preparation failure before allocation. Its first normal Retry saved an intent but admitted no successor. This follow-up certifies only that exact structured preparation boundary through the existing canonical Retry owner, with trusted failed settlement and no executor, lease, result or output. A real HTTP/queue regression reproduces the missing successor before the repair. Twenty-three current-source safety cases and the final 98-case Retry/HTTP group pass, including a fresh click after a denied intent, source revocation, accepted results and the older direct API remaining denied. The warning case uses actual configuration preparation and production event append; unknown warning shape or provider provenance stays denied. Full workspace typecheck, build, maps and diff checks pass. Renewed hosted normal Retry now succeeds on the final combined source: it preserves the exact saved request and linked requester, starts one native successor in a fresh attempt workspace, publishes the current-head 2/5 assessment and commits In review with passive response wake. - A later hosted normal Retry published a low-score assessment but lost its accepted source identity because the previous attempt failed before identity initialization. It repeated continuation errors, then submitted Done. This is failed acceptance, not a successful qualification. The initializer now retains the exact source only after execution-time failed-chat Retry authorization; generic Retry and Done policy are unchanged. A real queue/initializer regression fails with a null source on the previous code and passes with the newer saved source. Two real database completion cases verify that source yields the authoritative wait and commits In review. Renewed hosted normal Retry now retains the accepted source and exposes the authoritative current-review wait. It completed after two semantic feedback corrections (wrong contract revision, then wrong criterion IDs); the correct contract was present in emitted input. The third finish was accepted and committed In review, with no missing-wait loop or Done escalation. - Repeated board-message preparation failures now walk their canonical intent, failed dispatch and accepted parent identity, rechecking each contained predecessor. A production queue regression fails the second ordinary HTTP Retry before the repair and passes afterward. Changed source, contradictory execution evidence and cyclic receipts remain denied; traversal stops after twenty links. - New inherited omission cases failed before the repair. The final full PostgreSQL 18 wait/status suite passes 252 tests under Node 24.22. It covers omitted/null input, malformed-first recovery, same-head resume, exact-source replies, changed heads, revoked access, pending/uncertain/mixed publications, final commit revalidation, no duplicate review and no self-wake. - 15 existing publication tests passed before the pinned merge on a separate fresh database. They retain default partial-delivery and failure-fallback behavior. Context reads remain below the unchanged 16 KiB heartbeat and publication response limits with large saved output present. - Saved-input tests cover a real signed PR webhook, production accounting-budget refusal, normal policy recovery and canonical Retry. Historical producer-shaped tests prove the missing-App case red, then check trusted prior receipts and malformed, late, revoked or mismatched proof. A broader order-coupled test was reproduced on the prior source; its global fault injector now targets its own authorization query. The 43-case sequence passes without changing production behavior or its safety assertions. - New signed-ingress and real heartbeat execution tests reproduced the admitted-successor failure, then the next normal Retry failure. The final pinned merged source passes 249 full GitHub review/publication and Retry cases on a fresh database, including all 49 saved-input cases. A separate 74-case Retry/HTTP run also passes. Exact dispatch negatives cover source, company, agent, task, successor, receipt, pause, assignment, accepted success, controller and cleanup proof. Bootstrap contradictions remain denied. The canonical Retry suite passes 16 cases through its supported external PostgreSQL test connection. - The normal board HTTP route reproduced the unusable inherited-result refusal. It now returns actionable guidance without starting a successor. The actual UI Retry hook and banner tests pass 12 cases. - A production-shaped accepted board comment followed by a pre-provider qualification failure reproduced HTTP 409 before the repair. The final real HTTP/queue test certifies one canonical successor and preserves the accepted identity. It rejects changed source, actor, task, agent, result, controller, process, profile, allocation and receipt evidence. A post-acceptance source change keeps the same intent pending. Execution revalidation rejects removed or mismatched proof. The probe stops deliberately after production identity initialization; it does not claim provider execution. The final source passes the 75-case Retry group, the focused two-case board/inherited-policy group and 19 canonical Retry cases. Full workspace typecheck, build and map checks pass. - Full workspace `pnpm -r typecheck` and `pnpm build` pass. Map, module-boundary and diff checks pass. Full local `pnpm test:run` remains unverified. The 256-case explicit-native suite cannot initialize its packaged PostgreSQL because libzstd is absent; an existing command-scoped fallback did not resolve it. Current-head remote full-suite checks are required. Merged-source full GitHub+Retry (249), separate Retry/HTTP (74), full wait/status (252), profile/history and completion (75), and visible Retry hook/banner (12) checks pass. The fresh owned PostgreSQL 18 databases applied all 377 migrations. - Actual Rust Runner, OpenCode 1.18.34 and exact DeepSeek used unchanged production-generated synthetic contexts. Resume finished with an explicit server wait. The author-reply trial omitted continuation twice: its first report failed the existing empty-criteria check; the corrected second report still omitted continuation and received the exact server wait. It completed once with no missing-wait loop or Done escalation. Controller acknowledgments were synthetic; these probes do not establish hosted publication or board state. - Hosted combined testing proved automatic admission of the exact saved author reply, a useful GitHub answer and committed passive wait. It required two malformed-continuation rejections before a nullish fallback succeeded. A later normal Pause/Resume completed in one finish with explicit response_wake and no new review/comment. The admitted normal Retry then exposed the execution guard gap repaired here. Those remain checkpoint results. On the final combined source, renewed normal Retry completed actual native provider work and GitHub publication, committed the passive wait, released ownership and cleaned up its environment. A subsequent ordinary unmentioned author reply woke once, made no GitHub tool calls or new assessment, and completed its first finish with an explicit response wake. Both exact leases expired with successful cleanup and the queue was empty. However, final destination inspection found a separate automatic fallback overwrote the existing summary with a generic no-final-response message. External summary preservation therefore remains failed acceptance; the separate publication repair in #17726 owns that producer. Full combined publication preservation has not passed. These hosted runs include the pinned-base native additions; they do not qualify Copilot v63. - Current-head automated review is 5/5 with zero unresolved threads. All current-head CI gates passed (54 successful checks/contexts, four skipped, none pending). The earlier login-cleanup idle-work assertion also reproduced locally while its controlled baseline passed; its cause remains unknown, but the exact assertion passed in the fresh current-head CI shard. No timeout, assertion or production workaround was added. This PR is ready for review within its source, authority, native finalization, state, wake and cleanup scope. The independent automatic fallback publication regression is tracked by #17726; no complete combined workflow pass is claimed. ## Risks - The latest saved review defines the durable head boundary. Ingress must record a new head before admission. A reply receipt proves delivery only for its exact saved source; a shared GitHub comment ID alone cannot authorize this wait. - Historical App recovery cannot infer identity without direct proof or an earlier valid server response receipt. It never replays that response. - This does not infer review-score policy from instructions or override Done. No deployment, merge or repair of historical tasks is claimed by the PR. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, repository inspection, code editing and test execution. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused database and native suites; full local suite limitation described above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (verified on immutable latest head) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (verified on the latest head) - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 22:22:47 -05:00
DottaandPaperclip	f0877a9c7c	fix: recover native work and requalify saved computers (#17724 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/32/changes/paperclip/32 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and preserves their work after an interruption. > - Native Stop must confirm termination and save the workspace. > - Task and run views must show the current recovery receipt. > - A new external reply can arrive while that cleanup is pending. > - The reply must return to its original chat owner after cleanup. > - A successful fresh retry also becomes the task’s bound workspace for its next ordinary turn. > - Managed computer updates can invalidate a saved snapshot qualification. > - This pull request fixes those instructions, acknowledgements, handoffs and the existing native-agent Test path. > - The operator can continue with new input while the stopped turn remains preserved. ## Linked Issues or Issue Description What happened? Stopped native runs could keep the initial workspace-repair instruction after files were recovered. Native Stop could also leave the shared cancellation state at requested after remote termination was confirmed. A new GitHub reply saved during cleanup could then remain queued because cleanup did not ask its original inbound owner to reconsider the receipt. After a managed snapshot update, an existing native agent could also need qualification while the generic environment Test required instance-admin access. Expected behavior Show the current saved-work state. Acknowledge Stop only after exact termination proof. Let an authorized operator use the existing saved-agent Run test to qualify its current computer. After cleanup, recheck the saved reply's current authority and admit it once as new input. Preserve the stopped run and uncertain external effects. Steps to reproduce 1. Stop an active native turn before it returns a semantic result. 2. Send a new authorized external reply while cleanup is pending. 3. Let workspace recovery and provider cleanup finish. Inspect the run instruction, Stop acknowledgement and queued reply. Related work: #17594 preserves native processes through controller outages. #17401 addresses a separate bootstrap failure. This pull request does not replace those fixes. ## What Changed - Use the exact run's workspace recovery receipt for task and run guidance, including the Activity ledger. - Retire an obsolete exact-session notice through the existing bounded reconciler. Preserve the separate no-replay hold. - Write the shared native Stop acknowledgement only after the audited Stop intent and every exact remote lease receipt match. - Delegate saved external input to the live inbound owner. Recheck its source, actor, endpoint, principal and conversation generation. Promote the same receipt in the transaction that creates its new run. - Use that owner from the cleanup callback and existing stranded-queue catch-up after a lost callback or restart. A denied source cannot abort other items. - Reuse the current private task directory after a successful fresh retry. Re-read the company/task/workspace binding and validate the canonical stable or attempt namespace without recreating missing files; keep a later fresh retry separate. - Extend the existing agent-scoped qualification owner to native agents. The clean saved-agent Test qualifies its exact computer before runtime testing. Reuse a current proof; require a real probe and verified cleanup for a stale proof. Preserve draft tests and metadata-only external/pool previews. - Update execution semantics, capability maps and the recovery verification recipe. Add a production-page Storybook interaction for the normal native Test. ## Verification - Red: persisted recovery states returned stale instructions; an exact terminated native lease left shared Stop requested; the real GitHub subscribed-message fixture left zero successors after cleanup. The controlled baseline restored the previous production source, reproduced the failure, then restored the fixed bytes with matching SHA-256 hashes. - Green: 382 durable-chat, native-continuation, Stop and recovery tests pass. The real GitHub/PostgreSQL handoff matrix passes 12/12. It covers callback, application restart, callback/sweep race, valid input behind a deleted source, and denial for unknown stop, wrong allocation, paused endpoint, revoked principal, deleted or changed source, changed conversation generation and foreign scope. Repeated callbacks preserve one receipt, its actor, one successor and the unchanged stopped predecessor. - Workspace continuation red/green: the real PostgreSQL preparation flow created a fresh attempt and saved a marker, then rejected the next ordinary turn with the same binding mismatch observed in deployed use. The repaired preparation and filesystem suites pass 86/86: stable and attempt-root reuse, saved-marker preservation, a later clean sibling, changed current bindings, foreign company/task, closed workspace, traversal, missing path and symlink refusal. No task row reset or old-turn replay is used. - Qualification red/green: a native saved-agent qualification returned 422, and its normal Test skipped qualification. The repaired path passes 224 qualification/immutable-snapshot/environment tests and 176 UI/upgrade tests. It covers revoked authority, foreign or changed computer, current-proof reuse, failed/missing proof, draft settings, settings changed during the probe, and external/pool previews. On exact current source, the real local Harness / Runtime UI also qualified a newly saved disposable Computer with missing proof through actual Daytona, then completed the actual OpenCode/OpenRouter hello probe using `openrouter/deepseek/deepseek-v4.1-flash-1017`. Supported availability changed from Legacy-only to Paperclip Runner after proof persistence. Qualification and runtime leases both completed normal release/cleanup. The five prior task run IDs were unchanged; no task wake was created. This is macOS source-server/UI with real Linux Daytona and provider services, not a hosted UI claim. Test probes are archived by the existing inspection contract; release does not claim physical deletion. - Full workspace `pnpm -r typecheck`, `pnpm build`, token, map and diff checks pass at `f6450f4739fc8141815b8b7c35fde22a655c5f90`. The broad local `pnpm test:run` passed 24,899 tests and reported one existing nested-clone test timeout at 15 seconds. The complete failing file then passed 21/21 with the normal timeout; the exact case also passed on the pinned earlier baseline. The broad runner stopped before its later groups. No timeout or production workaround was added. Prior head `f6450f4` has 53 successful checks, two conditional Storybook skips and Greptile 5/5 with no inline threads. New workspace-reuse head `fcb13399ec70f92f71bd8250081251dca3f48962` passes workspace typecheck, full build, 86 scoped tests, maps and diff checks; Current-head CI has 53 successful checks and two conditional Storybook skips; Greptile is 5/5 with no inline findings. Deployed ordinary-turn reuse acceptance passed as described below. - Hosted Stop proof: the real GitHub issue-mention path used native OpenCode, `openrouter/deepseek/deepseek-v4.1-flash-1017`, Daytona and an exact low-trust task boundary. Normal UI Stop interrupted an active Bash call. Native and shared acknowledgements matched the exact lease. Automatic recovery saved exactly `STARTED\n` (8 bytes), with no finish marker. Reload showed Stopped and saved work recovered. Only the original cancelled run remained. Normal five-minute cleanup completed its exact provider-destruction transition. - Limits: the provider buffered command output, so the marker was read after recovery. The approved local provider credential could not inspect the hosted sandbox; its 404 is not deletion proof. Cleanup success comes from the exact lease's platform destruction transition. An earlier hosted stale instruction disappeared after rollout with unchanged raw run/action data; that historical cause remains unexplained. Original incident holds and publication receipts were untouched. - Hosted handoff proof: after QA deployed the combined candidate containing `46b2e99`, startup admitted the previously saved GitHub author reply without another comment, Retry or manual action. The new native run used the exact original wake receipt and saved comment. It completed once, committed an external-response wait, preserved the stopped predecessor, emptied the queue and left the task In review. The answer appeared in the original GitHub conversation. Workspace finalization and the exact lease's normal destruction transition succeeded. - Hosted workspace proof: on the combined candidate containing `fcb13399`, normal Retry preserved the original authenticated GitHub request and completed a fresh native attempt. One later ordinary same-head author reply then completed in the exact same workspace ID and attempt directory, with no Retry/fresh directive. Its input tree digest matched the Retry's finalized digest; final copyback preserved that digest. The persisted assessment row and all five prior run records were unchanged, exactly one new reply run completed, and the task returned to In review with an empty queue and no live run. Both exact leases completed normal verified destruction. This used real native OpenCode, the exact DeepSeek/OpenRouter model and Daytona, without workspace-row repair or a new remote marker. - Publication limit: the workspace audit compares the persisted assessment and run history. The ordinary reply exposed a separate destination-summary fallback that changed the visible GitHub summary despite the preserved assessment row. The publication track is repairing that behavior; this proof does not claim the visible summary was preserved. - Deployment boundary: current-head CI/review and hosted same-source Retry/ordinary-workspace reuse acceptance passed. The separate visible-summary publication failure remains owned by the publication track. Combined hosted UI confirmation of the native-agent Test path is coordinated separately. Actual missing-proof UI/provider qualification has passed in the isolated local instance above; prior hosted Stop and same-input handoff proof remains separate. No merge or deployment is performed by this pull request. ## Risks Recovery state and termination receipts must match the exact source run and allocation. Missing or foreign proof keeps the hold. The chat owner must retain the original receipt and actor and recheck current authority. Cleanup cannot grant authority or replay the stopped turn. Existing native ownership, Stop, budget, permission and uncertain-effect guards remain in place. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through Codex, with reasoning, repository tools, source editing and test execution. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected suites and isolated failing file pass; broad timeout retained above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (current fcb has 53 successful checks/two conditional Storybook skips) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 22:21:58 -05:00
DottaandPaperclip	338bf75761	fix(chat): admit task Retry after GitHub reconnect (#17733 ) ## Thinking Path > - Paperclip manages agents and their tasks. > - Task Retry must preserve saved input and enforce current access. > - A GitHub reconnect invalidates old runtime callbacks. > - Retry reused that old callback authority and rejected valid new intent. > - Retry also replayed an older failed request when a follow-up was already saved. > - This change admits a deliberate Retry under current authority and transfers saved input once. ## Linked Issues or Issue Description Refs #16965, #17053 and #17317. These repairs preserve work and contain old executors. This change fixes the subsequent chat admission boundary. A search found no overlapping open reconnect/admission fix. What happened? Ordinary task Retry returned `chat_failed_run_retry_not_authorized` after the same GitHub App reconnected. The endpoint was healthy, but the saved request still carried its old runtime generation. A newer queued follow-up also made Retry reject the older failed request instead of continuing the follow-up. No successor was admitted. HTTP qualification found two related defects. Concurrent clicks could omit the canonical chat action from a response. Signed-in board dispatch also passed an incomplete task resource to the permission resolver, which rejected a permitted mutation. Expected behavior A deliberate Retry checks current authority for the same App, sender, repository, conversation and task. It continues saved input once. A later reconnect, changed App, revoked access or changed source prevents dispatch. Old runtime callbacks remain invalid. Steps to reproduce 1. Admit a GitHub request and record a failed native run. 2. Save a follow-up while execution is blocked. 3. Reconnect the same GitHub App. 4. Click ordinary task Retry. 5. Before this change, admission returns 409. Without reconnect, the newer input alone also causes 409. Paperclip version or commit Baseline source: `41c42895fa862c586ff841f8c50200d93ffc83a4`. Deployment mode Source checkout on macOS. Regressions use real PostgreSQL transactions and the assembled chat, HTTP route and heartbeat admission services. Provider boundaries are simulated. No hosted recovery is claimed. ## What Changed - Record the admitting App identity for new GitHub deliveries. Historical deliveries can use their original working-comment binding. Reauthorize deliberate Retry under a pinned current generation and credential-reference fingerprint. - Select the complete saved-input queue from the failed run's actual conversation before replaying the older failed batch. Keep exact-run Retry bounded while ordinary task Retry supports larger queues. Transfer its receipts with the dispatch transaction. Recover the same ownership after restart or a lost response. Release unadmitted input after a definitive rejection. - Preserve canonical chat action IDs across concurrent clicks. Supply current task status and assignment fields to board mutation checks. - Keep exact-run Retry, stale webhook, containment, source integrity, publication, pause and budget checks. Update recovery contracts and verification recipes. ## Verification - Red: all three new PostgreSQL regressions failed on unchanged production code. Reconnect failed at chat authorization. Newer input failed at the obsolete-source guard. The combined case also failed. - Green: 61 selected chat integration cases passed. These include 27 new cases and 34 existing cases for containment, retained-source recovery, exact batches and source revocation. - A further red/green PostgreSQL regression reproduces the 51-message overflow identified during review. All 51 messages reach one successor, including a later Retry of that successor. A separate 52-message consumed-batch regression also passes. - A competing conversation retains its own deferred message. Distinct automatic policy contexts are checked independently; revoking only the later policy blocks the whole unadmitted batch. Batch validation still rejects a later message with a changed App binding or deleted source. A real PostgreSQL query capture reduced the 51-message preparation from 1,103 calls to fewer than 180. - All 16 durable task Retry tests passed. - The real HTTP route returns 202 and admits one queued successor for concurrent deliveries. Fault injection covers rollback during input handoff, response loss after dispatch commit, and permission loss when resuming an existing durable wait. A wait only adopts input after current admission succeeds; a one-shot settlement marker keeps status projection read-only. A new deliberate Retry works after another reconnect or a failed successor. - Current-App, repository, sender, source edit/delete, reassignment, endpoint pause and board-permission controls passed. Historical deliveries remain unchanged. - Final-head CI: 55 checks passed, three expected skips. The unchanged minute-bucket rate-limit assertion passed locally against PostgreSQL and on its one CI rerun. Greptile reviewed the final commit at 5/5 with no unresolved threads. - Workspace typecheck and full build passed. Map alignment, module boundaries, UI token gates and `git diff --check` passed. - Hosted task recovery and a live provider turn remain unverified. The local tests hold agent capacity to inspect admission without starting a model or publishing a GitHub response. ## Risks - A deliberate GitHub Retry can admit old saved input under current connection authority. It requires the same App binding and current access. It pins that admission; it does not rewrite the historical delivery or make stale callbacks valid. - The durable chat queue has an optional admission callback. It transfers input only when the transaction contains a valid queued execution or saved wait. Other chat consumers have no callback and retain their behavior. - No migration, Cloud dependency, deployment, configuration change or workspace cleanup is included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source inspection, code execution and GitHub tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 10:26:45 -05:00
Devin FoleyandPaperclip	7703220d38	perf(github): narrow response authority reads (#17686 ) Read current GitHub response identity and policy fields without transferring saved message bodies, instructions, or automatic policy wrappers. Preserve company predicates, shared locks, publication leases, malformed input handling, and stored content. Verify workspace typecheck, server build, 414 selected GitHub tests, three focused browser cases, all 55 checks, and Apex 5/5. Preserve the original browser cleanup deadlock failure; always restore experiment settings while reporting cancellation errors. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-25 20:52:09 -07:00
Devin FoleyandPaperclip	39e5c8c1c3	perf(server): narrow committed chat authorization reads (#17530 ) Select current chat authorization fields for committed responses under the existing scoped share lock. Preserve complete evidence for actual retries, source revocation, JSON types, and saved run data. Full build, typecheck, 113 focused tests, complete CI, and exact-head Greptile5/5 pass. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-24 19:01:19 -07:00
DottaandPaperclip	7ae81c9fa7	fix(slack): support rich questions and complete review delivery (#17255 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Questions and review decisions must work in the conversation where a task started. > - Slack omitted question context and did not publish several review kinds. > - Some answers saved in Paperclip also lost their authorized continuation or their final Slack reply. > - A settled card and a successful run did not prove that the user received the result. > - This pull request preserves complete questions and follows the existing answer and delivery receipts through continuation. > - Users can read the request, submit an authorized decision, and receive the resumed task's result in the same conversation. ## Linked Issues or Issue Description Refs #17045 and #17218. What happened? Slack question cards could omit instructions and editable starting text. Checkbox reviews, item verdicts, task suggestions, and connection requests had no publication path. Complex question answers submitted in Paperclip could fail continuation authorization. Several review continuations completed without delivering their result to Slack. Expected behavior Show the actual question and context. Use native Slack controls when they preserve the form contract. Otherwise provide an explicit Paperclip review link. Persist one authorized response, resume through the existing delivery path, settle the card, and deliver the useful result to its originating conversation. Steps to reproduce 1. Start a task in Slack and ask a text or choice question with instructions and help text. 2. Submit native text and choice responses, then submit a complex form through Paperclip. 3. Accept a checklist, submit item decisions, select suggested tasks, or decline a connection request. 4. Inspect the saved answer, continuation run, original Slack card, and final Slack reply independently. ## What Changed - Preserve full prompts, context, descriptions, recommendations, help, and editable starting text. Use direct buttons for short choices, inline searchable selects for longer lists, and native checkboxes with explicit submission for multiple choices. - Add native custom-answer and validated-text modals, ordinary confirmation buttons, checklist decisions, per-item verdicts with reasons, suggested-task creation/dismissal, and connection decline. Keep governed approvals, permission grants, and forms that cannot be represented faithfully in the full Paperclip review surface. - Keep Slack-specific canonical-validation feedback in its owning platform module. Shared callbacks delegate without interpreting provider presentation. - Reuse the existing canonical interaction transactions, callback receipts, publication queue, and continuation delivery. Share ordinary confirmation handling across Telegram and Slack. Recheck current authorization and commit response receipts atomically. - Settle cards with the saved answer and remove stale controls. Ignore passive Slack link callbacks so opening a task does not display a false answer error. - Validate exact multi-select receipt mappings during native continuation. Recover automation-origin connection replies from their published prompt after validating the durable connection receipt. - Keep oversized multi-select handoffs free of unused native tokens, and retain the complete Paperclip form for 101–128 choices plus Other. Make the original-conversation audience explicit before submitting a Slack modal. - Add rollback, provider serialization, callback, continuation, and negative authorization regressions. Update the connector playbook and feature/access maps. No React or application UI changes are included. - Add a durable Q01–Q11 qualification index and provider coverage matrix. Require distinct functional, good-experience, and best-available-control assessments; real unanswered/answering/settled evidence; saved results, continuation and navigation proof; and retained failures. Mark other providers unaudited rather than assuming Slack parity. ## Verification Candidate: `e0017d446b1016a38d65382366cd1408cce33039`. The eleven-core-format live Slack gallery is pinned to `c3d458b2a06b660448940fd8cf33c5906f0058d5`; subsequent boundary fixes and the sharing notice have automated verification, not new live screenshots. - Real Slack UI testing covered all 11 formats: text, closed choice, long single select, multiple choices, choices with a custom answer, pattern-validated text, confirmation, checklist, item verdicts, suggested tasks, and connection requests. Each has unanswered/answering/answered screenshots as applicable, an inspected saved result, one continuation, and an individual experience review in the local HTML gallery. - The dedicated private QA app/channel used signed callbacks to the local candidate. A separately approved password-protected, read-only preview proved connection-review, suggested-task review, and created-task navigation. Anonymous access, writes, and credential routes were denied. No OAuth grant was attempted. - Real testing exposed and repaired a multi-select continuation failure and a misleading error on link-button clicks. Failed attempts remain in the evidence. The final navigation replay opened the exact pending connection request and declined it in Slack without granting access. Suggested-task creation produced the selected child with the correct parent and description. - Final review-fix verification: 32 form tests, 28 provider/form-rendering tests, 30 publication tests, and 171 native continuation/authorization tests passed. Earlier 3 atomic-resolution rollback, 9 targeted callback/publication/Telegram, and 28 review-continuation cases passed. The final Slack/Teams modal subset also passed. The automation-origin connection cases, 101/128 custom-choice cases, and oversized multi-select continuation reproduced before their fixes and passed afterward; failing logs remain retained. These counts describe relevant runs, not a reliability percentage. - CI on `b5885b85fd` exposed a test fixture that assumed every Slack modal block was an input; the new sharing notice correctly renders a non-input block. The fixture now selects input blocks safely. The exact signed-modal correction/one-continuation test passed locally, together with 14 Slack form/modal tests. The failing CI log and its specific cause remain recorded. - `pnpm -r typecheck` and `pnpm build` passed. Server typechecking was repeated after the final source changes. Map and module-boundary checks and `git diff --check` passed. - The full `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL semaphore exhaustion on this host. Its 1,395-test chat suite found two stale native-control assertions; both were corrected and re-tested in a focused run. Focused database suites used separate disposable databases on existing local PostgreSQL. The full suite is not claimed green; new-head CI must complete before merge. - Final-head CI passed on `e0017d446b1016a38d65382366cd1408cce33039`: all test/build jobs and required `ci / verify` and `ci / e2e` gates are green ([run](https://github.com/paperclipai/paperclip/actions/runs/40115430194)). Greptile and Ponytail both reviewed this exact head at 5/5, the security scan passed, and all review threads are resolved. - All three QA app callbacks were restored to staging. Both temporary tunnels and local proxies were closed. The local endpoint was archived and its local credential copies removed. The candidate app was stopped. Limits: provider execution used a deterministic Claude protocol fixture, not an LLM. This is local real-provider UX proof, not deployed staging or broader Codie/Claude qualification. The public preview intentionally blocked ancillary artifact routes, so its destination pages could show a partial-history warning. The complete permission-grant flow and every oversized/mixed-form branch were not driven manually. Earlier fixture, test, and provider failures remain recorded. ## Risks - New reply routes require durable origin proof. Incorrect receipt matching could lose a reply or route it incorrectly. Tests cover missing, forged, mismatched, and revoked authority. - Governed forms keep their Paperclip resolution checks. This change does not grant new approval authority to Slack. - Existing completed review results can become deliverable through the repaired publication path without another provider run. - No schema, lockfile, workflow, public task-status model, or runtime-instruction change is included. This branch is not deployed. ## Model Used OpenAI Codex, GPT-6 family, with code execution and tool use. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the new head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the new head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 10:29:15 -07:00
DottaandPaperclip	3f1ad6b25b	fix(chat): resume task Retry after retained-source containment (#17317 ) ## Thinking Path > - Paperclip manages agent work through task and execution ownership. > - A task Retry must retain the user's request while the system verifies an old executor. > - Connected conversations also retain their external principal and accepted source batch. > - Chat Retry admission required containment before it could save a wait for that recovery. > - A saved chat receipt could also remain deferred after containment became available. > - This pull request lets the exact recovery-bound request wait and resume through its chat owner. > - Normal admission still checks containment, current access and policy before one successor starts. ## Linked Issues or Issue Description What happened? An ordinary task Retry on a connected conversation failed with `failed_chat_retry_not_authorized` while its retained remote executor still needed verification. The new recovery generation existed, but the chat action failed before it could save a dispatch wait. This occurred after the repairs in #17267 were confirmed live. The local regression found a second gap. A processed chat action with a deferred wake was not reconsidered after containment. Re-entering normal admission also tried to insert its existing durable receipt again. A historical hold for the same failed run could also hide the current Retry's workspace strategy, so the request still waited after the old executor was contained. Preparation could then read that sibling instead of the recovery record that owned the fresh successor, losing the fresh-workspace decision. Expected behavior Save the exact authorized Retry while its matching recovery generation verifies the executor. Once containment is recorded, reconsider the same chat receipt through normal admission. Start one successor with the original source batch. Keep publication, principal, pause and budget checks. Steps to reproduce 1. Admit a GitHub or Telegram request and record a failed native run with a retained remote lease. 2. Leave the executor's termination unconfirmed and exhaust its prior recovery generation. 3. Use ordinary task Retry to create a new recovery generation. 4. Process the connected Retry. The old admission code fails instead of retaining a wait. 5. With a deferred receipt, record exact containment and reconcile again. The old dispatch path leaves it stranded. Paperclip version or commit Reproduced from `412914cfa78d5abefcc34d3eb8d4bb86e461caac`. The branch is rebased onto current master. Deployment mode Hosted tenant for the reported failure. Isolated local PostgreSQL and the assembled heartbeat/chat services for the regression. Refs #17267 and #16965. This change does not add a Cloud dependency or change the absence protocol. ## What Changed - Permit a saved admission wait only for the exact current task Retry and active source recovery generation. - Keep promotion and execution dependent on independent containment proof. - Bind a Retry to active automatic recovery without replacing its generation, attempt count or deadline. - Share the current exact Retry strategy across historical holds for the same run. Retain each hold's stop and effect checks. - Select the canonical recovery record that proves the committed fresh successor during workspace preparation. - Let task receipt reconciliation return the same deferred action to its chat owner. - Recheck current board and external authority, then update the existing wake atomically with its successor. - Preserve the exact chat source batch. Keep generic board-comment filtering unchanged. - Add GitHub and Telegram cases for waiting, containment, one successor, stale generations, source edits, endpoint pause and supersession. - Update recovery documentation and capability maps. ## Verification - Red: real PostgreSQL tests reproduced the admission rejection for GitHub and Telegram. Six active-recovery cases then failed on the generation-ID assumption. Two same-run historical-hold cases remained deferred after containment on the old strategy lookup. A preparation regression then returned no readiness when the historical hold was stored first. - Green: 73 selected chat integration tests and 394 continuation, task Retry, workspace recovery, recovery-state and workspace-preparation tests passed. The chat tests use the real chat admission and heartbeat queue. Unrelated capacity stays occupied so no real provider starts. They assert no executor before containment, one successor afterward, unchanged source leases while waiting, preserved recovery budgets, and denial after authority changes. A separate source hold still blocks work. Both sibling insertion orders pass through the committed-readiness lookup and real workspace preparation: the new directory has independent Git storage and files, and unpublished old work remains intact. - `pnpm -r typecheck`, `node scripts/check-maps.mjs` and `git diff --check` passed after the review fixes. The source build passed on `f6660efa48e37a80cd9857217a537606de85b5c8`. - The local host reached its PostgreSQL semaphore limit. Successful runs used a test-only loader to reduce disposable cluster connection and worker limits. No repository database helper or application setting changed. - The full local `pnpm test:run` began before the final review edits, failed in its first server phase and was stopped after those failures. Later phases did not run. This is not a full local pass. The transient Retry-notice case passes after fixture isolation; the four Discord close-owned publication cases also pass in isolation. Other broad-run failure causes are not established. - Exact-head CI and security checks pass on `f6660efa48e37a80cd9857217a537606de85b5c8`; Greptile is 5/5 with zero unresolved review threads. Optional Paperclip Review completed neutral with a 5/5 static assessment of all 11 changed files and no findings; it did not run tests or hosted acceptance. All 57 current-head checks are terminal: 54 success, two skipped and the optional neutral review, with no failures or pending checks. - The real hosted workflow confirmed that the deployed absence repair recorded containment. A later ordinary UI Retry was accepted and saved a durable request, but it remains behind the historical source hold. The useful existing review remains unchanged. This follow-up is not deployed; a live successful successor and final task disposition remain unverified. No missing files are claimed recovered. ## Risks - The existing task reconciliation sweep can restage a deferred chat action. This reuses its receipt and generation; it does not grant new execution or publication authority. - Dispatch must keep the board Retry identity separate from the connected request's principal. The regression checks these bindings and preserves current chat authorization at each phase. - Historical holds reuse a strategy only from the locked same-run recovery that names the current Retry intent. A different run, malformed recovery record or independent source hold remains fenced. - Unknown executors without a matching active recovery still fail admission. Promotion and execution require containment even when admission saved a wait. - No schema or API shape changes. No credential changes, manual deployment or service restart. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution and browser verification. The exact served model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:51:34 -07:00
DottaandPaperclip	4173059081	fix(inbox): keep advisory GitHub review waits quiet (#17318 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Mine shows a person's participation and requests that need their response. > - Automatic GitHub reviews should remain quiet until the assigned work needs human input. > - Review bots used completion requests to keep tasks in review while CI or a PR was still open. > - Those requests correctly entered Mine, despite the automatic-work filter from #16356. > - This pull request gives published reviews an existing provider-event wait in the in-review state and withdraws superseded completion requests. > - Real input requests and separate approval gates keep their current behavior. ## Linked Issues or Issue Description Refs #16356 Refs #13314 An advisory score below 5/5, pending or failed CI, or an unmerged PR does not by itself require a person's approval in Paperclip. The bot can track those conditions without adding a completion card to Mine. ## What Changed - Explain the quiet review lifecycle in the GitHub review instructions. Use a yielded provider-event wait, or an existing saved monitor, after publication. Report done when the configured completion conditions are met. - Keep a confirmed, published review in `in_review` during that wait. Reuse existing authorization and monitor delivery. Add no schema, timer, or polling loop. - Recognize the committed external wait in review attention and recovery. Shared execution receives a verified publication flag; GitHub receipt lookup stays with the GitHub module. Other chat waits keep their current status. - Withdraw an older system-created completion request only when the same bot has published both reviews and its new report has no attention request. Keep explicit cards, separate approval gates, and requests without the prior review evidence. Finish validation stays read-only; withdrawal commits atomically with the new status after checking current governance, the admitted result, and both low-risk contracts. - Update the product rule, capability maps, and regression coverage. ## Verification - Final commit `7c502118eec36028d1b4537bb8797ef893940bf0`: 54 GitHub checks passed, two intentionally skipped, none pending or failed. Greptile is 5/5, the security scan passed, and all review threads are resolved. - Passed: `pnpm -r typecheck` and `pnpm build`. Rechecked server typecheck and build after the final authorization guard. - Passed: 195 tests across `external-chat-wait.integration.test.ts`, `status-arbiter.test.ts`, `review-instructions.test.ts`, and `issue-liveness.test.ts`. The database suite used an isolated PostgreSQL 18 container through `PAPERCLIP_TEST_DATABASE_URL`. - Acceptance coverage: 3/5 stays in review outside Mine; 5/5 with pending completion conditions stays in review; 5/5 with satisfied conditions finishes quietly; a real repository-access request enters Mine until resolved. Also covers obsolete-card withdrawal, transaction rollback, rejected reports, separate approval gates, multiple reviewed heads, saved monitors, and revoked endpoints, repositories, or membership. - Passed: `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check`. - The initial acceptance run exposed a missing authenticated user in the test fixture. Added the user required by the existing addressee authorization check. All 154 tests in that database suite then passed, including the final rerun. - Review found premature withdrawal during validation, stale repository access, and a missed receipt when a run reviewed multiple heads. The revision fixes all three and includes acceptance regressions. The saved-monitor commit guard now accepts the verified published-review state. - `pnpm test:run` was attempted but could not complete on this Mac. Embedded PostgreSQL failed to allocate semaphores (`semget: No space left on device`); a direct initialization probe confirmed the host resource limit. The isolated database run proves the affected suite, not the full repository suite. - These tests exercise persisted review receipts, finalization, and Mine queries. They do not prove a live model followed the new instructions or submit a live GitHub review. ## Risks - The bot must follow the new lifecycle instructions. A real explicit attention request still enters Mine. - Existing obsolete cards are withdrawn when the same bot publishes a later review and reports no human dependency. This change does not run a bulk cleanup of historical inbox entries. - Review attention now recognizes an external response path. It requires an applied current status decision, a successful bound run, and current conversation, repository, and principal authorization. - No migration or UI change. The full local test suite is unverified because of the host database limit; CI passes on the final commit. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code editing, and test execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (195 focused tests; full-suite limitation above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:14:40 -07:00
Devin Foley devinfoley Claude Opus 5.5	f3af9887ca	fix(chat): keep each chat conversation in its own wake batch (#17054 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat channels let a linked user talk to an agent from Slack, GitHub and other platforms. Each external thread is a chat conversation that is bound to one issue. > - One issue can have many conversations. A GitHub pull request review makes one conversation for each inline finding thread. > - When replies arrive behind an active run, the wake queue merges them into one deferred wake. The merge did not look at the conversation. > - A reviewed chat run can attest only one conversation. A merged batch from two threads always failed attestation with `setup_failed` / `reviewed_chat_execution_binding_not_authorized`, so no reply got an answer. > - This pull request keeps each chat conversation in its own wake batch. The single-conversation attestation check stays the same. > - The benefit is that replies in different review threads each get their own attested run, one after the other. ## Linked Issues or Issue Description Refs #16793 (related: chat origin during coalescing). No public issue exists, so the bug report follows. What happened? A linked user replied in three different inline review-finding threads of one pull request while the agent's review run was still active. The three wakes merged into one deferred wake. The promoted run failed in about 2 seconds with `setup_failed` / `reviewed_chat_execution_binding_not_authorized` and `providerWorkStarted=false`. None of the three replies got an answer. The failed-chat retry cannot recover this batch, because a retry source holds one conversation. Steps to reproduce 1. Bind two chat conversations (two GitHub finding threads) to one `in_review` issue. 2. Start a run for the assigned agent on that issue. 3. As the linked user, reply once in each thread. 4. Finish the active run. 5. See one promoted run that fails with `reviewed_chat_execution_binding_not_authorized`. Expected behavior Each thread gets its own run. Each run attests its one conversation and answers its reply. Cause `findExistingDeferredWake` matched on company, agent, issue and actor only. The active-run coalesce path had the same gap. `authorizeChatConversationForBoundRun` correctly denies a batch with more than one conversation. ## What Changed - `run-dispatch/domain/wake-context.ts`: add the `chatConversationId` wake-context key, a reader, and `wakeChatConversationsConflict`. A conflict means that both wakes have a conversation and the two conversations are different. - `issue-assignment-wakeup.ts` and `chat-channels/inbound/wakeups.ts`: stamp the conversation ID on each inbound chat wake. - `chat-channels/failed-run-retry.ts`: stamp the conversation ID on failed-chat retry wakes, so later replies do not merge into a retry from another conversation. - `wake-queue` admission: do not coalesce into an active run of another conversation. The deferred-wake lookup now matches only wakes of the same conversation or wakes without a conversation. - `heartbeat/queue.ts`: the same-scope queued or running coalesce target is skipped when the conversations conflict. - `doc/execution-semantics.md`: describe per-conversation chat wake batching. - Tests: an integration regression in the GitHub review block of `chat-channels.integration.test.ts`, and unit cases for the active-run branch in `wake-queue/application/use-cases.test.ts`. ## Verification - Red before the fix: `pnpm --dir server exec vitest run src/__tests__/chat-channels.integration.test.ts -t "replies in review finding threads"`. The main case fails with one run: `{ status: "failed", errorCode: "setup_failed", error: "reviewed_chat_execution_binding_not_authorized" }`. - Green after the fix, same command, 3 of 3 pass: - Replies in two finding threads give two successive runs. Each run succeeds, has `paperclipExternalChatExecutionBound: true`, and holds only its own comment and conversation. - Two replies in the same thread still give one run with both comments. - After the linked identity is revoked, each separated run is still denied (`setup_failed`) and the adapter does not execute. - `vitest run src/modules/wake-queue src/modules/run-dispatch`: all pass, with the new cases (different conversation, same conversation, run without a conversation, non-chat wake). - Related suites pass: `heartbeat-comment-wake-batching`, `durable-chat-wakeup`, `chat-failed-run-retry-boundary`, `issue-queued-comments-routes`, `heartbeat-stale-queue-invalidation`. - Full `chat-channels.integration.test.ts`: 823 passed, 1 timed out. The timed-out test (`GitHub terminal receipt composes pinned HTTP and durable source authority (held_token)`) passes alone with and without this change. The sandbox has a 4 GB memory cap. - Not run locally: `pnpm -r typecheck` and `pnpm build`. The server `tsc` process was killed at the 4 GB sandbox memory cap. CI runs both. - Runtime dependency note: the deferred-wake lookup adds one JSONB predicate to the existing deferred-wake query. It reads the same rows. The query count does not change. The integration regression runs it against embedded Postgres. ## Risks - Low risk. No schema change and no migration. - Behavior change: chat replies from different conversations on one issue now run one after the other, not in one batch. Before this change, such a batch always failed, so no working flow becomes slower. - Wakes without a conversation (non-chat wakes, and wakes queued before this deploy) keep the old coalescing. A deferred wake from before the deploy has no conversation key, so one later reply can still merge into it. That window closes when the queue drains. - The check keeps the single-conversation attestation in `authorizeChatConversationForBoundRun`. Revoked identities are still denied. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code through the Paperclip agent harness. It used tool use (shell, file editing, test runs) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-20 22:47:24 -07:00
DottaandPaperclip	d33b40a5c3	fix: recover saved work and provide reliable task Retry (#16965 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description What happened? A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. Expected behavior Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. Steps to reproduce 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 19:58:53 -05:00
DottaandPaperclip	4b05795712	fix(github): reuse review instructions across native turns (#16971 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub events continue an ordinary task and can reuse its provider conversation. > - Each automatic event appended the full review policy and PR description. > - Native GitHub follow-ups also repeated task history because only Slack used the compact path. > - This pull request separates admitted review configuration from commit updates. > - A verified resume sends the current messages and changed configuration. A fresh session gets full context. > - This reduces repeated input while retaining review rules and current tool checks. ## Linked Issues or Issue Description Refs #13959, #16645, #16901. What happened? Opening a PR and pushing twice stored three full review prompts. A later native turn repeated all three in a retained provider conversation. Expected behavior A new event supplies its commit and event data. The harness delivers stable review instructions once per retained conversation and configuration. Fresh sessions receive full instructions. Steps to reproduce 1. Enable automatic GitHub reviews for a repository. 2. Open a PR, let the review finish, and push two new commits. 3. Inspect task messages and the native submitted envelopes. Before this fix, the policy and old requests repeat. ## What Changed - Store compact automatic event messages. Fetch the PR description through the existing metadata tool when needed. - Load review instructions from processed deliveries bound to the company, task, agent, and source comments. Preserve each admitted policy snapshot. - Use the existing verified native continuation path for complete GitHub text batches. Deliver configuration changes using a key over endpoint, PR conversation, revision, and instructions. - Keep full bootstrap input for fresh sessions and failed resume, including board follow-ups with no new GitHub event. Retain existing framing for incomplete batches, attachments, interactions, and recovery. - Keep the same configuration identity across automatic review, discussion, and automatic review. Coalesced requests use the last accepted request's admitted policy. - Add regression and database coverage. Update the execution contract and verification recipe. ## Verification - Red: the new event-size regression failed because policy and PR body repeated. The GitHub continuation regression failed because it returned no compact prompt. - Green: focused policy, continuation, native input, and adapter prompt tests passed (231 tests). Native resume/fresh-bootstrap cases passed (9 tests). - The signed-webhook/database case passed at `6296168253`. It checks configuration edits, queued older snapshots, coalesced policy selection, discussion continuity, board follow-ups, and company/task/agent/comment boundaries. The private PostgreSQL template used 12 connections because the host reached its semaphore limit with the default settings. - The earlier broad channel/session integration run passed 1,432 tests. Later review fixes invalidated that run. The latest focused database case passed after those fixes. - Reviewer regressions failed against the previous lookup, then passed with the fix: a mixed-revision batch delivered multiple policies, and a discussion lost the configuration identity. The board wake-to-rendered-prompt regression also failed before repair. - `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` passed on the final source. - Full local `pnpm test:run` exposed follow-up/fixture failures during development. Those cases are fixed and pass separately. The stale run was stopped after source edits. Full-suite validation for the final commit passed in CI. - All required CI checks passed for `6296168253`, including tests, typechecks, build, and clean public npm install checks. The two unrelated Storybook checks were skipped. - A live two-turn Codex smoke could not start: the local macOS sandbox helper rejected execution of the installed Codex 0.153.4 binary. No live GitHub/provider acceptance or deployment is claimed. ## Risks Skipping instructions is safe only after a verified checkpoint resume with the correct previous run. The full bootstrap remains available if resume fails. Changed or previously unknown configuration is sent again. Tests cover these cases. Tool authorization and publication checks still run under the current server policy. No database migration is required. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment model identifier and context window are not exposed in this session. No delegated agents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 10:39:17 -05:00
DottaandPaperclip	bc091f0a21	fix(recovery): recover retained workspaces before saved messages (#16906 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A stopped run can leave useful files in its original sandbox. > - The server keeps that sandbox when it cannot confirm file copyback. > - The task can then save new user input but offers no safe way to recover the files. > - A retry of the old agent turn can repeat work that already finished. > - This pull request recovers files through the existing workspace owner and delivers only saved human input. > - The user can recover a task without certifying infrastructure facts or losing completed work. ## Linked Issues or Issue Description Refs #16883 and #16835 for publication receipts and verified process termination. Related: #16478 clears stale holds after copyback already succeeded. This change performs the missing export and records its proof. #16901 manages the main GitHub response comment and does not perform workspace repair. What happened? A failed run retained its workspace. A later user message stayed saved behind a file-repair hold. The task asked the user to verify process termination and file recovery but offered no repair control. Sending a message could show an optimistic Working state and move the task to Todo before admission succeeded. Expected behavior The harness should recover the original files when it can prove ownership and termination. It should preserve newer edits and completed operations. Only waiting user input may start another turn. Missing evidence must keep the source held with a specific explanation. Steps to reproduce 1. Interrupt a sandbox run before workspace finalization succeeds. 2. Retain its exact allocation and save a user follow-up. 3. Observe the file-repair hold and missing end-user repair path. ## What Changed - Add source-bound background recovery for native and legacy runs. An accepted native result is not required. - Share the physical copyback lock and durable owner. Resume only the exact stopped allocation, export files, verify persistence and obtain a fresh stop receipt. - Persist retry limits and idempotent requests on the existing recovery record. Limit each server to two background exports. Add an indexed candidate scan and a separate cleanup obligation. - Add a task-editor recovery endpoint. Deny agents and viewers. Reject caller-supplied stop or repair claims. - Deliver waiting human input once through ordinary admission. Preserve pause, approval, budget and assignment gates. Require historical conversational-adapter evidence for legacy-to-Runner continuation. - Include exact GitHub publication receipts in continuation context. Preserve unknown effects and the original failed run. - Show observed recovery progress, a transient-failure retry action and specific service blockers. Keep saved messages visible. Suppress false startup feedback and implicit Todo promotion behind a file hold. Hide unavailable workspace links. - Update execution semantics, capability maps and production-component stories. ## Verification - Red: the original read model failed the new recovery-progress assertion; the other 23 checks in that file passed. - Passed: recovery regressions with 28 cases with real PostgreSQL, files, tar and Git operations. Lifecycle provider calls are controlled test boundaries. Coverage includes native/legacy export, concurrent owners, rollback after export, interrupted archive export with a new database connection, lost replies, historical discovery, durable retry limits, separate cleanup and saved-input delivery. The actual candidate query uses its partial index with 3,000 unrelated history rows. A first automatic repair can bind verified old-controller stop proof without an accepted native result. Review also reproduced and fixed export-only continuation and cleanup lost after a committed repair. Cleanup remains discoverable without waiting input and after task reconciliation. - Passed: two GitHub integration cases verify that continuation context carries the existing publication receipt and does not write another response. Company and task boundaries are checked. - Passed: 293 continuation, workspace-sync, platform-receipt and OpenAPI checks. After file repair, actual dispatch retains pause, cancellation, approval, real budget-policy and assignment gates. - Passed: 174 route checks, including editor admission, agent/viewer denial and rejection of fabricated repair claims. - Passed: 351 task detail, workspace properties and comment-reopen checks after updating the new read boundaries and asynchronous workspace-link expectations. Two new route cases keep held tasks Blocked when a comment is saved. - Passed: 265 task-thread and recovery-notice checks. The broader continuation suite also passed; its new legacy migration case first exposed a gap that the focused recovery suite now covers. - Passed after rebase: `pnpm -r typecheck` and `pnpm build`. The later master merge adds an unrelated permission migration; recovery migration generation and safety checks passed after the merge. - Passed: token gates, module boundaries, map structure and migration checks. The recovery index is migration 0360 after current master. - Browser: inspected the production recovery component and its retry-to-queued transition with fixture data. The first dev dependency load failed; a reload after optimization resolved it. This is UI evidence, not a live provider recovery claim. - Every test shard passed on final commit `bbbc09efb4`. The older local `pnpm test:run` was stopped after the complete CI test matrix passed. Local full typecheck and production build passed. The first CI build compiled and booted, then lost its runner during the clean public npm-install check. An isolated retry passed the build, production boot and clean public npm-install verification. - Greptile reviewed final commit `bbbc09efb4` at 5/5. All review threads are resolved. - The change is not deployed. Hosted recovery remains unverified until the served revision includes this change. ## Risks - Recovery crosses process, provider and persistence boundaries. Missing or mismatched ownership, stop, descriptor or file evidence keeps the hold. It never selects a replacement allocation or silently starts over. - File recovery does not certify external action outcomes or authorize replay. New input still uses normal execution admission. - Historical sources without sufficient durable evidence can remain blocked. Destructive reconstruction stays separate. - The index migration is idempotent. It changes no task data. Snapshot pruning follows repository policy. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with code execution and browser tools. `gpt-6-astra` assisted earlier investigation. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 21:51:46 -05:00
DottaandPaperclip	3b8324a233	fix(github): keep one main PR review comment and reconcile stopped work (#16901 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub connections send review requests to an assigned agent. > - A PR needs one main comment that shows its current review state. > - The response ledger used one comment per incoming request. Pushes and follow-up messages created more placeholders. > - Thread subscriptions also accepted requests addressed to external bots. Recovery could leave an acknowledged request without a final notice. > - This pull request shares the main PR comment, filters other recipients, and reconciles stopped work. ## Linked Issues or Issue Description Related: #16874 showed the repeated placeholders. #15416 introduced the per-request response path. The prior summary path used one PR-level marker. What happened? Repeated pushes created new main PR comments. A subscribed `@greptile review` message also woke the Paperclip bot. If execution stopped before staging a milestone, its comment could keep claiming that work was active. Expected behavior One bot keeps one main comment per PR. Progress and review results update it. A message addressed to another recipient does not wake the bot unless it also explicitly mentions this bot. Stopped work gets an accurate notice. Steps to reproduce 1. Accept two review requests in one PR conversation. 2. Add `@greptile review` to that subscribed conversation. 3. Fail the assigned run before it stages a milestone. Run channel maintenance. ## What Changed - Add a durable main PR response target. Keep per-request receipts and the existing publication lease. - Recover the target from App-owned provider markers. Adopt older placeholders and point verified duplicates to the current review. - Use a queued notice before a PR run starts. Preserve version ordering and final responses. - Filter leading mentions addressed to another recipient. Explicit mentions of this bot take precedence. Incidental mentions keep normal thread behavior. - Reconcile acknowledged requests after terminal execution. Show paused state for saved requests on a blocked task. Preserve newer work and confirmed or uncertain tool delivery. - Keep a request open when its stale acknowledgement is suppressed. Close placeholders after replies published elsewhere. Retire old coalesced placeholders before replacing their receipts. Resolve native request identity when legacy wake fields are absent. - Update guidance, connection documentation, and feature/access maps. ## Verification - Red: all three behavioral regressions failed on the original code. A second review created a second comment; an external-bot request was processed; stopped execution retained its working placeholder. - Green: the same three cases passed after the fixes. - Review regressions also failed before their repairs: a delayed acknowledgement closed a new request; normal and stored formal-review receipts left a live placeholder; coalesced completion lost a duplicate receipt; native terminal work without legacy wake fields stayed queued. All 13 combined regression and safety cases passed after repair. - The same legacy receipt-loss bug was also reproduced in the superseded-acknowledgement branch. Its four affected cases passed after adding retirement before that receipt replacement. - A further review regression reproduced paused requests rejecting resumed progress and skipping stopped-run cleanup. Four cases failed before repair; all eight resume and neighboring safety cases passed afterward. Both paused-notice branches keep the request open. - Repeated paused maintenance reproduced two needless GitHub edits. The writer now skips an identical complete comment body. Nine affected cases passed, covering repeated maintenance, resumption, stopped work, final assessment preservation, and main-comment reuse. - Safety cases cover lost receipts, legacy adoption, newer progress, published assessments, provider-verified mention aliases, and old replies that must not complete new requests. - `pnpm -r typecheck`, `pnpm build`, map validation, and module boundaries passed. The server TypeScript check passed after the final repair. The broader local GitHub workflow and policy run passed 188 tests. Full CI passed on the final commit (`554979d79285b68bcbf90cd639e3d2d053f4ea4b`), including all three chat shards, server/workspace/Runner suites, typecheck, build, and E2E gates. Greptile is 5/5 with no open review findings. One serialized job received a runner shutdown signal mid-test; its single retry passed. - Preserved failure evidence: the first CI run exposed an overbroad recipient filter and an old queued-text assertion. The next commit passed all three CI chat shards. Greptile then found the receipt transitions covered by the new regressions above. Local full-suite runs were stopped when those defects required source changes; no clean full local pass is claimed. - Tests use real PostgreSQL and controlled provider responses. They do not establish a live GitHub App rollout. The original instance needs the fixed server before its comments change. ## Risks - Leading mentions now select a recipient even when that account has no Paperclip connection. A leading human mention also selects that person. Mid-message incidental mentions retain normal subscription behavior. - Existing comment adoption edits only comments proven by this endpoint's request ledger or prior summary marker and App identity. Deleted and replaced comments remain protected. - Failure cleanup uses current authority and does not replace newer progress or unresolved provider writes. It does not restart failed execution. - No schema migration is required. ## Model Used OpenAI Codex, GPT-6. The exact serving model ID and context-window size are not exposed in this session. The model used reasoning, tools, code execution, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 21:51:20 -05:00
DottaandPaperclip	2cbca4ceb0	fix: retain channel tools across task continuations (#16806 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat channels give an agent tools for its assigned task. > - Each later run must resolve the task and its accepted requester again. > - Some native runs store the task in a database column without the older JSON fields. > - Tool discovery read only the JSON fields. GitHub calls also required a new inbound comment. > - This pull request uses the durable task binding and accepted request history. > - Later runs retain their channel tools while current access checks still apply. ## Linked Issues or Issue Description What happened? A GitHub task can find `begin_review` on its first run, then receive an empty tool list on a later native run. If discovery is repaired alone, a call can still fail because the later run has no new inbound comment. Slack and voice tool access also read the older JSON task fields. Expected behavior A later run for the same task can find and call its assigned channel tools. The current requester, task, company, agent, endpoint, and access rules must still allow the call. Revoked access must fail. Steps to reproduce 1. Start a GitHub task and accept its linked requester. 2. Confirm that the runtime can find `begin_review`. 3. Start a later native run with `nativeIssueId`, accepted parent identity, and no JSON task ID or new wake comment. 4. Search for `begin_review` again. Before this change, the search returns an empty list. 5. Repair discovery alone and call `read_pull_request`. Before the request-history fix, the call fails with no active conversation capability. Paperclip version or commit Reproduced in source tests before this patch. The branch includes master at `fa50cab5685b9cdbb26ad9c610cc5b6c373d3ff4`. Additional context This is a core run-binding bug. The tests use PostgreSQL and synthetic provider responses. They do not send live reviews, Slack messages, or phone calls. ## What Changed - Resolve run task IDs from durable columns, with legacy JSON fields as a fallback. - Use the same binding in the gateway, access policy, identity service, and channel runtime preparation. - Recover accepted channel request history across later runs. Keep company, task, agent, and requester boundaries. - Recover older wake sources through verified parent runs, including after same-user board follow-ups. Clear the board message’s own legacy wake fallback. Keep cycle checks without a fixed limit on valid task history. - Retain the GitHub origin after a board comment from the same accepted user. Skip GitHub origin lookup results from other providers. - Keep ordinary board tools available when the requester changes. Accept legacy user comments without an author-type field. Still reject explicit malformed task IDs. - Prevent a later failure reply from replacing an earlier final GitHub answer. Recheck this rule under the publication lease. - Rotate the native tool contract to replace retained sessions with an older tool list. - Add continuation, publication, revocation, and boundary tests. Update the access and feature maps. ## Verification - Red: the later GitHub assigned-tool search returned an empty list before the binding fix. - Red: the later GitHub read failed with no active conversation capability before the request-history fix. - Green: runtime MCP discovery, GitHub read/begin/submit review, and response publication pass for monitor and same-user board continuations. - Green: Slack history, retained access controls, revocation, and voice gateway continuation pass. - Green: 35 channel, failure-publication, legacy-origin, long-history, and provenance tests pass. Both review findings had failing tests before their fixes. - Green: 107 gateway, operation-credential, identity, and provenance tests pass. These cover both first-run CI failures without changing their tests. - Green: 122 runtime, discovery-policy, and native-session tests pass; 2 unrelated tests are skipped. - Map checks and whitespace checks pass. - Updated to master in `cfcaba22628772d7fe1fb32b57ff7031c7072781`. The sole conflict was two tests added at the same location. Both the continuation test and the publication 403/401 test are retained. - Current local verification: all 179 gateway, credential, requester-identity, access-policy, runtime-context, and session-resume tests passed. Repository typecheck and build passed. Map and whitespace checks passed. - The new review regression first returned no source for a same-user board follow-up over legacy wake metadata. After the repair, all 27 provenance and channel cases passed. The combined GitHub discovery/read/begin/submit path and revoked access are covered. - The broader local channel run passed 176 cases and failed four timing/setup cases: a 15-second Slack authorization timeout, webhook-verification polling, a Slack acknowledgement deadline, and Discord setup polling. All four passed in isolation without source, assertion, or deadline changes. The earlier failure record is retained; the exact load/order cause is not established. Full CI must pass on the latest commit. - Current head: `7b9a08dcbc7b8140b589742f136944270b0496bb`. Repository typecheck and build pass on this commit. Greptile completed on this exact commit with 5/5; all four review threads are resolved. The final-head [CI run](https://github.com/paperclipai/paperclip/actions/runs/39278921653) passed: 55 checks passed and 2 intentionally skipped, with no failures or pending checks. This includes the full sharded test matrix, browser tests, typecheck, build, and Runner verification. Results were delayed during the [GitHub Actions incident](https://www.githubstatus.com/incidents/8tzbpcnl75jp). An additional unsharded local full test run has not completed; the completed full-suite evidence is CI. - Earlier verification: the first PR commit failed two server suites. Those failures were fixed and verified before this update. The original 4 GiB sandbox could not complete full local typechecking or tests; the current checkout completed typechecking and build. The full test suite runs through the CI shards. No full local test-suite pass or live provider delivery is claimed. ## Risks - Request history can restore the wrong authority if its scope is too broad. The resolver checks each accepted parent against the task, company, agent, and requester. Tests cover invalid and cyclic history. - The native contract change starts a new provider session for older contracts. - No database migration or public API schema changes are required. Each tool call still checks live access. ## Model Used - OpenAI Codex, GPT-6 family. The runtime does not expose the exact deployment model ID or context-window size. The agent used reasoning, repository tools, code edits, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 11:20:39 -05:00
DottaandPaperclip	9966f31bee	fix(chat): keep permission failures scoped to the rejected operation (#16847 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - External channels deliver agent replies through a durable publication queue. > - A provider can reject one operation while other operations remain allowed. > - The shared error classifier treated generic 403 and permission errors as failures of the whole connection. > - Failure handlers then changed the user's enabled setting and removed all assigned channel tools. > - This pull request limits permission failures to the rejected operation and preserves connection intent during authentication failures. > - Unrelated work can continue, while confirmed authentication failures still stop endpoint traffic. ## Linked Issues or Issue Description Refs #16137. Follows #16138, which distinguishes local authorization errors from provider errors. This change also narrows provider permission failures. Related #16806 handles a separate task-continuation tool-binding defect. A provider 403 or missing-scope response currently disables a connection that may still support other operations. The expected behavior is a visible operation failure with unrelated work preserved. Confirmed authentication failures must stop sends without changing the user's enabled setting. ## What Changed - Classify generic 403, adapter permission errors, and Slack missing-scope/no-permission responses as failed operations. - Preserve rate-limit retries, known destination quarantine, and confirmed authentication failures. - Preserve connection status and enabled intent when publication, provider-reply, or form-open authentication fails. The endpoint still enters attention and its runtime stops. - Add GitHub regressions for retained discovery and successful later publication after 403, and no further sends after 401. - Mark authentication health failed and fence persistent receiver startup, background reconciliation, and receiver callbacks. Keep signed GitHub lifecycle events available during an authentication stop. Verify Discord reconciliation and Photon restart preserve the stop, while transient Discord disconnects still recover. - Update provider, publication, and modal regression expectations and the execution contract and verification maps. ## Verification - Red: five new classifier cases returned endpoint attention before the repair. The signed-installation-removal regression also reproduced a blocked lifecycle event after a 401 before narrowing the receiver guard. - Green: all 35 classifier tests pass. - Green: 19 PostgreSQL-backed cases pass through the production channel services, including GitHub discovery/publication, Slack provider replies and question modals, Telegram credential/destination failures, rate limiting, and Discord/Photon authentication-stop persistence. - Map structural checks and `git diff --check` pass. - `pnpm -r typecheck` and `pnpm build` pass. All CI gates are green on `7e379bcd98aa8363e42efe7f8a4b2589b7111ae6`; Greptile is 5/5 on that exact commit with no unresolved findings. - The monolithic local `pnpm test:run` was started, then stopped after its revision passed the full CI matrix and was superseded by the authentication-stop repair. Final full-suite verification passed in CI shards; all 19 final local provider regressions pass. - The tests run from this source checkout on macOS arm64 with isolated PostgreSQL and controlled provider failures. They do not prove live GitHub recovery or deployment. No live connection settings were changed. ## Risks - Permission failures now leave unrelated work available. The rejected operation remains failed and is not automatically replayed. Provider authorization still applies to every request. - Keeping a connection enabled must not bypass an authentication stop. GitHub verifies blocked tool discovery and later sends; Discord and Photon verify background reconciliation and restart cannot clear the stop. Failed connection health is an execution gate while the endpoint requires attention. - Explicit disable, disconnect, signed installation revocation, and existing destination-specific denial handling remain unchanged. No migration or API schema change is required. - Previously disabled connections are not automatically re-enabled because an upgrade cannot safely infer the user's intent. ## Model Used - OpenAI Codex, GPT-6 family. The runtime does not expose the exact deployment model ID or context-window size. Used reasoning, shell tools, repository edits, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 09:54:22 -05:00
DottaandPaperclip	669d329f25	perf(slack): avoid unused catalog revision scans (#16630 ) Slack catalog discovery now checks live task, user, endpoint and credentials without calculating an unused compatibility revision. Full authority callers retain the unchanged revision and search-token path. Validation: 44 affected cases, full repository typecheck/build, current-head full CI/E2E and security checks, Greptile Base 5/5 with no unresolved findings. Identical production catalog output with revision statements reduced from two to zero in real-PostgreSQL comparisons. Hosted latency acceptance remains pending. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 22:21:03 -05:00
DottaandPaperclip	3db33c131e	perf(slack): overlap endpoint preparation and narrow credential reads (#16593 ) Prepare Slack endpoints in pairs with a per-call concurrency limit of two, and resolve only the current bot token for live tool authority. Preserve candidate-order errors, active-sibling settlement, credential rotation and authorization revocation. Verified with real PostgreSQL regression controls, full repository CI/E2E, security checks and current-head Greptile Plus5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 16:05:46 -05:00
DottaandPaperclip	f1d4bc1617	perf(slack): resolve only signing secret during webhook checks (#16548 ) Resolve only the current Slack signing secret during workspace checks and authenticated setup evidence. Preserve full runtime credential resolution, company bounds, rotation, and auditing. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 03:05:05 -05:00
DottaandPaperclip	becdc3b08e	perf(slack): remove delays before mention acknowledgements (#16491 ) Slack mentions now start eyes reactions after the authorized task/comment commits, independently of attachments, subscription and scheduler acceptance. Omit unused optional URL-preview polling while retaining message text, links and files. Keep receipt failures and missing-file warnings independent, including terminal cleanup of older multi-line diagnostics. Verified with full current-head CI and E2E, Superagent and legacy security checks, Apex 5/5 with no unresolved findings, 19 selected real-PostgreSQL receipt/authorization/cleanup cases, and full frozen-source typecheck/build. The first CI attempt's unchanged Cursor probe timeout is retained in the PR; the one failed-only retry passed under unchanged deadlines. Live Slack latency still requires deployed acceptance. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 17:36:50 -05:00
DottaandPaperclip	25b16e65fc	fix(observability): detail workspace and connector preparation (#16454 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server prepares a repository, sandbox workspace, and connector access before an agent can reply. > - Existing traces measure large preparation phases but leave important work inside those phases unclear. > - Repository source materialization and Browser Use discovery can do more work than their callers suggest. > - New detail must preserve access checks, run ownership, and the existing work order. > - This pull request adds fixed OpenTelemetry spans inside those operations. > - Operators can use one deployed trace to select the next performance fix. ## Linked Issues or Issue Description Refs: #16419, #16425, #16428. What existing behavior does this improve? Startup traces for server workspace preparation, native sandbox synchronization, and connector assignment resolution. Current behavior Workspace and connector aggregate spans can take many seconds. The trace does not separate private source cloning, seed reuse, archive work, progress persistence, or each connector resolver. Browser Use resource selection builds the full authorized tool catalog. The configuration and native backend can each resolve connector assignments. Proposed behavior Show the awaited work as fixed child spans. Separate discovery queue wait from catalog construction. Preserve the initiating run context when another run releases a scheduler slot. Add no new lifecycle event writes for workspace detail callbacks. Reason and benefit Operators need evidence about the hosted critical path before another performance change. These spans distinguish filesystem work, provider transfer, authorization, transaction entry, locks, and reporting overhead. Breaking changes None. Authorization, query order, transaction ownership, synchronization, existing progress events, and task results retain their contracts. The change adds opt-in Observability spans under the existing OTLP endpoint gate. ## What Changed - Trace private source directory setup, Git credentials, clone, requested ref, source identity, and checkout publication. - Trace sandbox source capture, seed cache/replay, shallow clone and archive, overlay copy/archive, verification, upload, warm stamps, and descriptor persistence. - Separate existing awaited progress reporting from useful workspace work. Keep new detail callbacks trace-only. - Trace each connector resolver and Browser Use catalog/grant work. Split discovery queueing, descriptor sources, and policy evaluation. - Trace Slack authority, credential reads, catalog synchronization, transaction entry/settlement, and advisory locks. - Update the real filesystem staging trace expectations and the Honeycomb investigation guide. - Add a real scheduler regression for a queued job released by another run. Assert run ownership, parent spans, and exactly one completed queue span. ## Verification - Passed: server and adapter-utils TypeScript checks (`pnpm --filter @paperclipai/server exec tsc --noEmit` and `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`). - Passed: 252 checks across 9 focused files. These cover real filesystem staging, native workspace synchronization, trace ownership, endpoint gating, and workspace preparation. - Passed after review: 26 trace-context, tracing-failure, and discovery-scheduler checks across 3 files (these overlap the first run). The new queued-discovery regression fails when context binding is removed and passes after restoration. An initial submillisecond timestamp assertion was replaced with explicit span-completion order to avoid SDK clock precision assumptions. - Passed on implementation commit `0e68968a4bd61001893d833f281f12bb110edbe1`: `pnpm -r typecheck` and `pnpm build`. The later commit updates only an engine test expectation. - The first CI workspace lane found an old engine trace-tree expectation. It expected the earlier span set. The updated assertion now checks the new detail spans, their parents, and single completion. Its targeted executor check passes. - The full test suite, build, release registry, and end-to-end gates pass in CI on final head `40e37d89ef679109d60a96bb5731a0616efeaa43`: 55 passing checks and 2 optional Storybook skips. The full suite was not duplicated locally after the focused run. - Greptile reviewed the final head at 5/5. No unresolved review threads remain. - After deployment, filter Honeycomb by the hashed run ID and verify `service.version`. Inspect `heartbeat.prepare.workspace`, `runtime_selection.workspace_sync`, and the connector assignment parents. The guide in `doc/observability.md` explains each new family and the nested intervals. - No hosted run has used this new instrumentation yet. Local checks cannot establish hosted latency improvement. ## Risks - Tracing adds spans when an operator configures an OTLP endpoint. Detail remains capped at 16 spans per phase per execution attempt. - Parent and child durations overlap. The documentation explains how to avoid counting them twice. - A discovery job can start in another run's scheduler continuation. The bound callback preserves the initiating async context. - New spans export fixed names and the existing hashed run ID. They add no paths, repository URLs, connector resource IDs, credentials, tool schemas, or exception text. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 12:21:32 -05:00
DottaandPaperclip	09a2f0b7b0	refactor(server): isolate provider message admission and runtime policies (#16289 ) ## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn external messages into shared task workflows. > - The main service still contains some provider decisions. > - These decisions belong with the existing channel modules. > - This pull request isolates provider message-admission and runtime policies. > - Shared authorization, persistence, and runtime ownership keep their current order. ## Linked Issues or Issue Description Refs: #16235. Stack part 3/3. Base: `master`. #16276 and #16283 are merged. What existing behavior does this improve? Provider ownership and readability in the chat channel service. Current behavior The entry point combines shared workflows with provider decisions. Proposed behavior Provider modules own their decisions. Shared modules own product state and authorization. Reason and benefit Maintainers can find provider behavior in its channel folder. Breaking changes None intended. Public service methods and durable data shapes stay compatible. ## What Changed - Move Discord SDK admission and its barrier to the Discord message module. - Move Photon live-message eviction and startup recovery policy to its runtime callbacks module. - Move Slack mention eligibility and Teams setup-channel eligibility to their message modules. - Keep Photon deferred ingress and Speko conversation policy with their provider modules. - Keep ordered SDK message batching in a shared runtime module with a live per-message ingress callback. - Keep receiver leases, cache lifetime, startup failure cleanup, authorization locks, and transaction ownership shared. - Reduce `chat-channels.ts` from 3,570 to 3,548 lines. ## Verification - The same 46 runtime/admission integration cases and all 18 Photon channel cases passed before and after in the owned Linux container. - The existing first-channel Teams orphan-admission case passed before and after moving its eligibility predicate. - The same 94 SDK adapter and lifecycle-normalization tests passed locally. - Server typecheck and module-boundary checks passed. - Token comparison confirms ordered SDK batching and every processor argument, plus Photon invalidation and its live runtime lookup. - The shared factory creates no cache or worker. It reads the ingress policy for each message and awaits processing in order. - The extraction rebased without conflicts onto master after #16276 and #16283 merged. Git range-diff reports the same patch. Final head `7b63b569e2a0dbcc959310f565f77f367fa64aa0` has the same patch by range-diff. The 94 SDK/lifecycle tests, server typecheck, and boundary check passed on the final head. All CI gates pass on that exact head. Greptile reviewed it at 5/5 with no inline findings or unresolved review threads. - One Runner CI test failed its original short timing window. The unchanged test passed earlier-head CI and local reproduction. The single failed-job retry passed. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321 channel integration cases passed. The normal UI, CLI, database, adapter, plugin, and serialized API test selections ran. - The first Linux `pnpm test:run` returned 1. Original-source runs reproduced the local failures. Container-only repairs supplied missing tools, process reaping, and sufficient Node heap. The existing mocked-fetch CLI telemetry fixture passed with the image opt-out flag overridden only for that fixture. - One local venue limitation remains: adapter-utils T24 fails unchanged on both original and final source in the Linux container. Its entire 56-case suite passes before and after on the host. All other failed or aborted suites passed in repaired reruns. No test assertion or timeout changed. - These fixtures prove application integration. They do not prove live provider execution. ## Risks - A captured option value or changed callback order could affect deferred ingress or stale-runtime handling. - The ingress callback and runtime getter remain live. Core maps and receiver lease operations keep one owner. - No schema migration or public API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 10:50:34 -05:00
DottaandPaperclip	7df55f3dcc	refactor(server): isolate provider webhook lifecycle decisions (#16283 ) ## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn external messages into shared task workflows. > - The main service still contains some provider decisions. > - These decisions belong with the existing channel modules. > - This pull request moves webhook recovery and payload-scope decisions to their provider modules. > - Shared authorization, persistence, and runtime ownership keep their current order. ## Linked Issues or Issue Description Refs: #16235. Stack part 2/3. Base: `master`. #16276 is merged. What existing behavior does this improve? Provider ownership and readability in the chat channel service. Current behavior The entry point combines shared workflows with provider decisions. Proposed behavior Provider modules own their decisions. Shared modules own product state and authorization. Reason and benefit Maintainers can find provider behavior in its channel folder. Breaking changes None intended. Public service methods and durable data shapes stay compatible. ## What Changed - Move GitHub revoked-installation recovery classification to `github/webhook-scope.ts`. - Move Teams payload-scope validation to its message module. - Record normalized GitHub, Telegram, and Teams lifecycle events through one shared durable call. - Keep Slack callback ordering and persistence failures outside parser catches. - Reduce `chat-channels.ts` from 3,598 to 3,570 lines. ## Verification - The same 86 selected webhook/lifecycle integration cases passed before and after in an owned Linux container. - The same 17 selected Telegram edit and Teams edit/delete/restore cases passed before and after in that container. - The same 94 SDK adapter and lifecycle-normalization tests passed locally before and after. - Server typecheck and module-boundary checks passed. - Token comparison confirms the shared recording payload and runtime fence, plus the Teams scope and parser inputs. - The host database baseline failed before assertions because System V shared-memory IDs were exhausted. No other PostgreSQL server or host setting was changed. Database verification used an isolated container under its unprivileged user. - The extraction rebased without conflicts after #16276 merged. Git range-diff reports the same patch. The 94 SDK/lifecycle tests and server typecheck passed on this base. All CI gates pass on rebased head `d835d7c3815c168226fa646aec6370b6a8ac94c1`. Greptile reviewed that exact head at 5/5, with no inline findings or unresolved review threads. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321 channel integration cases passed. The normal UI, CLI, database, adapter, plugin, and serialized API test selections ran. - The first Linux `pnpm test:run` returned 1. Original-source runs reproduced the local failures. Container-only repairs supplied missing tools, process reaping, and sufficient Node heap. The existing mocked-fetch CLI telemetry fixture passed with the image opt-out flag overridden only for that fixture. - One local venue limitation remains: adapter-utils T24 fails unchanged on both original and final source in the Linux container. Its entire 56-case suite passes before and after on the host. All other failed or aborted suites passed in repaired reruns. No test assertion or timeout changed. - These fixtures exercise application state and the service entry point. They do not prove live provider execution. ## Risks - A scope or ordering error could record an event for the wrong provider context. - The selected parser returns normalized data. The shared service keeps durable recording and runtime fencing. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 10:38:12 -05:00
DottaandPaperclip	f16c0ad5c8	refactor(server): isolate chat conversation and control policies (#16276 ) ## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn external messages into shared task workflows. > - The main service still contains some provider decisions. > - These decisions belong with the existing channel modules. > - This pull request separates conversation binding and Telegram control policy. > - Shared authorization, persistence, and runtime ownership keep their current order. ## Linked Issues or Issue Description Refs: #16235. Stack part 1/3. Base: `master`. What existing behavior does this improve? Provider ownership and readability in the chat channel service. Current behavior The entry point combines shared workflows with provider decisions. Proposed behavior Provider modules own their decisions. Shared modules own product state and authorization. Reason and benefit Maintainers can find provider behavior in its channel folder. Breaking changes None intended. Public service methods and durable data shapes stay compatible. ## What Changed - Keep shared conversation lookup and completion in `inbound/conversation-binding.ts`. - Move Photon binding-completion policy to its messages module. - Move Telegram command classification, forum behavior, and chronology exemptions to its commands module. - Keep the original conversation generation separate from the retained task binding. - Reduce `chat-channels.ts` from 3,649 to 3,598 lines. ## Verification - The same 62 selected Telegram/Teams integration cases passed before and after extraction. - All 18 Photon channel integration cases passed before and after extraction in separate test processes. - The initial combined baseline command had a Photon database-startup hook timeout. No Photon assertion ran in that invocation. The unchanged Photon suite then passed in isolation with its existing timeouts. - Server typecheck and the module-boundary check passed. - A token comparison confirms the moved lookup, completion, and generation logic, after normalizing the explicit policy callbacks. - All current-head CI checks passed. Greptile reviewed this exact head at 5/5 with no findings. - Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321 channel integration cases passed. The normal UI, CLI, database, adapter, plugin, and serialized API test selections ran. - The first Linux `pnpm test:run` returned 1. Original-source runs reproduced the local failures. Container-only repairs supplied missing tools, process reaping, and sufficient Node heap. The existing mocked-fetch CLI telemetry fixture passed with the image opt-out flag overridden only for that fixture. - One local venue limitation remains: adapter-utils T24 fails unchanged on both original and final source in the Linux container. Its entire 56-case suite passes before and after on the host. All other failed or aborted suites passed in repaired reruns. No test assertion or timeout changed. - These fixtures exercise the assembled service and persisted state. They do not prove live third-party execution. ## Risks - Changing query order or losing the original generation could bind a message to the wrong task. - The workflow retains the original lookup result and all existing query and write conditions. - No schema migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 10:21:48 -05:00
DottaandPaperclip	f69c6a8a35	fix(github): retry and recover review publication (#16284 ) ## Thinking Path > - Paperclip manages agent work and its published results. > - GitHub review bots publish task-bound assessments through governed tools. > - A review can survive a caller timeout because its publication intent is durable. > - A transient provider failure still returned a failed receipt before recovery. > - Gateway replay could return an empty cached result, and publication retries had no dedicated scheduler deadline. > - This pull request adds bounded retries, receipt recovery, and an independent durable worker. > - A temporary GitHub failure can recover without another review or duplicate publication. ## Linked Issues or Issue Description Refs #16275. That change increased the GitHub tool deadline. This change repairs retry and recovery behavior. What happened? A one-off GitHub HTTP 503 or 429 left the caller with a failed publication receipt. The saved retry did not have its own scheduler worker. An identical gateway call after a timeout could return an empty cached result instead of the saved publication receipt. Expected behavior Temporary failures should retry under current authority. Pending work should resume at its saved deadline after restart. Exact retries should return the current receipt. Lost responses must not create duplicate comments, findings, or checks. Steps to reproduce 1. Admit an advisory PR review request and begin its assessment. 2. Return HTTP 503 or 429 for the first summary-comment PATCH, then let GitHub succeed. 3. Submit the assessment through the run-scoped tool gateway. 4. On master, the result says publication failed. With this change, the same call returns the processed receipt and neutral check. 5. For receipt recovery, use a shorter caller deadline and repeat the exact call after publication finishes. Paperclip version or commit Reproduced on master at `6cb6ac3b8e40cb90d734ba115a2f22f850ff9477`. ## What Changed - Classify transient transport, server, and rate-limit failures. Honor Retry-After and primary reset deadlines. Stop retries for permanent permission and validation errors. - Retry safe reads and installation-token issuance within a bounded call window. Retry publication writes through the existing outbox and provider receipt checks. - Wait for up to three publication attempts within a 45-second retry window. Keep the durable eight-failure cap and enforce saved deadlines on explicit retries. - Register an independent GitHub publication worker with transaction-owned wake notifications, startup recovery, and per-PR lease deadlines. - Recover exact comment, progress, and assessment gateway calls from their stored publication. Require the same company, agent, task, run, connection, tool, arguments, and current policy. Clear stale timeout errors after recovery. Approved exact retries read the current receipt through the existing signed replay path; the worker retains the verified approval for deferred writes and still rejects revocation. - Add regression coverage and update the connector guide and feature/access maps. ## Verification - Red: both one-off HTTP 503 and 429 regressions failed on master with a failed publication receipt. - Green: the same tests passed after the change. - GitHub review workflows: 133 covered against isolated PostgreSQL 18. The final combined run passed 132 and hit one fifteen-second timeout under concurrent build load. All three variants of that workflow passed in isolation. The earlier combined run passed all 131 workflows before the two approval regressions were added. - Retry policy and delivery coordinator tests: 35 passed. - Additional focused checks passed for the actual eight-failure cap, exact-argument conflicts, policy revocation, and timeout receipt recovery. The approved deferred-publication and revocation regressions failed before the review repair and passed after it, with one approval request preserved. - `pnpm -r typecheck`: passed. `pnpm build`: passed. `node scripts/check-maps.mjs`: passed. - The broad local `pnpm test:run` path remains blocked by the host's exhausted PostgreSQL shared-memory slots. The changed database-backed workflows use an isolated Docker database instead. All Paperclip CI jobs passed at `a6d81b67cb`, including the required verify and E2E gates. Greptile is 5/5 at the same head, with the approval-recovery finding resolved. The external Superagent Security Scan also passed. - Live acceptance passed on a dedicated single-repository local test App and native Paperclip Runner. A test-only preload returned one HTTP 503 before the final summary PATCH. One submit_review call completed in 17.305 seconds; the bot published its 3/5 advisory review and neutral check, then automatically finished the task. [Published review](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/pull/9#issuecomment-6168809007), [neutral check](https://github.com/paperclipai/paperclip-permissions-smoke-20260926-pap57-fee7428e/runs/115863564014). This live test ran at de6862072c; approval-specific recovery is covered by the later regressions. The preload is excluded from the repository and removed from the running local instance. ## Risks Retries make additional bounded provider requests. A provider write with a lost response is recovered through its existing App-owned identity before another write is permitted. The worker rechecks authority and the PR head; revocation and superseded work stop publication. Calls can still exceed their deadline, and repeated failures eventually exhaust the retry cap. A pending publication is not a completed task. The agent still owns task completion after receiving a confirmed receipt. No schema migration is needed. ## Model Used OpenAI Codex (GPT-6 family), with code inspection, editing, and test execution. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 08:40:32 -05:00
DottaandPaperclip	e9b6c4d44c	perf: remove history scans from task and chat queries (#16278 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task reads and external chat notices must remain responsive as run history grows. > - The chat notice sweep casts indexed UUID columns to text. > - Run redaction and task provenance filter large JSON contexts with expressions that lack matching indexes. > - These queries scan history and read large stored contexts on common requests. > - This pull request adapts two existing fixes to current master and adds the missing provenance index. > - The benefit is less database work with the same query results and access rules. ## Linked Issues or Issue Description Refs #15927. This change adapts that chat sweep fix to the extracted inbound notice module and retains the contributor credit. Refs #12499. This change adds that missing nested-context index through a current, concurrent migration. It retains the contributor credit. This combined change replaces those older layouts for the measured query paths. It also covers task creation provenance, which those pull requests do not address. What happened? The chat notice sweep scans wakeup history. Issue redaction scans run contexts because only one branch of its JSON condition has an index. The task creation-source filter also scans contexts because its native and legacy ID expression lacks an index. Expected behavior The queries use indexed lookups and return the same rows. Company access, task origin precedence, secret redaction, and chat notice ordering remain the same. Steps to reproduce Run `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-slow-queries.ts --baseline-ref=f8af817f8d`. The benchmark creates disposable PostgreSQL data and compares the old query with the current query on the same fixture. ## What Changed - Compare chat UUID columns directly with safely parsed payload IDs. Reject malformed and noncanonical IDs without a cast failure. - Add the nested `paperclipIssue.id` run-context index and the exact native/legacy task-provenance expression index. - Build both indexes concurrently. Register the migration for top-level execution and invalid-index retry repair. - Extend chat payload and migration regression coverage. - Add a repeatable synthetic SQL benchmark and document its scope and command. - Keep the five newest schema snapshots. ## Verification - Focused local checks: 76 selected tests passed in five files. These cover FIFO notices, malformed IDs, conversation state, redaction, provenance, and company boundaries. - Disposable PostgreSQL 18.1 fixture: 335 tasks, 1,400 runs, 13,000 wakeups, 377 actions, and 192 KiB synthetic contexts. - Five warm samples per query, after two warmups. Every returned SQL column matched before and after, compared as positional tuples so duplicate unnamed projections are retained. \| Query \| Before, median \| After, median \| Improvement \| \| --- \| ---: \| ---: \| ---: \| \| Chat notice candidates \| 26.21 ms \| 6.43 ms \| 4.1× \| \| Task creation-source list \| 159.69 ms \| 6.51 ms \| 24.5× \| \| Issue redaction contexts \| 131.88 ms \| 0.34 ms \| 389.1× \| The run-history context projection returned the same 200 rows and kept the same index plan and buffer counts. Its timings were noisy (37.30 ms before and 56.78 ms after in the last run; 34.25 ms and 34.14 ms in the previous run). No improvement is claimed for this control. Its repeated JSON projection remains a separate optimization opportunity. The populated plans use the wakeup primary key, the provenance expression index, and a BitmapOr over both redaction indexes. These are local SQL measurements. They do not prove an equivalent change in hosted API latency. - `pnpm -r typecheck` and `pnpm build` passed locally. - The full local `pnpm test:run` was started and then interrupted (exit 130) when the Mac reached its PostgreSQL shared-memory limit. It did not complete and is not reported as passed. The final SQL benchmark passed after its disposable database resources were released. The full general and serialized test suites passed in CI. - All CI checks for final head `193acca6abaa992acad12f4b631bd31dd3234641` passed, including full Vitest coverage, E2E checks, typecheck, build, migration order, and lifecycle boundaries. - Greptile reviewed that exact head at 5/5 with no actionable comments. ## Risks - Both indexes add storage and write cost. Initial builds read stored run contexts and can take time on large histories. - Concurrent index builds avoid holding a write-blocking table lock for the whole build. The existing migration retry path removes invalid partial indexes before retrying. - The chat parser accepts only canonical lowercase UUID text. This preserves the prior text comparison, including rejection of uppercase and malformed values. - Staging has not been migrated or deployed. Local PostgreSQL and synthetic data do not reproduce hosted load or network latency. ## Model Used - OpenAI Codex, based on GPT-6. The exact serving model ID and context window were not exposed in this session. Used reasoning, repository tools, browser inspection, and local code execution. The adapted fixes retain their original contributor credits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 07:37:51 -05:00
DottaandPaperclip	2c17d3ca52	fix(github): recover webhook verification in attention state (#16263 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review bots require signed webhook delivery before activation. > - Cloud can deduplicate an old ping after a connection repair. > - A fresh event could verify active connections, but an attention-state connection discarded it before authentication. > - This pull request allows signed delivery verification while the connection needs attention. > - It discards ordinary work until the manager completes the existing reconnect and activation checks. ## Linked Issues or Issue Description Refs #16259. Related: #16138 addresses the separate classification of local authorization failures. What happened? Live verification of #16259 found that a GitHub App already marked as needing attention remained stuck waiting for a signed ping. The Cloud relay had already delivered that ping. New signed events were discarded by the attention-state ingress gate before they could verify delivery. Expected behavior A fresh valid event from the bound installation verifies delivery. It does not activate the connection, change repository access, or start work. The manager can then complete the normal reconnect flow. Steps to reproduce 1. Put an existing App connection into attention state with webhook verification cleared after repair. 2. Deliver a fresh signed comment event for its bound installation. 3. Continue setup. Before this fix, webhook verification remains missing and reconnect cannot proceed. ## What Changed - Use a content-free verification path for signed ordinary GitHub events while the connection needs attention. - Allow a fresh signed event from the same installation to verify webhook delivery. - Verify delivery even from disabled or unavailable repositories. Discard the event before content storage, resource reconciliation, or runtime creation. - Keep newly verified connections in the provider-setup step. Later installation events cannot activate them before reconnect. Preserve normal installation recovery for existing qualified connections. - Extend recovery regressions to cover attention state, foreign installations, invalid signatures, missing installation IDs, unchanged resources, no wakeups, and successful reconnect after verification. - Document the recovery behavior. ## Verification - The targeted regression reproduced the attention-state failure before the change. - All ten recovery cases pass after the change. The tests cover disabled and unavailable repositories, content-free verification, and installation events after verification. - Server typecheck passed. - All 260 GitHub integration tests passed after the review fixes. Final server typecheck also passed. - `pnpm -r typecheck` and `pnpm build` passed. - The full local suite is running. One unrelated workspace-runtime conflict-reference test failed and then passed on its isolated rerun. Final totals and CI/review outcomes will be updated before merge. - Live verification will use the existing advisory review bot after deployment. ## Risks - Attention-state ordinary events authenticate under the current credential lease without creating a durable content record. They return before resource reconciliation or runtime creation. - Verification is still bound to the current credential lease, runtime generation, valid signature, and installation ID. It does not activate the bot or replace normal setup checks. - Paused, revoked, archived, and unconfigured connections retain their existing ingress rules. No schema changes or permission expansion. ## Model Used OpenAI Codex (GPT-6), with code inspection, editing, test execution, and browser verification. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 05:23:42 +00:00

1 2