3 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	65fa1cd617	fix(server): schedule GitHub review checks from committed changes (#17662) ... <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/213/changes/paperclip/213 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review checks report the state of admitted agent work. > - The chat maintenance loop searches for check work even when none remains. > - Existing action rows already record unfinished publication and retry deadlines. > - Committed changes can wake those rows through the shared coordinator. > - An unresolved check must still detect a remote head change when a webhook is missed. > - This pull request schedules only outstanding checks and leaves settled history quiet. ## Linked Issues or Issue Description **What happened?** The GitHub chat maintenance callback repeatedly queries review-check actions. Empty queues and completed history still incur database reads. Local state changes wait for the maintenance callback. **Expected behavior** Committed check creation, review binding, run changes, and publication-lease release wake the worker. Existing retry and remote-head deadlines apply only to unfinished checks. A settled worker makes no recurring database or provider calls. **Steps to reproduce** Configure a GitHub bot with no outstanding checks, or finish all of its checks. Leave the instance idle. The old maintenance callback continues to call the check service. Refs #17661, the merged terminal-response parent. Related #17656 migrates response acknowledgements. Related #16385 changes branch-policy admission; this PR preserves that separate policy boundary. Late stale-head projection repair remains a separate follow-up slice. ## What Changed - Move the existing check-state reader into the GitHub provider owner. Preserve current authority, run selection, assessment ownership, and check replacement behavior. - Register a scoped worker with the existing coordinator and scheduler. Existing received and retryable-failed actions remain durable truth. No new table is added. - Signal check enqueue, task review receipts, manual review insertion/binding, run lifecycle changes, and publication-lease release through existing transaction ownership. - Preserve the saved failure backoff, eight-attempt policy, and competing lease expiry. Keep five-second remote-head reconciliation only while a received check remains unresolved. - Check admission before each HTTP request. Shutdown leaves durable work recoverable without spending a provider failure attempt. - Remove check dispatch from the periodic GitHub callback. Keep the separate late-projection repair and the manual combined recovery method compatible. - Add an idempotent concurrent partial index for unfinished check selection. Retain the newest five generated snapshots. - Add eleven real-database worker cases and startup wiring coverage. Document producer coverage and the external reconciliation limit. ## Verification Current reviewed source: `2bba08403b195bf646eda48992b46d6f5cebf756`. - Normal merges preserve the complete review-check patch, the landed terminal-response parent, and current master changes. No landed SQL migration or journal entry changes. - At `6995f84b240a14a7b8660e1cf095b3aa1e1e3168`, **331 distinct tests pass**: 264 complete GitHub workflow cases, 30 completion cases, eight issue-tree producer cases, 16 login-cleanup cases, and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. Login cleanup uses its real embedded PostgreSQL fixture. - Combined-master acceptance: **277 distinct tests pass** at `9c538aed1251fd7f7cecb829a0ecbce6110fb1ed`: 264 complete GitHub workflow cases and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. A new PostgreSQL database receives all migrations through 0402.. - Migration `0402_ancient_malice.sql` preserves the original partial index definition and concurrent idempotent creation. All 400 prior migration receipts stay unchanged during upgrade. One new receipt is added. Replay adds no receipt. The index is ready and valid on fresh and upgraded PostgreSQL databases. The five newest snapshots are retained. - A rollback-only query probe adds 10,000 processed actions and one pending action. The actual parameterized candidate predicate selects the one pending row through the partial index, with two shared buffers and 0.007 ms execution time. This measures candidate discovery, not all lease or provenance reads. - Database build, server typecheck and build, module boundaries, migration safety and migration order pass. Audit: [GitHub review-check scheduling](https://github.com/paperclipai/paperclip/blob/2bba08403b195bf646eda48992b46d6f5cebf756/doc/plans/2026-10-25-github-review-check-sleep-audit.md). - Fresh checks and Greptile review are required for this head. Prior scores do not count. These checks do not prove deployed host suspension or fleet savings. ## Risks - Unresolved queued/running checks still make remote-head requests at the existing five-second deadline. GitHub webhook receipt does not prove future pushes cannot be missed. This PR does not claim active reviews become network-idle. - Run and configuration hints rebuild outstanding checks for the affected company. Enqueue, review, and lease hints narrow to the affected endpoint/repository/PR. Startup and scope reconstruction read existing pending rows; they are not constant-cost operations. - Delivery admission failures remain discoverable through the unresolved check's external reconciliation deadline. Not every shared delivery mutation gains a new immediate signal. - Notifications use the existing process-local database ownership contract. Independent SQL writers and other replicas need their own supported integration. - Provider markers, publication leases, and saved actions preserve recovery after uncertainty. This change does not expand Cloud alarm coverage or prove deployed host/database suspension. - Each remaining sleep slice needs its own review and CI checks before merge. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context-window size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 17:08:27 -05:00
DottaandPaperclip	338bf75761	fix(chat): admit task Retry after GitHub reconnect (#17733) ... ## Thinking Path > - Paperclip manages agents and their tasks. > - Task Retry must preserve saved input and enforce current access. > - A GitHub reconnect invalidates old runtime callbacks. > - Retry reused that old callback authority and rejected valid new intent. > - Retry also replayed an older failed request when a follow-up was already saved. > - This change admits a deliberate Retry under current authority and transfers saved input once. ## Linked Issues or Issue Description Refs #16965, #17053 and #17317. These repairs preserve work and contain old executors. This change fixes the subsequent chat admission boundary. A search found no overlapping open reconnect/admission fix. **What happened?** Ordinary task Retry returned `chat_failed_run_retry_not_authorized` after the same GitHub App reconnected. The endpoint was healthy, but the saved request still carried its old runtime generation. A newer queued follow-up also made Retry reject the older failed request instead of continuing the follow-up. No successor was admitted. HTTP qualification found two related defects. Concurrent clicks could omit the canonical chat action from a response. Signed-in board dispatch also passed an incomplete task resource to the permission resolver, which rejected a permitted mutation. **Expected behavior** A deliberate Retry checks current authority for the same App, sender, repository, conversation and task. It continues saved input once. A later reconnect, changed App, revoked access or changed source prevents dispatch. Old runtime callbacks remain invalid. **Steps to reproduce** 1. Admit a GitHub request and record a failed native run. 2. Save a follow-up while execution is blocked. 3. Reconnect the same GitHub App. 4. Click ordinary task Retry. 5. Before this change, admission returns 409. Without reconnect, the newer input alone also causes 409. **Paperclip version or commit** Baseline source: `41c42895fa862c586ff841f8c50200d93ffc83a4`. **Deployment mode** Source checkout on macOS. Regressions use real PostgreSQL transactions and the assembled chat, HTTP route and heartbeat admission services. Provider boundaries are simulated. No hosted recovery is claimed. ## What Changed - Record the admitting App identity for new GitHub deliveries. Historical deliveries can use their original working-comment binding. Reauthorize deliberate Retry under a pinned current generation and credential-reference fingerprint. - Select the complete saved-input queue from the failed run's actual conversation before replaying the older failed batch. Keep exact-run Retry bounded while ordinary task Retry supports larger queues. Transfer its receipts with the dispatch transaction. Recover the same ownership after restart or a lost response. Release unadmitted input after a definitive rejection. - Preserve canonical chat action IDs across concurrent clicks. Supply current task status and assignment fields to board mutation checks. - Keep exact-run Retry, stale webhook, containment, source integrity, publication, pause and budget checks. Update recovery contracts and verification recipes. ## Verification - Red: all three new PostgreSQL regressions failed on unchanged production code. Reconnect failed at chat authorization. Newer input failed at the obsolete-source guard. The combined case also failed. - Green: 61 selected chat integration cases passed. These include 27 new cases and 34 existing cases for containment, retained-source recovery, exact batches and source revocation. - A further red/green PostgreSQL regression reproduces the 51-message overflow identified during review. All 51 messages reach one successor, including a later Retry of that successor. A separate 52-message consumed-batch regression also passes. - A competing conversation retains its own deferred message. Distinct automatic policy contexts are checked independently; revoking only the later policy blocks the whole unadmitted batch. Batch validation still rejects a later message with a changed App binding or deleted source. A real PostgreSQL query capture reduced the 51-message preparation from 1,103 calls to fewer than 180. - All 16 durable task Retry tests passed. - The real HTTP route returns 202 and admits one queued successor for concurrent deliveries. Fault injection covers rollback during input handoff, response loss after dispatch commit, and permission loss when resuming an existing durable wait. A wait only adopts input after current admission succeeds; a one-shot settlement marker keeps status projection read-only. A new deliberate Retry works after another reconnect or a failed successor. - Current-App, repository, sender, source edit/delete, reassignment, endpoint pause and board-permission controls passed. Historical deliveries remain unchanged. - Final-head CI: 55 checks passed, three expected skips. The unchanged minute-bucket rate-limit assertion passed locally against PostgreSQL and on its one CI rerun. Greptile reviewed the final commit at 5/5 with no unresolved threads. - Workspace typecheck and full build passed. Map alignment, module boundaries, UI token gates and `git diff --check` passed. - Hosted task recovery and a live provider turn remain unverified. The local tests hold agent capacity to inspect admission without starting a model or publishing a GitHub response. ## Risks - A deliberate GitHub Retry can admit old saved input under current connection authority. It requires the same App binding and current access. It pins that admission; it does not rewrite the historical delivery or make stale callbacks valid. - The durable chat queue has an optional admission callback. It transfers input only when the transaction contains a valid queued execution or saved wait. Other chat consumers have no callback and retain their behavior. - No migration, Cloud dependency, deployment, configuration change or workspace cleanup is included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source inspection, code execution and GitHub tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 10:26:45 -05:00
DottaandPaperclip	f6608f382f	refactor(server): isolate GitHub task preparation and review containment (#16231) ... ## Thinking Path > - Paperclip helps people manage AI agents for work. > - Chat channels turn provider messages into shared task workflows. > - The main service still implements provider details beside common authorization and persistence. > - These details belong in the existing channel folders. > - This pull request moves GitHub task prompts, review provenance, and shared guest containment behind explicit typed functions. > - The service keeps its shared workflow and selects the provider implementation directly. ## Linked Issues or Issue Description Refs: #16192. Stack part 2/5. Depends on #16230. Base: `codex/chat-channel-inbound-policies`. **What existing behavior does this improve?** The ownership and readability of provider behavior in the chat channel service. **Current behavior** The service implements GitHub task prompts, review provenance, and shared guest containment inline. **Proposed behavior** Channel modules own these details. The core calls their typed functions. Runtime and database effects retain their current order. **Reason and benefit** Maintainers can find channel behavior in its channel folder without following a generic provider framework. **Breaking changes** None intended. Public service entry points and durable data shapes stay compatible. ## What Changed - Extract GitHub task prompts, review provenance, and shared guest containment into the existing channel folders. - Keep shared company scope, authorization, and transaction ownership in their current workflow. - Reduce `chat-channels.ts` from 3970 to 3906 lines. ## Verification - The same 130 selected GitHub and low-trust integration cases passed before and after extraction. Existing transactions still own containment, authority receipts, and review rows. - Server typecheck passed. - `node scripts/check-module-boundaries.mjs` passed. - Full workspace typecheck and build passed on the cumulative stack. All current-head CI gates passed. Greptile completed at 5/5 with no findings. - The full local `pnpm test:run` command exited with three database fixture-startup hook timeouts. It reported 19,529 passing tests and 250 skipped tests. No assertion failed. Its later test groups did not run because the driver stops on the first failing group. - The timed-out files were `environment-custom-images-service.test.ts`, `generic-mcp-connection.test.ts`, and `modules/wake-queue/adapters/postgres.test.ts`. All 166 tests in these files passed in isolation on both baseline `edcfcaf3f6` and stack head `faaca26d46`, with existing timeouts and assertions. The tests and database-startup helper are unchanged. - Provider fixtures prove application integration. They do not prove live third-party execution. ## Risks - Incorrect inputs or changed await order could affect provider handling and recovery. - Existing integration tests exercise the assembled service before and after extraction. - No database migration or API change is included. ## Model Used OpenAI GPT-6, running in Codex with reasoning, tool use, and code execution. The exact deployment identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-13 20:54:18 -05:00