mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 12:59:13 +01:00
5f0da562dbfdf3229f3cd4bb3dc742ab541986b4
5850
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
5f0da562db |
fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
143d20e466 |
fix(server): retain native Codex Plan write boundaries (#18034)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Codex task chat keeps a saved Plan inside the turn that wrote it. > - The compact conversation API omits ordinary tool history. > - That omission also removes the write receipt needed to place the Plan. > - This pull request retains a small successful write receipt in conversation reads. > - The UI can then keep the saved Plan at its write boundary after readback. ## Linked Issues or Issue Description **What happened?** A real native Codex Product E2E attempt saved its Plan and displayed it inside a settled turn. The Plan used the fallback position. The browser's final conversation-only API response contained no `write_document` receipt. The full event snapshot retained the successful write. The native placement assertion failed and remains failed in the original report. **Expected behavior** Conversation readback must retain enough evidence to place the saved Plan at its successful write boundary. It must keep arbitrary tool bodies and ordinary tool history out of the compact response. **Steps to reproduce** 1. Save a canonical Plan through a native `write_document` call. 2. Read the run with `view=conversation`. 3. Before this fix, the response omits the write receipt and the UI places the Plan at the end-of-turn fallback. Related eval evidence repair: Refs #18026. This is a separate product readback fix. It does not resolve the failed human-input call or the Claude tool-availability failure from those attempts. ## What Changed - Include only successful canonical `plan` document results, matched to a native `write_document` call in the same run, turn, session and source instance. Ordinary document writes stay excluded. - Build the small receipt inside PostgreSQL before its body crosses the database connection. Preserve only fixed event metadata, tool identity, the `plan` key and mutation disposition. Keep the existing payload decoder for ordinary conversation events. - Keep cursor paging, run scope, ordinary-tool exclusion, and full event reads unchanged. - Document the compact read contract in OpenAPI. - Scope this fix to the canonical native dynamic-tool call/result pair used by Codex. The retained Claude sample proves compatibility with that captured shape. It does not qualify every current ACPX Claude producer. Provider-activity-only paths and OpenCode keep the explicit fallback. ## Verification - Real PostgreSQL red: the new boundary test fails because the first selected event is assistant text rather than the saved write. The six existing tests pass. - Real PostgreSQL/API green: 36 tests pass across task cold-open and OpenAPI routes on an isolated Linux current-master checkout. - UI: 104 transcript tests pass, including completed native placement from the body-free receipt. - Review regression red: a later notes write incorrectly enters conversation readback (one failure, six passes). After the fix, the PostgreSQL/API suite passes all 36 tests. - Fixed the new UI fixture's string-to-Date type mismatch found by hosted CI. Original failed CI remains visible; fresh exact-head checks are pending. - Existing token/style gates pass. - Retained native Codex and Claude fixture shapes both pass. Each fixture includes the canonical item pair and parallel provider `tool.execution.*` activity. All identifiers and document bodies in the fixtures are synthetic. This captured-shape test is historical readback compatibility, not fresh current-source Claude acceptance. The Codex canonical pair is emitted by `packages/paperclip-runner/src/drivers/codex/codex-session-server-requests.ts` in the admitted dynamic-tool handler. - Projection-collision regression red: a non-Plan result selected by conversation identity was relabeled as a Plan. The SQL projection now uses the same full Plan predicate as selection. The regression and both native fixtures pass. - The database test checks cursor paging, a Plan followed by notes, failed/read/control-plane/unrelated-turn exclusion, schema and ID bounds, omission of a large tool body, unchanged full-history reads, and ordinary conversation payloads containing NUL. - Local server typecheck fails on 80 installed workspace export mismatches outside changed paths. No changed-path type error or local full build pass is claimed. Clean hosted checks must pass before merge. - Current-head hosted CI and legacy status pass. One failed-check retry followed recorded runner shutdowns and an unrelated native-distribution timeout; that test passed all 33 cases locally. Original failed logs remain retained. - Zero provider calls or paid attempt retries. Original failures and accounting journals remain unchanged. This is not a new live qualification result. ## Risks - Conversation pages include matching Plan results within the existing 1,000-row page limit. The query adds a run-scoped match against canonical tool calls and projects results inside SQL. PostgreSQL still reads the stored JSON to select receipts; private bodies are not transferred to the application. It adds no separate database round trip or background work. - Matching requires the existing canonical call/result identities. Missing or malformed historical receipts keep the explicit fallback. This fixes compact conversation readback; it does not redesign placement in an unrelated full-history multi-document UI flow. - This restores the existing UI write-boundary mechanism. It does not change Plan acceptance, permissions, or completion rules. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7d75ae0569 |
fix(server): recover Teams attempts from saved claim deadlines (#17592)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves a claim before each provider operation. > - A crashed operation can leave that claim unfinished. > - Periodic maintenance scans for abandoned claims even when none exist. > - This PR wakes recovery from committed changes and the saved claim deadline. > - Empty work becomes quiet while unknown effects remain safe to review. ## Linked Issues or Issue Description **What existing behavior does this improve?** Teams abandoned file-transfer recovery. This is slice 27, stacked on consent expiry #17590. **Current behavior** The remaining Teams maintenance callback scans every 30 seconds for expired transport claims. It runs from shared publication maintenance. **Proposed behavior** The Teams owner reconstructs existing claim deadlines at startup and reloads a transfer after its committed update. At expiry it clears the claim and preserves the existing unknown-outcome or conflict state. Normal receipt settlement removes the deadline. **Reason and benefit** Remove the last periodic Teams maintenance scan without resending a possibly delivered card or file. **Breaking changes** No schema or public API change. Explicit diagnostic recovery methods remain compatible. Consent expiry stays with its separate parent worker. ## What Changed - Add a Teams-owned worker for consent_sending, uploading, file_info_sending and conflict rows with a saved attempt expiry. - Notify from the existing versioned update before writes. Preserve transaction ownership and shared uncertain-write reconciliation. - Expose a narrow per-transfer recovery method that uses the existing lock order, integrity checks and atomic projection. - Remove the periodic Teams maintenance callback and its publication-maintenance invocation. - Add real claim, receipt, conflict, backlog, restart, precision, failure, drain and shutdown tests. Wire the owner through existing app admission gates. - Retain the existing signoff cancelled-run selection test and GitHub inline retry-window race assertion. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-attempt-recovery.md` and its linked audit. ## Verification Published head: `24c54399909105984e2a10f5869064984f69c472`, targeting master after consent #17590 merged at `4a89456bb3f9aef87cfe0e922e8b9d1a777a6c7e`. A normal ancestry merge reconciles the landed squash with the previously verified child. The complete 13-path binary patch is identical before and after reconciliation, and all 30 unrelated master paths are preserved. This adds no new worker behavior; the 239-case local evidence below belongs to the verified unchanged patch. All 58 current-head checks are now clean, including the complete required CI run. Greptile is 5/5 at this exact head. The current Ponytail turn is paused without a verdict, which is an explicitly authorized nonblocker; no failing review is waived. The complete 13-path slice preserves the reviewed attempt implementation and inherits the parent's report migration 0397, bounded consent worker and retention CI test repair. No new schema, queue table, shared scheduler contract, workflow or lockfile change is added by this duty. - **239 distinct selected cases pass:** 221 across five complete transfer/publication/consent/runtime/app-wiring suites, 16 selected real Board/Teams-authority/GitHub-retry integration cases, and two delegated Slack policy cases. Other chat integration cases (1,579) and connection-management cases (81) were skipped, not passed. Each database owner uses a new isolated PostgreSQL database. - Real Board publication saves an active attempt deadline; its confirmed receipt retires recovery, and SQL stays quiet afterward. The inherited consent producer and authority cases still pass. Lifecycle coverage exercises claim stages, unknown-outcome quarantine without repeating provider I/O, conflicts, rollback, restart, precision, backlog/fairness, failure recovery, drain and shutdown. The changed GitHub metadata/retry race passes without duplicate assessments. - The old server10 CI failure, run `40557802358` job `122390238707`, was a delegated Slack invitation rate-window assertion (2,122 passed/one failed/15 skipped in that shard). The verified parent already contains its Date-only fixture fix. Both linked and invitation cases now pass their full caller/target/shared-policy scenarios on a fresh server fixture; no duplicate policy change or new owned file was added. That old failure is preserved rather than called a pass. - Server production typecheck, build and module-boundary checks pass. Test files were executed; production typechecking does not statically check them. The existing test-only signoff cancelled-run selection fix is retained but was not executed locally in this acceptance; fresh remote E2E gates remain required. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-25-teams-attempt-recovery-sleep-audit.md#current-consent-parent-preparation-and-acceptance-october-28) records current scope, commands, counts and limits. Earlier 235-case and combined-stack counts belong to their stated revisions and are not added to 239. The [stack ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository suite, live Teams/Slack tenant, external wake, physical suspension or savings claim is made. - **Exact-head Greptile 5/5 and all 58 checks are clean.** Actual failing Ponytail findings must be repaired; a missing/stopped/paused turn without a verdict is an authorized nonblocker. An active no-verdict turn waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older reviews and parent checks are not treated as current passes. ## Risks - Recovery must not infer that an expired claim means a provider effect did not happen. It retains unknown outcomes and does not resend. - Late receipts must still match current claim identity. Current-state rechecks and existing protocol tests preserve this fence. - Supported deployment uses one active writer process through its owning DB client. Independent SQL and additional processes need a separate wake contract. - Cloud external alarm coverage and sleep eligibility are unchanged. Other chat duties still run periodically until their own migrations. - Voice-report parent review repairs are propagated: explicit single-active-writer scope and an indexed report window. Thirty report-worker/startup tests pass on the combined child. Fresh remote checks, older stack findings and final master reconciliation remain required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
SudheerandClaude Opus 5.5
|
2e5bbde8cc |
fix(ui): remove duplicate audit links from agent overview (#17867)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The agent detail page shows an overview of one agent in the board UI > - The streamlined shell shows the agent contextual sidebar beside this page > - The sidebar has an Audit group with Activity, Runs, Costs, and Budgets links > - The overview also ends with an Audit card grid that has the same four links > - The duplicate links add noise and make the overview longer with no benefit > - This pull request removes the Audit card grid from the agent overview > - The benefit is a shorter overview with one clear place for audit navigation ## Linked Issues or Issue Description **What existing behavior does this improve?** The agent overview page shows a section of Audit link cards at the end of the page. **Current behavior** The overview shows four cards: Activity, Runs, Costs, and Budgets. Each card opens the company audit page, filtered to the agent. The agent contextual sidebar shows the same four links. Both use `agentScopedAuditHref`. #12746 added the sidebar links. #12747 added the overview cards in the same PR stack. **Proposed behavior** Remove the overview cards. Keep the sidebar Audit group as the single entry point. **Reason and benefit** Operators see each audit link one time. The overview stays focused on agent status and recent tasks. ## What Changed - Remove the Audit card grid from the agent overview in `ui/src/pages/AgentDetail.tsx`. Before: <img width="1488" height="377" alt="Screenshot 2026-10-27 at 5 33 29 PM" src="https://github.com/user-attachments/assets/fd27fee2-63c4-420e-8b39-405e4a633f63" /> After: <img width="1526" height="271" alt="Screenshot 2026-10-27 at 5 33 41 PM" src="https://github.com/user-attachments/assets/bca0635d-3be6-42cd-b1d6-dc36b89ae1b0" /> ## Verification - Run `pnpm --filter ./ui exec tsc --noEmit`. The result is clean. - Run `pnpm --filter ./ui exec vitest run src/pages/AgentDetail`. All 7 files and 56 tests pass. - Run `pnpm check:token-gates`. All gates are clean. - Manual check: opened an agent overview page. Made sure that the Audit cards are not at the end of the page. Made sure that the sidebar Audit links still open the agent-filtered audit pages. ## Risks - Low risk. The change only removes UI markup. - The agent-scoped audit routes and the legacy URL redirects do not change. - Operators who used the overview cards must use the sidebar Audit links. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code. The model used tool use, code search, git history inspection, and local test execution. Reasoning mode: adaptive thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
4a89456bb3 |
fix(server): schedule Teams consent expiry from saved deadlines (#17590)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves consent and transfer state in the database. > - Unused consent must expire even when no new message arrives. > - The maintenance loop currently checks this state every 30 seconds. > - This PR schedules each existing consent deadline and wakes on committed changes. > - Empty or settled consent work stops querying while expiry remains recoverable. ## Linked Issues or Issue Description **What existing behavior does this improve?** Teams unused file-consent expiry. This is slice 26 of the recurring-server migration stack, following merged #17586 and targeting master. **Current behavior** The Teams maintenance callback scans for expired unused consent and abandoned transport attempts together, even when consent work is empty. **Proposed behavior** The Teams owner reconstructs unused consent at startup and reloads one transfer after a committed write. The shared scheduler runs its saved expiry. Abandoned attempts keep their current recovery until a separate slice migrates them. **Reason and benefit** Remove unnecessary consent queries without losing expiry, atomic publication projection, or recovery after restart. **Breaking changes** No schema or API change. The explicit combined recovery method remains available. Cloud sleep eligibility and external alarms remain unchanged. ## What Changed - Add a Teams-owned scoped deadline worker for the three unused consent phases. Registered reconstruction yields after one 100-row page per scope refresh and resets cursors on registration; the explicit diagnostic inventory still returns every eligible row. - Signal before insertion and the existing versioned update inside the owning transaction. Nested writes wait for outer settlement. - Extract the existing locked per-transfer recovery operation. Preserve authority checks, atomic projection, bounded lock waits and unknown-outcome quarantine. - Remove unused consent from periodic maintenance selection and execution. Keep abandoned attempts as a separate obligation. - Register the worker through existing app lifecycle gates. Add lifecycle, producer, production wiring and precision regressions. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-consent-expiry.md` and its linked audit. ## Verification Published head: `9e84cfab1f795279c8cec413a946631f1d781bd2`. This follow-up changes only the inherited voice-retention test and its CI failure ledger; all production behavior is unchanged from `36f43922`. It normally merges actual landed master/report parent `8049fbf51d0701ca6b242e4b1198e5449f7661b2`. The verified 12-path Teams patch was byte-identical across the actual-parent ancestry integration. This test-only CI follow-up adds one existing retention test path, bringing the final scope to 13 paths. Every unrelated master change, report documentation fix, and migration 0397 is preserved. No database, shared-worker, workflow or lockfile change belongs to this slice. - **223 distinct selected cases pass:** 209 across five complete transfer/publication/consent/runtime/app-wiring suites, plus 14 selected real Board-producer and Teams-authority integration cases. The other 1,580 integration cases were skipped, not passed. The protocol and production consumers use separate new PostgreSQL databases migrated through the application migrator. - A real 105-transfer regression failed against the original worker because one registered startup pass selected 105 rows instead of yielding at 100. The corrected worker uses the existing paged shared contract. Restarting the same service midway through discovery selects the first 100 again, restores all 105 saved deadlines and expires them at the original deadline. SQL is quiet before the deadline and after settlement, with no consent-card or upload requests. The one-shot inventory still returns all 105 rows. - A guarded mutation removing only cursor reset fails at five rows instead of 100 after restart. The original production source was restored before the complete suites. The initial regression fixture incorrectly faked network timers before its first SQL connection and timed out; that setup failure is retained separately from the corrected 105-versus-100 regression. No assertion or timeout was weakened. - The real Board producer registers the worker before publication, commits the transfer wake, preserves the saved expiry, remains SQL-quiet until due and atomically projects expiry to a cancelled publication without another provider request. Existing authority and consent transitions remain covered. - Complete lifecycle coverage retains all three consent phases, precision rounding, rollback and held outer commit, exact-row hints, expiry recheck, failed projection retry, corrupt-row fairness, drain/resume, shutdown and separation from abandoned-attempt maintenance. Production server typecheck, build and module boundaries pass. Server production typechecking does not statically check test files; those files were executed. - [Current bounded reconstruction and failure ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#bounded-reconstruction-and-current-parent-acceptance-october-28) records exact scope, counts, guarded mutation, corrected timer setup, earlier fixture failures and limitations. Earlier 222-case and combined-stack results belong to their stated historical heads; they are not added to 223 or claimed rerun here. The [stack ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository-suite, live Teams tenant, Cloud wake, host/database suspension or savings claim is made. - **Fresh exact-head Greptile 5/5 and all CI are required.** Actual failing Ponytail findings must be repaired. A missing/stopped/paused turn without a verdict is an authorized nonblocker; an active no-verdict review waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older scores and parent checks are not accepted as current passes. - **Preserved inherited CI failure:** prior head `36f43922` had exact-head Greptile/Ponytail 5/5 but CI run `40708566195` server6 job `122887395220` failed the retention fairness assertion (2,291 passed/one failed/one skipped in that shard). A fixed final UUID reproduces the wrong expectation that an undiscovered page-two row is already deleted. The corrected test controls time, expects that row to remain before discovery and requires its deletion in the next fair batch, then all remaining history and idle SQL. The existing five-second wait budget and global timeout are unchanged. All 13 retention lifecycle cases pass; bypassing company rotation fails the corrected assertion, and the original shared helper is restored byte-for-byte. Earlier 223-case Teams acceptance remains valid for unchanged Teams source and is not relabeled as a new complete repository run. [Full CI repair ledger](https://github.com/paperclipai/paperclip/blob/9e84cfab1f795279c8cec413a946631f1d781bd2/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#inherited-retention-ci-fairness-repair-october-28). Fresh CI and exact-head reviews are required for this test-only follow-up; old scores are stale. - **Current merge gates:** all 58 checks are now clean for this exact head and Greptile is 5/5 with no actionable findings. The current Ponytail turn is paused without a verdict, an authorized nonblocking state. Workflow `40712464720` attempt 1 independently hit npm `ECONNRESET` during the serialized8 job's tool self-installer, before its tests ran. Exactly one targeted job retry was requested after the original workflow finished; attempt 2 completed successfully on the unchanged head. No code, dependencies, workflow, timeout or review timer changed for that retry. The original setup failure is retained in its [job log](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122901243926); the successful replacement is [job 122910350074](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122910350074). ## Risks - Missed supported producer signals could delay expiry until restart. Real insertion, cancellation, consent transitions, caller-owned transactions and Board publication are covered. - Corrupt or locked rows must retain retry without starving healthy work. Paging, fairness, atomic rollback, redaction, drain/resume and shutdown tests cover these paths. - Independent SQL writers and other processes need their own notification contract. No persistent DB listener is added. - Attempt recovery still scans periodically until the next slice. A local deadline is not a durable Cloud alarm. This PR does not claim whole-instance sleep. - This slice is reconciled with its landed parent; current-head remote checks/review and live-master composition proof still gate merge. Later stack duties remain separate PRs. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.11 |
||
|
DottaandPaperclip
|
820c458465 |
fix(runner): restore per-turn controller checkpoints before resume (#18018)
## Thinking Path > - Paperclip coordinates agents and preserves their work between turns. > - Remote native sessions save controller checkpoints in the database. > - External persistence leaves a small identity marker on local disk. > - A later `per_turn` continuation must restore that checkpoint before it reads the controller identity. > - The restore path handled warm sessions only, so valid per-turn continuations failed before provider execution. > - This PR restores the referenced checkpoint and retains the existing ownership and lifecycle checks. ## Linked Issues or Issue Description Related: #17439 added external controller persistence and the warm restore path. #18011 repairs warm preparation and fresh bootstrap checks; it does not cover ordinary per-turn continuation. **What happened?** A remote OpenCode run completed with `per_turn` lifecycle and saved its controller checkpoint in the database. Its next turn reused the normalized session. Admission read the remaining external-persistence marker as a full controller file and failed with `runner_state_identity_mismatch: durable_identity_unreadable`. The prior run's complete checkpoint and harness backup still existed. The failure occurred before provider work started. **Expected behavior** Restore the exact prior controller checkpoint before identity inspection. Then apply the existing company, task, session, prior-owner and suspended-state checks before a provider can start. **Steps to reproduce** 1. Complete a remote native OpenCode task with `per_turn` lifecycle and external controller persistence. 2. Send a follow-up that resumes the same saved session without a session reset. 3. Observe admission reject the valid external controller marker before provider execution. **Paperclip version or commit** Observed at `40c1ab36cdac70c85dc8fe5e912431834fdb691b`. The same warm-only restore remains on current master. **Deployment mode** Hosted Linux with Daytona and native OpenCode. The original model was `openrouter/deepseek/deepseek-v4.1-flash-1017`. ## What Changed - Restore external controller state for ordinary remote per-turn continuations through the existing checkpoint loader. - Bind that read to the current company and task, the marker's run, and the expected normalized session. - Keep local, warm and restart-recovery paths, prior-owner checks and harness backup checks intact. - Refresh a prior-run projection from the latest checkpoint after a rejected active-owner attempt. Keep current-run full state and the existing legacy file-backed fallback intact. - Add restoration and denial regressions, execution documentation and session verification recipes. ## Verification - Red: the real controller generated an external marker; a valid per-turn OpenCode continuation failed with the exact `durable_identity_unreadable` error before tool-authority construction. The repair passed the same case and its ownership guards. - Review regression red/green: a real controller advanced its checkpoint after an active-owner rejection. Before the follow-up fix, a later valid continuation retained a null attach seed. After the fix, it restored the newer attach template and prepare command. Legacy full-file roots with no database checkpoint retain their existing behavior. - At current head `1f6589233e6121d082a0c0c4444ea895b6e8cd7c`, 86 focused executor cases and all 14 execution-driver cases pass. Coverage includes checkpoint restoration, direct/backup suspension, Codex, legacy files, warm admission and canonical bootstrap, missing or malformed evidence, wrong bindings, active/foreign owners and unsuspended runners. Tests stop before provider launch. - The real PostgreSQL encrypted-checkpoint integration passes, including normalized-session refusal before write and stale-writer fencing. Independent source review found no actionable safety defect. - `node scripts/check-maps.mjs`, `git diff --check`, full `pnpm -r typecheck` and `pnpm build` pass on current head. - The local full suite did not complete: native PostgreSQL startup failed under the host sandbox, an initial Docker rerun lacked migrations, and the migrated retry crossed the review-fix revision change. Logs are retained; that mixed-revision run was stopped and its owned database removed. Full test evidence comes from current-head CI. - Current-head CI passes: 53 successful checks, two intended Storybook skips, none pending or failing. Greptile is 5/5 on that exact head with no open review threads. The only manual merge conflict with #18011 was in the verification recipes; both recipes and the unchanged per-turn restore helper are preserved. **Live continuation acceptance remains unverified, so this PR stays draft.** Two disposable OpenCode/Daytona attempts never reached turns two and three: - At `72b49208b1`, the first turn executed tools and produced the exact 33-byte workspace note, but exceeded its four-minute settlement deadline. The first completion lacked a registered attachment; the second decorated a typed attachment reference with filename/size/hash text. The strict UUID parser rejected that reference before checking a valid criterion reference. The scoped attachment, asset and publication receipt exist. Replaying the actual guard rejects the original input before a database lookup; correcting only that reference passes against the recovered row and receipt. This did not show a publication-persistence or controller defect. The task remained unsettled. - At `1f6589233e`, a separately approved replacement used a temporary prompt-only fixture change requiring exact `deliverable:<attachmentId>` references, with metadata in `verification.detail`. Production code, grading and deadlines were unchanged; the fixture patch was removed afterward. This attempt had no recorded provider authentication, harness-ready, model, tool or result event. Ingress acquisition took 84.072 seconds; transport connection took 205.835 seconds within the 239.832-second failed native execution. A local settlement GET also timed out after 30 seconds. The underlying startup stall is unresolved; these observations do not establish a remote provider outage. Final recovery was blocked with `ambiguous_remote_recovery_evidence`; no successful settlement or continuation is claimed. Both attempts used a Darwin arm64 source host with Node 26.4.0, isolated PostgreSQL 18, a Linux x64 runner and OpenCode 1.18.34. The representative qualified model was `openrouter/deepseek/deepseek-v4-flash-0731`; the original incident used `openrouter/deepseek/deepseek-v4.1-flash-1017`. The controller defect occurs before provider work. Runtime artifacts came from immutable build `e7ea2e2961`, whose Runner package source is identical to the candidate. The current-source manifest, artifact, distribution, asset and all 110 archive-hash checks passed. Each attempt had $10 company/agent model budgets and zero automatic workflow/task retries. Accounting was incomplete; zero recorded spend is not proof of zero cost. Earlier setup failures are also retained: host PostgreSQL bootstrap denial, cached-pack compatibility admission before provider work, and an [official artifact build](https://github.com/paperclipai/paperclip/actions/runs/40696838122) that stopped during an unrelated Pi npm install at its 300-second bound. No manifest or integrity guard was weakened. Owned local processes and databases are stopped, and the temporary fixture patch is removed. Standard fixture cleanup had left one replacement sandbox started; it was stopped and deleted through the SDK. Every owned sandbox was then verified absent with a final 404 readback. Resource cleanup does not establish successful task finalization. An earlier head retained one [voice-recovery timing failure](https://github.com/paperclipai/paperclip/actions/runs/40695892339/job/122843005098): a deadline test observed two failed-provider inspections instead of one. A single isolated shard rerun passed all 1,579 tests without a source change. Its timing explanation remains an inference. The current reconciled head passed without a retry. ## Risks Restoration writes a local projection before the existing admission checks. The encrypted checkpoint loader validates company/task/run and normalized session before writing. A later ordinary continuation refreshes a prior-run projection so rejected admission cannot freeze an active owner's older checkpoint. Active, foreign or unsuspended prior owners still cannot start a successor. Missing or malformed evidence remains blocked. The change adds no retry, credential, provider binary or lifecycle-policy change; it preserves the existing legacy fallback. This PR does not deploy the repair or recover the original hosted tasks. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. The session exposes GPT-6 as its model version; a more specific model ID and context-window size are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d8bd8fe9fb |
feat(evals): browse E2E suites across targets and models (#18027)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - E2E tests show whether people can complete real app and agent journeys. > - Operators need to choose a suite and inspect its evidence across deployments and models. > - The history page listed campaigns without a suite browser or clear coverage filters. > - QA smoke also appeared separate from the other retained E2E measurements. > - This pull request puts suites, deployment targets, and named selections in one history browser. > - The benefit is that a passing subset stays distinct from a full suite run, with direct links to evidence. ## Linked Issues or Issue Description Related: #18009 added clear per-test details and links to the tested application. This PR extends the shared E2E history browser. **What happened?** The public directory called full app testing “Product E2E Evals” and made users choose campaigns before they could find a suite. The history list could not filter independently by tested app, agent execution, model, or run selection. A recent passing subset could be mistaken for a full suite result. **Expected behavior** Browse current and retained suites in one E2E category. Keep QA smoke in normal campaign history. Show the latest matching run and latest full suite run. Preserve the recorded definition and denominator when filters change. **Steps to reproduce** 1. Open the E2E history directory. 2. Find a suite, then compare a narrow passing run with a full run across execution environments and models. 3. Find the QA smoke measurement and open its retained screenshots. **Paperclip version or commit** The previous report UI was at `8279c99008`. This branch is rebased on current master. **Deployment mode** Static E2E history and campaign reports generated from retained measurements. ## What Changed - Call full-product testing “E2E tests.” Keep Runner protocol evals separate because they use a mock control plane. - Add a searchable suite sidebar that includes current suites and suites found only in retained history. - Add correlated suite, coverage, deployment target, execution, provider, model, profile, runtime, result, date, and named-selection filters. Keep filters in the URL for reload and back navigation. Sidebar suite links clear other filters; the Suite dropdown keeps deliberate intersections. Incompatible suite/preset URLs explain the empty selection and link to the suite runs. - Show latest matching and latest full run links. Distinguish full, subset, unfinished, and unknown coverage from the recorded suite definition and execution counts. Keep failed full runs visible. - Retain the tested application URL and runtime mode in new history records. Older records show “Not recorded.” Browser-only journeys do not gain an inferred agent or model. - Add an index-only refresh command that can import a reviewed public campaign once. It rejects private Cloud bundles and does not replace retained campaigns or upload evidence. - Keep existing report URLs, test IDs, commands, and measurement timestamps stable. Campaign links carry suite/profile/environment; target/model/provider/runtime remain history-level filters. - Add unit and synthetic browser regressions, including mobile navigation, long source refs, sidebar filter reset, and empty-selection repair. Inset dropdown carets in the shared report styles. Document direct Node entry points without adding package.json shortcuts. - Add a prominent Reset all filters button, blue active-filter borders and tint, and visible active counts for the main and advanced filters. Reset clears all 13 history filters, sidebar search, URL state, and the attention view. Campaign-page filters use the same active styling. ## Verification - Post-rebase focused Vitest: 127 tests passed across 10 files. The latest follow-up selection passed 100 tests across 9 files. E2E support TypeScript passed after both changes. - The synthetic Chromium journey covers full and subset coverage, unfinished selections, correlated filters, browser-only QA selection, URL/back/reload, and desktop and touch navigation. The expanded journey verifies 13 active history fields, the collapsed advanced-filter count, reset of every field and sidebar search, and active border color changes that revert after reset. It makes zero provider calls. - Hub Python tests: 7 passed. The browser harness also exits promptly when Chromium is unavailable; its HTTP server closes on launch failure. - Real history verification covered 419 campaigns and 59 suites. All 418 previous campaign records remained deeply equal after importing the reviewed QA campaign. Publication used the authoritative history ETag. Only mutable index/history/pointer files were written; immutable campaign bundles stayed unchanged. - A live browser walkthrough passed from the two-family eval hub through QA smoke, Basic app journeys, target filtering, the retained report, screenshot gallery, history, and the logo link. The QA URL was clickable and the image loaded. No browser JavaScript errors occurred. The original QA normalized results and all three PNG hashes remained unchanged. - Live [E2E history](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history), [core compatibility suite](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history&suite=core-compatibility), and [QA smoke runs](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history&preset=qa-smoke). - The live follow-up verified the incompatible Stock harness/QA-smoke URL, the repair link to 21 Stock harness campaigns, and sidebar navigation that clears preset/model filters. Dropdown carets were inspected in dark, light, and mobile views. No JavaScript errors occurred. - The final live follow-up verified reset of history controls, URL state, sidebar search, and the attention view, plus reload/back restoration and mobile touch reset. Active filter borders and tint were inspected in dark and light views. Campaign-page reset restored all three QA cases. All 419 history records and original QA measurement/evidence hashes stayed unchanged. - Full repository typecheck, test, build, and E2E CI gates passed on final head `1334cb79a5`. Both enforced aggregates, `ci / verify` and `ci / e2e`, completed successfully. Fresh Greptile review scored that exact head 5/5 with no unresolved review threads. ## Risks - Historical metadata can be missing. Unknown targets and runtime details stay explicit instead of being inferred from a worker environment. - Full suite coverage follows the saved definition and selection. Filtering to one model does not redefine that denominator. The latest full-run shortcut can cross coverage selections and says so in its detail text. - This changes shared report presentation. It adds no application runtime or database changes. The browser harness and index refresh do not invoke models. - Public history publication must keep its existing conditional-write and retained-evidence boundaries. Private bundles, raw logs, credentials, and screenshots are not part of this PR. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
b25f3e2af8 |
fix(onboarding): show native proposal confirmation recipes (#18025)
## Thinking Path > - Paperclip lets people manage agent work through tasks. > - First-task onboarding asks a new agent to propose work before it starts. > - The user must accept a saved proposal before the agent can create its child task. > - A native agent can ask for approval in a question card, which does not record proposal acceptance. > - The skill names confirmation cards but does not show their native tool arguments. > - This change adds executable native proposal recipes and checks their real approval path. > - The task admission rules and Product E2E assertions stay in place. ## Linked Issues or Issue Description **What happened?** A fresh onboarding agent asked the user to approve a child task through an Approve/Revise question. The question looked like a decision, but its answer cannot authorize child creation. The first-task skill teaches native question calls for interviews and names proposal confirmation cards without showing the native mapping. **Expected behavior** The agent should create a confirmation card for a single-task proposal, or a checkbox card bound to the saved plan revision. Only the saved human acceptance should admit the child task. **Steps to reproduce** 1. Start first-task onboarding with a native agent. 2. Choose one task and ask the agent to propose a child before it starts work. 3. Have the agent use a question card with Approve/Revise options. 4. Answer Approve. Child creation remains blocked because no proposal confirmation was accepted. Related public context: #17940, #13517. ## What Changed - Show the native `request_human_input` arguments for confirmation and plan-bound checkbox proposals. - Explain that question answers supply scope and cannot grant proposal authority. - Resume the agent after either acceptance or rejection, and use new proposal keys for revisions. - Parse the shipped recipes and execute them through the real native tool authority and interaction service. - Verify that pending proposals, agent-only acceptance, and an answered Approve/Revise question cannot admit child work. - Use the existing isolated PostgreSQL test helper when a local test database is supplied. - Fix the required clean Grok npm-install check to prefix synthetic revision identifiers with `g`. The former all-numeric, leading-zero identifier was invalid SemVer and caused a reproduced `ETARGET` failure. Customer release versions are unchanged. ## Verification - Red: both new executable recipe checks fail against the preceding skill. - Green: 44 focused tests pass with zero skips against a task-owned PostgreSQL server. The actual tool authority and approval service are exercised. - Human acceptance of either recipe admits the child. Agent-only acceptance does not. Answering the question leaves the issue set unchanged. - Red/green real HTTP checks prove rejection saves the assignee continuation, starts no extra model run, and still denies child creation. - `git diff --check` passes. - Full repository `pnpm -r typecheck` and `pnpm build` passed on the final approval-guidance revision. - The first full local test attempt used an unmigrated external fixture and was stopped after the cause was confirmed. A new migrated fixture resolved that setup error; the second full attempt reached its explicit 10-minute limit before finishing. Both attempts are preserved and are not claimed as passing. The temporary owned PostgreSQL container was removed. The complete Linux CI workflow passed on the final PR head. - The exact synthetic-version failure was reproduced with real npm and an empty local packed registry. The new maintained regression keeps the invalid-version negative control and installs corrected numeric-hash and normal-hash controls. All 27 existing Grok helper tests and three offline registry-boundary checks pass. - Final head `52d0b92a00dcb0653b6fb48415cfaea979b8f21f`: [CI 40706933035, attempt 1](https://github.com/paperclipai/paperclip/actions/runs/40706933035) passed; 55 successful checks, two skipped and one neutral. Greptile is 5/5 for this exact head with no unresolved threads. The original failed build remains preserved. - Live fresh-workspace acceptance has not passed yet. It follows the reviewed testing rollout; this source test does not prove model compliance. ## Risks The change guides model tool selection. An agent can still ignore the skill. Product admission remains the authority, and the live journey must still verify human approval, native child execution, downloaded artifact bytes, context reuse, and cleanup. There is no schema or API change. The CI helper changes only temporary verifier versions and preserves the existing registry integrity, install, lifecycle, and isolation checks. ## Model Used OpenAI Codex `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
8049fbf51d |
fix(server): schedule voice call reports from saved deadlines (#17586)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain reports and project guest transcripts into intake tasks. > - Chat maintenance repeatedly scans recent calls even when no report is due. > - Empty scans keep the tenant database active. > - Existing session and report rows already contain the work and its deadlines. > - This pull request moves report refresh to the shared coordinator and removes the final voice maintenance lane. > - Idle voice configuration and old call history no longer cause report scans. ## Linked Issues or Issue Description Refs #17583, #17580, #17578, #17576. This is slice 25 of the recurring-server sleep stack. It now follows merged #17583 and targets master. **What existing behavior does this improve?** Voice call-report refresh and guest transcript projection. **Current behavior** The chat interval scans terminal calls for reports. It repeats the scan even when there are no eligible calls or all saved deadlines are in the future. **Proposed behavior** Committed session transitions and signed callbacks wake a scoped worker. Startup reconstructs existing rows. Later refreshes use saved deadlines. The worker becomes quiet after all eligible windows expire. **Reason and benefit** Stop empty database scans without losing late transcript and billing updates. Keep the existing 24-hour refresh policy, provider response ordering, and guest-task trust restrictions. ## What Changed - Add a voice-owned report worker using the existing scoped deadline helper. No queue table or field is added; the existing session table receives a report-window index. - Save each attempt and its retry before provider I/O. Compare that ownership on settlement so an older response cannot overwrite a later callback or claim. - Preserve available reports when a response is incomplete or stale. Preserve their finite refresh window. - Route voice-change and reply call-ended writes through transaction-aware report signals. - Preserve guest transcript locks and use the observed document revision when updating an existing transcript. - Register the worker in app startup. Remove voice reconciliation from the chat interval. Keep explicit one-shot diagnostics. - Document producer scope, lifecycle behavior, tests, and external wake limits. ## Verification Published head: `758cfe15c3e6283d38ba512c8416ccce4d200862`. The follow-up after `c28de398` changes only two documentation files; the verified production, migration and test source is unchanged. It normally merges landed retention/master `980775e0e956eecdc28c73c9b4aca0407ad45e1a`. The 20 owned paths remain the complete slice, including officially regenerated migration 0397, its snapshot and required pruning of snapshot 0392. The non-database reviewed patch was preserved during ancestry integration; the only subsequent source edit makes one existing report test’s provider responses call-specific. All production report behavior and unrelated master changes are preserved. No shared worker, workflow or lockfile change belongs to reports. All 58 current-head checks are now clean, including the previously failing approval shard; exact-head Greptile is 5/5 with no new actionable issues. Ponytail’s current turn stopped without a verdict, an authorized nonblocking state. The prior conditional documentation requirement was fixed; old CI failures remain recorded below. - **Prior-head evidence at `9bf20467df`: 303 distinct targeted cases passed**, not rerun wholesale for this integration: all 132 voice integration cases; 170 cases across 11 complete report, retention, cleanup, recovery, company-removal, shared-worker and startup/hostname suites; and one complete snapshot/schema drift case. This includes 15 report, 13 retention and 14 cleanup lifecycle cases. Voice integration and the four other database owners use separate initially empty databases and migrate themselves. Report cases use their dedicated database after the setup repair. - Five real connection saves over 205 sessions with saved future report deadlines selected 1,025 session rows before the notification repair. They now select zero and preserve the original deadline. The report notification helper accepts only exact session scopes. Configuration, authority and connection hints retain their existing live recovery/reply/cleanup behavior but do not rebuild report history. Real session terminal writes, callback hints, report claims and receipts still reactivate reports. - Database/server typechecks and builds, migration numbering/safety, and module-boundary checks pass. Final server types/build also pass after integrating the retention review repairs. The parent 205-admission, five-configuration-save test still observes zero retained-history rows and preserves its deadline. - The old eager reader selected all 105 saved-report headers before yielding and failed the 100-row budget. Registered reconstruction now uses the existing paged contract and yields at 100 rows per scope refresh. A partial scan can restart on the same owner and reconstruct every saved deadline. At due time all 105 report IDs are fetched exactly once. Future deadlines remain quiet between actual retries. - The company-fairness case places a small company deterministically on page two and requires it in the next 25-work batch after discovery, with all 105 reports processed before the failed provider's retry. Guarded mutations catch a missing cursor reset and bypassed company rotation. Original source was restored before final acceptance. - At prior head `9bf20467df`, migration 0396 was generated with `pnpm db:generate` on the parent 0395 lineage. It creates the partial `(ended_at, id)` report-window index concurrently and replayably through the existing migration runner. All 394 historical SQL files and the journal prefix are unchanged; snapshots 0392 through 0396 remain. Fresh application migration and two replay invocations succeed with a valid index. The real plan case retains 20,001 old sessions, 205 recent sessions, precision-safe keysets, and fewer than 1,000 rows removed by filtering on each page. - Existing real hangup/callback/reply-error producers, callback ownership fencing, monotonic report results, saved retry before provider contact, guest transcript projection after a document lock, transaction rollback/nesting, drain and shutdown remain covered. No table, field or provider protocol is added. Available reports retain the existing finite 24-hour refresh policy. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/9bf20467dfd50b81848e1c8dd7ddc905d206418f/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#current-parent-reconstruction-and-migration-acceptance-october-28) records the failed fresh fixture, eager read, fairness-clock attempts, guards, commands, counts and limits. Older 174/138-case and workspace-wide results belong to their stated heads. No new complete local repository suite, live provider, Cloud wake, host/database suspension or savings claim is made. - [Report configuration-scope repair](https://github.com/paperclipai/paperclip/blob/9bf20467dfd50b81848e1c8dd7ddc905d206418f/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#configuration-scope-review-repair-october-28) records the current 303 cases and measured rescan repair. The prior `80207ba6bb` head had Ponytail 5/5; that review is stale after this change. Fresh exact-head Greptile 5/5 and all CI remain required. Actual failing Ponytail findings must be fixed; a stopped or paused turn without a verdict is an authorized nonblocker. Active no-verdict review waits are capped at 30 minutes from the original request/start (25-minute monitor cutoff). Older scores and parent checks are not inferred as current passes. - **Current-master verification: 185 distinct cases pass**: final complete voice integration 132, complete report/cleanup/retention 15+24+13, and snapshot/schema drift 1. Each database owner used its own empty fixture database. The first combined run had 183 passing cases and one report-test failure; it is preserved in the audit. A deliberately earlier-ended competing call reproduces the old one-shot mock consuming the intended response. The corrected test selects responses by provider session ID, verifies independent costs for both calls, then rejects stale target data. The isolated repair and full 132-case suite pass; production behavior and timeouts are unchanged. These 185 cases are not added to the older 303 count. - **Current migration lineage:** official `pnpm db:generate` after actual master creates `0397_known_madrox.sql`; the reviewed concurrent/idempotent report-index SQL is retained byte-for-byte. All 395 existing SQL files and journal entries are unchanged; snapshots 0393–0397 remain and the new snapshot links parent 0396. The application migrator boots a fresh database, repairs a deliberately invalid index during two concurrent replay invocations, records one history row, and replays again successfully. Database build, server build/typecheck, migration checks and module boundaries pass. [Full current-master acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/c28de398fef074f650eff532952ab6f5563739a8/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#landed-master-migration-and-acceptance-october-28). - **Review follow-up and preserved CI timeout:** Ponytail’s prior `c28de398` review was 4/5 advisory, with no must-fix production defect and a request to fix the stale design-doc migration reference before shipping. That document now names migration 0397 and links current 185-case acceptance; older 302/303-case evidence is explicitly historical. The optional suggestions about one-shot diagnostic inventory cost and inherited test-clock helpers remain disclosed. CI run `40703490340` failed the unchanged approval-queue backoff test on its existing 15-second timeout and then an activity-log FK in teardown (2,097 pass/one fail/one skip in that shard). The complete unchanged 28-case approval suite passes on a new isolated database; the exact CI timing cause remains unproven. No timeout, workflow or production logic was changed, and no old-head rerun or failure waiver is claimed. This documentation head requires fresh CI and exact-head Greptile. [Review and timeout ledger](https://github.com/paperclipai/paperclip/blob/758cfe15c3e6283d38ba512c8416ccce4d200862/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#review-documentation-and-preserved-remote-timeout-october-28). ## Risks - Process-local commit signals do not observe independent SQL writers or another server process. - Available reports still refresh for 24 hours. The provider's complete flag does not promise immutable final data. - Transcript projection retries remain within the existing report window. This change does not add indefinite projection repair. - One-shot diagnostic `reconcile(limit)` inventories the eligible window before slicing; it remains a manual diagnostic cost, not recurring idle work. Some inherited test helpers advance fake time during assertion retries, an optional reliability follow-up noted by review. - A local deadline cannot wake a stopped host. Cloud alarms and sleep eligibility remain unchanged. No deployed sleep or cost saving is claimed. - This PR depends on the preceding stack. Final master reconciliation and remote review gates remain required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code editing, and local test execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8fe5d5defc |
fix(evals): honor the shared feedback read deadline (#18026)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E evaluations must retain feedback from each paid attempt. > - The collector reads saved feedback and native tool events before teardown. > - A five-second request limit can stop a valid read before the shared read deadline. > - This pull request gives event reads the remaining shared read budget. > - The total deadline and reserved checkpoint time stay the same. ## Linked Issues or Issue Description **What happened?** A paid Ask attempt passed the product grade but retained partial feedback invocation coverage. The collector recorded `events_read_failed`. Its event endpoint later returned HTTP 200. The retained timing is consistent with a request that exceeded the five-second request limit. This change fixes that deterministic failure path; it does not prove the cause of every historical partial read. **Expected behavior** A valid read that takes eight seconds can complete within the 30-second collection deadline. A stalled read must still time out and leave time to save the available feedback. No read or paid attempt is retried. **Steps to reproduce** 1. Return a terminal native run and an empty saved-feedback list. 2. Make the event read take eight seconds and respect the supplied request timeout. 3. Run `collectAttemptFeedback` with a 30-second deadline. 4. Before this fix, the read times out after five seconds and invocation coverage stays partial. Related accounting and reporting work: Refs #18010. ## What Changed - Use the remaining shared read budget for event API requests. Keep the five-second cap for other reads. - Add regressions for an eight-second event response, a stalled event response that preserves saved feedback, and a stalled early feedback read that still permits the event scan. - Document the read budget, reserved persistence time, and no-retry behavior. ## Verification - Red: the initial two regression tests fail before the fix. The other 42 feedback tests pass. The early-read starvation review regression then fails against the first implementation (44 pass), and passes after narrowing the change. - Green: 58 tests pass across feedback collection, interrupted recovery, process shutdown, and scheduled finalization. - Full Product support run: 4,609 pass, three skip, and the retained embedded PostgreSQL bootstrap test fails on this host. The log remains available. This is not a full local pass. - Local Product typecheck fails on installed workspace export mismatches outside the changed files. Hosted CI must check the exact commit with clean dependencies. No local full repository build pass is claimed. - Original paid grades, partial feedback receipts, and accounting journals remain unchanged. This repair makes zero provider calls. ## Risks - A stalled event read can consume the remaining read budget. Other endpoints keep their five-second cap. The shared deadline still reserves up to five seconds for the final checkpoint. - A slow event page can leave later runs uninspected. Those runs remain partial or unavailable. The change does not treat missing evidence as success. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
25f1070fbd |
fix(runner): preserve checkpoint proof across warm preparation (#18011)
## Thinking Path > - Paperclip coordinates agent work and preserves its execution state. > - Native Runner checkpoints preserve authority across controller changes. > - Warm sessions also need checkpoint writes while their successor prepares. > - External persistence replaces the local state file with a marker. > - Two existing checks treated those valid states as lost ownership or prior activity. > - This PR uses the existing preparation lease and canonical controller state to keep those checks accurate. > - The result is safe warm-session retirement and fresh-session startup on reused sandboxes. ## Linked Issues or Issue Description Related: #17439 and #17320. The open #17401 changes the recovery welcome window and does not cover these paths. **What happened?** A configuration change could fail before the next provider turn with `runner did not durably suspend before checkpoint`. The task already pointed to its preparing successor, but that successor did not yet have a native coordinator. The old completed runner's maintenance checkpoint was rejected. A separate fresh session on a reused sandbox could fail with `resumed_unavailable`. Checkpoint registration had replaced the full local state file with an external-persistence marker. The bootstrap check still required the old full-file format. Process supervision then reported unconfirmed termination before launch. **Expected behavior** Retire the completed warm runner under the same process's live preparation lease. Admit a fresh remote session only when the registered controller proves exact identity, no prior activity and pending bootstrap commands, and the remote session directory can be claimed atomically. **Steps to reproduce** 1. Complete a native warm turn with external checkpoint persistence. 2. Change configuration so instruction preparation must retire that session, then submit a follow-up. 3. Separately, admit a fresh native session on a reused sandbox lease after external checkpoint registration. **Paperclip version or commit** Reproduced in the checkpoint implementation introduced by #17439. Fixed against current master. **Deployment mode** Source checkout tests and a hosted Docker/Daytona incident. No deployment or original-task retry is part of this PR. ## What Changed - Accept completed-run maintenance under the exact successor's live, same-process preparation lease only before a native coordinator exists. - Preserve original controller generation, accepted-result, no-new-work, company and task checks. - Pass the registered controller through the existing connection callback and read its current canonical state for untouched-session proof. - Keep marker-only, identity, activity, backup and atomic remote-directory guards. - Add regressions and update execution semantics and session verification recipes. ## Verification - Warm retirement red: the real Postgres writer rejected the preparing successor's `runner.suspend` checkpoint with `native_runner_checkpoint_owner_lost`. - Warm retirement green: 14 Postgres ownership cases pass, including stale/foreign/expired owners, conflicting native authority and new provider work. - Bootstrap red: real controller registration and external persistence caused the resumed and replacement positives to fail. Six external negative cases passed. - Bootstrap green: all 34 bootstrap cases, the bootstrap-state predicate and 13 connection-driver cases pass (48 total). - `node scripts/check-maps.mjs` and `git diff --check` pass. - Full `pnpm -r typecheck` and `pnpm build` pass. - Full local `pnpm test:run` remains partial/interrupted. The default attempt could not initialize embedded Postgres on this Mac. A dedicated Postgres 18 attempt ran for 17 minutes without a completed file result before interruption; the generic helper separately confirmed unsupported embedded Postgres (`initdb` failed after five attempts) and ignores the external database URL. The generic probe skipped one test. Setup and interrupted-run logs are retained; neither is claimed as a behavioral red or a full local pass. - Real source-hosted Codex/Daytona acceptance was attempted at `6ce7fcacc4` with `gpt-6.1-sol`, `xhigh`. The actual task stopped before provider work at `runner_ingress_auth_failed`, with zero authenticated connections and zero source events. A separate environment-qualification handshake reached challenge/response; that does not prove task authentication. The three-turn warm reuse, configuration retirement, useful output and reload workflow remains unverified, so this PR stays draft. - The live attempt used the cached Linux amd64 Runner binary `7437a4e32a099180889be534fcfb2a086046514490c1448aba46c55848983ed8` from the immutable Daytona image. The current staging binary could not be copied because SSH timed out, and the configured current EXE image returned registry 403. No compatible current local or CI-downloadable artifact was available. Earlier setup failures and cleanup receipts are retained; no provider success is claimed. Disposable test resources were cleaned up. - Exact-head Greptile completed at 5/5 with no open feedback: https://github.com/paperclipai/paperclip/pull/18011#issuecomment-6469515952. All current-head CI checks pass: 53 successful checks and two intended Storybook skips. ## Risks The preparation fallback affects maintenance checkpoints only after an accepted result. An existing native coordinator prevents that fallback. Stale controller generations, expired or foreign preparation leases, wrong tasks and new provider work remain denied. The bootstrap check retains the live controller object and reads current state each time. It does not trust a marker or restore local state beneath a live controller. Pending turns, prior events/connections, backups and existing remote directories retain recovery gates. No schema, provider binary, credential, permission or lifecycle-policy change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. This session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.10 |
||
|
DottaandPaperclip
|
c08da3e0a7 |
perf(server): narrow skill list reconciliation reads (#18023)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The skill library lists current company skills and their agent attachments. > - Each list request also reconciles bundled skills, folders, and local files. > - These reads return full agent records and large metadata that their callers do not use. > - Conditional requests do this work before they return an unchanged response. > - This pull request selects the fields each caller needs and avoids unused folder lookups. > - The result is less database transfer and fewer round trips while reconciliation stays current. ## Linked Issues or Issue Description Refs #17941. That change reuses bundled categories within an import. This change removes other repeated work in the same list path. **What happened?** Warm skill-list requests still made 119 SQL calls in a local HTTP fixture. The folder pass returned full skill metadata. The local-source pass also returned remote skills. Attachment counts loaded full agent configurations and monthly spend. Bundled reconciliation resolved folder paths that its release check did not use. **Expected behavior** The list must preserve its current result, attachment counts, access rules, local-file reconciliation, and bundled release checks. Internal reads should return only the data their callers need. **Steps to reproduce** 1. Use a company with many remote skills and agents with large saved configurations. 2. Read its skill list, then repeat the read with its ETag. 3. Measure the queries and response values before the server returns `200` or `304`. 4. Run `pnpm --filter @paperclipai/server exec tsx scripts/benchmark-skill-list.ts --samples=5 --network-delay-ms=3.5` for a synthetic reproduction. **Paperclip version or commit** Baseline service: `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. The branch is based on current master. The paired mode changes only the skill-service module; both versions use the same imported helpers. **Deployment mode** Built from source. Local HTTP server and disposable PostgreSQL 17. No hosted deployment is part of this PR's local proof. ## What Changed - Read only the desired-skill selections of nonterminated company agents for attachment counts. Resolve each agent's references once against the full visible skill list. Preserve IDs, keys, aliases, duplicate handling, and ambiguous slugs. - Read only `sourceKind` from metadata for bundled folder classification. - Limit local-source reconciliation to local-path rows. - Skip folder-path enrichment only for bundled reconciliation. Normal imports and list responses keep their folder paths. - Add real-database byte regressions and a local HTTP benchmark. The benchmark can alternate the old and new services against one database, actor, and process. It checks complete response equality and conditional responses. The company-skill service owns these reads. The query predicates and freshness checks remain at their existing owners. This change adds no cache, schema change, or new background work. ## Verification - The attachment byte regression failed before the fix: 601,418 captured response-value bytes exceeded its 2,000-byte bound. The same case passed after the fix. It checks fresh configuration edits, IDs and keys, duplicate aliases, ambiguous slugs, paused agents, terminated agents, and company scope. - 111 focused service and authorization tests passed against isolated real PostgreSQL 17 databases. They include full result equality, retained stored Markdown and metadata, local-file updates, release checks, project visibility, and import authorization. - The default embedded PostgreSQL probe skipped the first local attempt on this Mac. The completed run used a local-only bootstrap adapter for disposable databases. Queries, migrations, services, authorization, and filesystem work used their real implementations. The adapter is not part of this PR. - In repeated synthetic HTTP byte runs, calls fell from 119 to 98 per request. Result-value bytes fell from about 7.75 MB to 1.64 MB per request, about 79%. These values are measured before Drizzle decoding. They exclude protocol framing and do not measure provider billing. - A separate-process run without the byte observer did not show a latency gain. The paired experiment removes that comparison gap: it alternates both services against the same database and checks identical complete responses. On final commit `3076fe1402085f7286ff58e642d7caca892a5f80`, five observer-free pairs had median times of 2.582 s before and 1.850 s after (28.3% lower). The candidate was faster in all five pairs. Its range was 1.802–2.199 s; the baseline range was 2.327–2.725 s. Every normal response had the same complete body hash, and both versions returned the expected unchanged conditional responses. These are local service/HTTP timings with synthetic data and 3.5 ms delay per network direction. They are not hosted acceptance. - Final-head workspace typecheck and build passed. The full local test command did not complete because embedded PostgreSQL bootstrap failed on this Mac; I stopped that owned runner after 16 native-runtime cases failed. All 16 cases then passed when rerun alone against disposable PostgreSQL 17 using their existing external-database path. The 111 focused PostgreSQL 17 tests above also completed successfully. This does not constitute a completed full local suite; all current-head remote CI gates passed (55 successful checks and 2 skipped). Module-boundary and whitespace checks pass. An initial nested package command used the global pnpm; subsequent commands were pinned to pnpm 9.15.4. - The first canary dry run failed while installing an unrelated standalone package: npm returned 404 for the newly published Rollup 4.66.0 tarball. The exact download subsequently returned HTTP 200. The one retry passed without code changes. - Benchmark commands: add `--capture-values=false` for observer-free timings. Add `--baseline-ref=e1ba77f06771a27114b3999004b0ca5e3c172dd4 --capture-values=false` for paired service acceptance. The normal mode uses the production router. The paired mode uses equivalent default-list HTTP handlers and a synthetic authenticated board actor; it does not test login. ## Risks The SQL selection must stay in sync if the desired-skill format or folder classifier needs new fields. Existing parsing and reference resolution still interpret the selected data. The synthetic fixture has large unused payloads. It proves the mechanism and preserves required content. Its percentage is not a projection of whole-instance transfer or billed savings. Hosted latency still needs verification after merge and deployment. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, shell execution, real PostgreSQL, and local HTTP testing. The host does not expose a precise backend revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jamie Buchanan
|
3339582810 |
fix(adapter-utils): stop asking the GitHub broker on every local git call (#17886)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs use `git` and `gh` through the managed GitHub launcher. > - The launcher asks the broker for credentials before every `git` and `gh` call. It also asks for local commands that do not use the network, such as `git status`. > - Each request runs a transaction and an UPDATE in the database. On a busy install this gives hundreds of thousands of writes each day and long COMMIT waits. > - This pull request stops the request for local `git` commands and writes the identity summary only when it changes. > - The benefit is fewer broker requests and fewer database writes. The credential rules do not change. ## Linked Issues or Issue Description - Fixes #17879 ## What Changed - The managed `git` launcher does not ask the broker for local subcommands (`status`, `diff`, `log`, `rev-parse`, `add`, `config`, `branch`, `checkout` and others). It reads past global options such as `-C` and `-c`. An unknown subcommand or option still asks. `commit` and `var` still ask, because the managed author comes from the broker answer. - The launcher keeps no broker answer between invocations, so no GitHub credential is written to a file. The suggestion in the issue to reuse one answer per run is not part of this pull request, because the reused answer would have to rest on disk. - `resolveGitHubOperationCredentials` writes `run_identity_contexts.github` only when the summary changes. - `doc/execution-github-identity.md` describes the behavior. ## Verification - New launcher tests run real `git` against a fake broker that counts POSTs. A 12-command sequence (init, status, config, rev-parse, add, diff, commit, log, branch, show, status, diff) made 12 POSTs before and 1 POST after. The one POST is `commit`. - The same tests check that `var`, `remote`, `fetch`, an alias, and an unknown option still ask, and that `git var GIT_AUTHOR_IDENT` returns the managed author. - A new test covers the unchanged-summary write. - `vitest run` on the launcher, launcher environment, callback bridge, github-operation-credentials, run-identity, heartbeat-github-launchers, native-github-access, openapi-routes, run-trust-preset, http-log-redaction, skill-source-github-access and chat-channels tests: 12 files, 1901 passed, 1 skipped. - `vitest run --project @paperclipai/adapter-utils` on `src`: 1903 passed, 12 skipped, 1 failed. The failure is `sandbox-callback-bridge.test.ts`, a file this PR does not change. It passes alone (89 of 89, three runs). - `tsc --noEmit` passes in `packages/adapter-utils` and in `server`. - Three deliberate breakages each made a test fail: `var` back in the local list, the broker always asked, and the summary write skip removed. - Not run: the full server suite and `pnpm build` (the local machine has no `cargo` for the runner binary). ## Risks - In a partial clone or an LFS repository, a local command that must fetch a missing object runs without a token. - A `git` command that is not local (`commit`, `fetch`, `push`) and every `gh` command still asks the broker once. - No migration. No credential is stored at rest. ## Model Used - Claude Sonnet 5.5 (`claude-sonnet-5-5`, Anthropic) through Claude Code, with tool use (shell, file edits, test runs), wrote the code and tests. - Claude Opus 5.5 (`claude-opus-5-5`, Anthropic) through Claude Code wrote the issue text and this description. - A person asked for the change and submits it. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
980775e0e9 |
fix(server): schedule inbound voice history retention by deadline (#17583)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls leave history after provider cleanup is confirmed. > - The chat maintenance tick repeatedly deletes old admission history. > - Each eligible row already contains the dates needed for its retention deadline. > - This pull request schedules that deadline and wakes on committed changes. > - Idle retention stops querying while the existing seven-day policy remains enforced. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound voice history retention. This PR follows merged #17580, which owns physical call cleanup, and now targets master. Call reports remain a separate migration. **Current behavior** Voice maintenance scans admission history repeatedly. It deletes confirmed, terminal, unbound admissions when both expiry and last update are at least seven days old. **Proposed behavior** Load existing retention deadlines at startup. Wake from the existing admission transaction notifications. Delete at the later timestamp plus seven days. Lock and recheck current eligibility before deletion. **Reason and benefit** Configured idle endpoints and future history do not cause recurring retention SQL. Failed deletion retains a retry. Restart reconstructs the same saved deadlines. ## What Changed - Add a voice-owned retention worker using the existing scoped deadline helper and coordinator. Yield after 100 history rows per scope refresh and reset reconstruction on registration. Keep complete one-shot inventory reads. No table, field, or migration is added. - Extend `notifyVoiceInbound` to notify retention before the same committed writes that already notify cleanup. Callers need no new recovery wrapper. - Preserve confirmation, terminal-state, session, task, and seven-day eligibility conditions. Round PostgreSQL submillisecond deadlines up. - Lock and recheck the current row before deletion. New bindings or moved dates invalidate stale candidates. - Remove history pruning from recurring voice maintenance. Keep the explicit one-shot diagnostic path. Leave call reports for their own slice. - Document producer ownership, retry/drain behavior, tests, and external wake limitations. ## Verification Published head: `53f2fafcb64d104887eb4455c292b48e6dd10253`. It includes the normal merge of actual landed master `71b448b9cb841e6fa18fe883585430c5081245da` after cleanup #17580. That ancestry integration at `768c650365` preserved the complete binary diff of all 13 reviewed retention paths. This follow-up changes only the existing cleanup fairness test and its failure ledger, bringing the final scope to 14 paths; all production source is unchanged from that integration. All unrelated master changes, cleanup repair, migration `0396_tan_paladin.sql`, and gateway clock repair are preserved. Database files, shared worker, workflows and lockfile match actual master. The earlier 282-case/type/build evidence below belongs to unchanged retention source at its stated head; current repair acceptance is 37 distinct complete cleanup and retention lifecycle cases. All 58 current-head checks are now clean, including the Canary Dry Run; Greptile is exact-head 5/5 with no new actionable issues. The current Ponytail turn stopped without a verdict, which is an authorized nonblocking state. The earlier failing CI remains preserved below. - **282 distinct targeted cases pass**: all 127 voice integration cases and all 155 cases in ten complete retention, cleanup, recovery, company-removal, shared-worker, startup and hostname suites. These include 13 retention and 14 cleanup lifecycle cases. Voice integration, parent recovery and company removal start on separate empty databases and migrate themselves. Retention uses its dedicated database after the setup repair; cleanup uses its already-migrated dedicated database. - Ponytail's prior exact-head 4/5 review found configuration-driven history rescans. Five real connection saves selected 1,025 rows over 205 future admissions before repair. Retention now receives only admission hints and startup reconstruction; the same saves select zero admission rows and retain the original deadline. Cleanup still receives its full configuration and authority scopes. Restrictive binding foreign keys prevent implicit unbinding outside the admission-write boundary. - Successful deletion now returns its confirmed result without reloading the deleted row. A real SQL-count case and a guarded negative probe cover that path. Changed or not-yet-due history still reloads current eligibility and its deadline. - Server typecheck, build and module-boundary checks pass. No schema, driver, shared scheduler contract or provider protocol changes are added by this slice. - A real 105-row future-history case failed before the reader repair because startup selected 105 rows rather than yielding at 100. The existing paged shared-worker contract now bounds each scope refresh. The same-owner restart case reconstructs every deadline and deletes all 105 eligible rows at the original deadline. It verifies SQL quiet before that deadline and after settlement. Removing only cursor reset fails at five rows instead of 100 after restart; guarded source was restored before final acceptance. - Existing cases retain the later-of-two-timestamps seven-day policy, submillisecond rounding, current-state confirmation and binding checks, concurrent binding lock, rollback/outer commit, first write during drain, failure retry and shutdown. Real signed ingress, cleanup confirmation and maintenance-lane removal remain covered by the complete voice suite. - The fresh retention fixture first failed on missing schema and now applies the application migrator. The initial parent merge duplicated a test import and failed to parse before any voice case ran. Both failures are retained in the audit; the corrected complete suites pass. - Prior head `daca270a99` had a failed server (4/12) check, run 40557790710/job 122386307340. Its actual log reports a runner shutdown and cancellation before completion. This is incomplete acceptance, not a pass or a proven product flake. New-head CI must pass independently. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/68b62b6d13f28ac5f17f35b98adf5488752f9b97/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#current-parent-paging-and-restart-acceptance-october-28) records the exact scope, commands, counts and preserved failures. Earlier 218/133/134-case and full-workspace results belong to their stated heads. No new full local repository suite, live provider, Cloud wake, host/database suspension or savings claim is made. - [Notification review repair and current 282-case evidence](https://github.com/paperclipai/paperclip/blob/68b62b6d13f28ac5f17f35b98adf5488752f9b97/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#configuration-rescan-review-repair-october-28) records the new source. The prior head had green CI and a 4/5 Ponytail review; neither is accepted for this new commit. Fresh exact-head Greptile 5/5 and all CI remain required. Completed actual failing Ponytail findings must be repaired; a stopped or paused turn without a verdict is not a merge blocker under the user’s authorization. An active Ponytail review waits at most 30 minutes from its original request (25-minute monitor cutoff). - **Preserved current-master CI failures:** run `40697141057` failed server shard 7/12 in cleanup’s fairness test and Canary Dry Run on an npm `rollup-4.66.0.tgz` 404. A fixed final admission UUID reproduces the same fairness assertion: the second company was sometimes beyond the first 100-row page, so expecting it among the first 25 calls was invalid. The corrected test controls time and proves service within the next 25-call batch after discovery, all 105 successful calls before the failed provider’s saved retry, exactly one retry at its deadline, and idle SQL after settlement. Bypassing company rotation fails the corrected assertion; the shared helper was restored byte-for-byte. Both complete cleanup (24) and retention (13) suites pass after restoration. An intermediate 23/24 run incorrectly demanded idle before reconstruction finished; the audit preserves that failure and its corrected pending-intent wait. No production behavior, timeouts, dependencies or workflow gates changed. The exact npm metadata and tarball now return 200; new-head canary CI must pass independently. Prior Greptile/Ponytail 5/5 on `768c650365` are stale for this new head. [Full CI repair ledger](https://github.com/paperclipai/paperclip/blob/53f2fafcb64d104887eb4455c292b48e6dd10253/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#current-master-ci-repair-october-28). ## Risks - Supported admission writers must use the existing transaction notification boundary. Independent SQL writers and additional processes need a separate wake contract. - Retention never deletes unconfirmed cleanup or session/task-bound history. Those records remain with their owners. - The in-process deadline does not wake stopped compute. This PR does not extend Cloud alarms or relax sleep eligibility, and does not claim deployed host/database suspension. - Call-report reconciliation and other chat maintenance still need separate migrations. ## Model Used OpenAI Codex with reasoning, shell, and code-editing tools. The exact model version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green for this new head - [x] Greptile is 5/5 on this new head with no unresolved actionable findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
71b448b9cb |
fix(server): schedule inbound voice cleanup from durable obligations (#17580)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls need approval, expiry, and safe provider cleanup. > - The remaining voice maintenance tick scans admissions every second, including idle endpoints. > - Admission status does not prove that the provider call has ended. > - Existing admission rows can retain the cleanup owner, confirmation, and retry deadline. > - This pull request replaces the cleanup scan with committed changes and actual deadlines. > - Idle cleanup stops querying while unfinished calls keep a recoverable path to termination. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound voice admission cleanup. Its reply-delivery parent #17578 has landed; this PR targets master. Live-session recovery is in #17576. It preserves separate ownership for those services. **Current behavior** The voice maintenance tick scans expired admissions even when no cleanup is due. A session can commit before the separate approved-admission write. Expiry can then terminate that authorized call. A rejected guest call can be labeled ended before provider termination succeeds. Seven-day retention can delete that unfinished cleanup. **Proposed behavior** Wake cleanup from committed admission and session changes. Schedule the original expiry or saved retry. Claim current state under the admission lock before provider I/O. Commit approval with session reservation. Keep unconfirmed termination durable and transfer bound calls to live-session recovery. **Reason and benefit** Configured idle endpoints and confirmed history cause no recurring admission cleanup SQL or provider requests. Approval, retries, crash recovery, and cleanup remain supported. History retention and call reports stay separate migration items. ## What Changed - Recover terminal and rejected legacy admissions whose session binding was lost. Copy known termination or bind a proven matching live session for cleanup without granting approval. - Register inbound cleanup with the shared coordinator and scoped deadline worker. Yield after each 100-row scope page and reset cursors on registration. Remove only cleanup from periodic voice maintenance. Keep complete manual reconciliation. - Add commit-aware notifications at admission, approval, rejection, guest binding, pending-tool expiry, lifecycle confirmation, and session termination writes. - Commit authorized session binding and admission approval together. Recheck cleanup ownership under the same admission lock. Recover older interrupted bindings only from matching session evidence. - Add nullable cleanup confirmation, next-attempt, and provider-account fields to the existing admission table. No new queue table is added. Generated migration 0396 backfills only provable historical termination in committed 1,000-row keyset batches and adds a concurrent partial index. - Save the retry before provider contact. An unconfirmed end receipt retains cleanup. Bound calls remain owned by live-session recovery. - Verify the signed provider account before using a rotated key for unbound cleanup. A different account cannot inspect or end the saved call. Legacy admissions require their original credential fingerprint. - Prevent history pruning and company deletion from erasing unfinished cleanup. Company deletion locks endpoints before the voice owner's check, matching ingress serialization. - Document producer coverage, ownership, deadlines, verification, and remaining sleep limits. ## Verification Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply parent #17578 has landed. A normal merge integrates actual master `e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated changes, and regenerates this slice’s migration after landed migration 0395. Fresh exact-head reviews and CI are required. - The previous Ponytail 3/5 finding is repaired: legacy terminal admissions no longer enter approval-only repair, and rejected admissions with valid matching live sessions retain their rejection and persist the binding needed for session-scoped cleanup confirmation. Five regressions failed with ownership conflicts before the repair. Five conflicting-session regressions prevent an unrelated leg from being claimed. No cleanup operation grants approval or starts a provider call. - **24 complete cleanup lifecycle cases and 122 complete voice-store integration cases pass** on the repair. After actual-master integration and migration regeneration, the complete 24-case cleanup suite passes again on a new empty PostgreSQL 17 database, and the database migration integration test passes separately. The cleanup owner, voice-store test and admission schema are byte-identical to that tested repair; the cleanup fixture only changes the migration filename. Migration numbering/safety, database build, production server typecheck and build pass. - Migration `0396_tan_paladin.sql` was generated with `pnpm db:generate`. Every landed SQL migration and the complete parent journal prefix are unchanged. Generated snapshots retain exactly 0392–0396, pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and committed 1,000-row historical backfill are retained byte for byte under the new generated filename, registered for top-level execution. The real application migrator boots a fresh database, repairs an interrupted concurrent index, serializes replays and verifies eligible-history backfill across actual transactions. - The preceding published head `48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but failed CI server shard 9/12: one existing connected MCP rate-limit test expected two real-clock calls to share a fixed 60-second bucket. A controlled window rollover reproduced its exact failure. That existing fixture now freezes only `Date` around the same-window 429 assertion, verifies acceptance after advancing one window, and restores the real clock in `finally`. All 92 gateway acceptance cases pass. Production rate limiting, timeouts and assertions are unchanged. This adds one existing CI fixture to the slice; the original failed run remains recorded. - The first regression invocation skipped the suite because it used the wrong database variable; that is not a pass. The corrected invocation reproduced five ownership conflicts. The first fixed cleanup run passed 22 and failed two new low-level deletion test calls on a retained-session foreign key; the regressions now check the cleanup deletion guard at its owning boundary, with production deletion ordering unchanged. The final full suite passes all 24. - The first complete voice-store invocation passed 121 and timed out on one transcript case at the unchanged 15-second limit. That case passed in isolation on a fresh database. One subsequent complete invocation, with builds finished, passed all 122 without source, assertion or timeout changes. The underlying timeout cause is unproven; the original failure remains recorded. A new PostgreSQL startup also failed from exhausted host semaphore capacity; fresh databases on the already-owned server were used without removing shared IPC resources. - Prior source `b8b12199b9` passed 266 distinct cases and carried three guarded mutation proofs; those older results are not added to this repair’s case count. Its CI run 40641168596 failed E2E shard 3 when legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub initially rejected a retry while the run was active, then accepted one targeted retry after completion; attempt 2 passed on that same old head. The lock holder and underlying cause remain unproven. The unchanged Node policy checker’s older failure on ignored generated Copilot manifests and passing tracked-input mirror are also retained in the audit; no workflow, checker, vendor manifest or ignore rule was modified. - [Current repair and failure record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28). Current-head CI and Greptile 5/5 gate merge. A completed failing Ponytail verdict must be fixed; missing or interrupted reviews do not block, and active pending review waits are capped at 30 minutes under the maintainer’s explicit instruction. No new full local repository suite, real provider, Cloud alarm, host/database suspension or savings claim is made. ## Risks - The migration distinguishes admission status from physical termination. Legacy ended guest rejections without proof remain cleanup obligations. Missing original credentials can keep those old obligations unresolved. - Future writers must use the existing notification boundary. Independent SQL writers and multiple processes need their own wake contract. - Pending inbound cleanup now blocks company deletion instead of allowing cascade deletion to lose the provider obligation. Existing voice-history deletion restrictions remain. This deliberately includes an active anonymous inbound ring and an old call whose termination cannot be verified. An operator must stop ingress and settle the call before deletion; this slice adds no abandonment override and never treats missing credentials as proof of termination. This is the retained cleanup ownership policy, not a claim that every deletion attempt can complete immediately. - History retention and report processing still run in the remaining maintenance lane. This PR does not extend Cloud wake coverage or claim measured host/database suspension. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.9 |
||
|
DottaandPaperclip
|
6ce0909db4 |
fix(agents): skip runtime probes for unchanged connections (#18016)
## Thinking Path > - Paperclip manages AI agents and their runtime settings. > - Operators save identity, model, connection, and run-policy settings from the agent page. > - PostgreSQL stores connection bindings as JSONB and can return keys in a different order. > - The save route compared JSON strings and treated an unchanged binding as a new connection. > - That false change started a remote runtime and credential probe during ordinary saves. > - This pull request compares binding values and checks real execution selection changes separately. > - Ordinary edits save without a probe. New connections and execution selections retain validation. ## Linked Issues or Issue Description Fixes #14518. Related work: #17760 also addresses prospective environment resolution. #14027 also adds explicit connection removal. This change is limited to the save validation gate and its regressions. **What happened?** A title-only save took 9.8–16.0 seconds in a hosted reproduction. JSONB key order made the unchanged AI connection look different. The route then waited for a remote runtime probe. **Expected behavior** Identity, thinking-effort, and run-policy edits skip the probe when the connection and execution selection stay the same. A changed connection, harness, model, provider, engine, or environment still validates adoption. **Steps to reproduce** 1. Save an agent with a managed AI connection in PostgreSQL. 2. Edit its title or thinking effort without changing the connection or execution selection. 3. Observe that the old route starts the adapter environment test. The fixed route saves without that test. ## What Changed - Use value equality for the stored and validated AI connection bindings. - Compare the effective execution selection separately. Preserve runtime probes for real selection changes and existing Copilot checks. - Add PostgreSQL route regressions for shared, delegated, default-user, and pooled bindings. Cover failed connection, model, engine, environment, and harness changes. - Keep the existing credential rejection and reconnect tests. Document the save behavior. ## Verification - `pnpm -r typecheck` passed on the rebased implementation. - Focused PostgreSQL 17 route selection: 21 passed, 58 excluded by the name filter. This covers ordinary saves, selection-change rejection, and credential health. - Full PostgreSQL 17 connection suite: 79 passed, zero skipped. Final `pnpm build` passed at `8fde75fa1a8fe63c71b05a3db149edf289295672`. - Before the fix, the PostgreSQL regression reached the unintended probe and failed for unchanged shared, delegated, and default-user bindings. The pool control passed. - An early controlled HTTP comparison of the binding fix used a synthetic 500 ms provider probe: five measured saves had a median of 798 ms before and 56 ms after. Probe count fell from five to zero. This is local supporting evidence, not a hosted latency result. - Earlier adjacent-route run: 159 of 160 tests passed. One existing roster-edit case exceeded its 15-second timeout. That case passed alone with the normal timeout. The cause of the earlier timeout remains unproven. - Local full-repository tests were not run. Native PostgreSQL startup fails because the host has exhausted System V semaphores. The focused tests use an isolated PostgreSQL 17 Docker fixture. - `git diff --check` passed. Hosted improvement remains unverified until deployment. - Greptile and Ponytail Paperclip Review both rated the exact head `8fde75fa1a8fe63c71b05a3db149edf289295672` 5/5, with no actionable defects. - CI attempt 1 passed 51 checks. Three self-hosted runners lost communication, interrupting the chat shard, serialized server shard, and canary dry run. GitHub annotations identify the runner communication failures; the serialized shard log also records a shutdown signal. Only those failed jobs were retried, once, on the unchanged commit. The [full CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40692641103/attempts/2) then passed on attempt 2 at the same commit. All 55 passing checks and the legacy Snyk status are green; no checks remain pending. ## Risks This changes when connection adoption validation runs. Separate selection checks preserve validation for real runtime changes. Regression tests verify that rejected selections do not replace the saved configuration. Company access, connection authorization, activity records, and Copilot checks retain their existing paths. There is no schema or migration change. ## Model Used OpenAI Codex, configured model `gpt-6.1-sol` with ultra reasoning. Work used source inspection, shell tools, browser network timing, and PostgreSQL route tests. The context window size is not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
83709f75eb |
fix(ui): hide task row retry status labels (#18017)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task lists and the inbox help people scan their work. > - Shared task rows show system interruption messages beside task titles. > - These messages add detail that the person cannot act on from the row. > - This pull request removes the three system interruption labels from those rows. > - People can still open the task to inspect progress and use Retry. ## Linked Issues or Issue Description Refs #16965, which introduced these labels. Related recovery report: #17386. This change only removes row text and does not resolve that report's retry behavior. **What existing behavior does this improve?** The shared task rows in task lists and the inbox. **Current behavior** Rows show “Retry available,” “Thinking,” or “Trying again soon” beside the title for unresolved system interruptions. **Proposed behavior** Omit all three labels from shared task rows. Keep the existing system-interruption guard so the row does not show a legacy recovery chip in their place. Keep task status, other badges, and retry controls inside the task. **Reason and benefit** People can scan task titles with less incidental execution text. **Breaking changes** The three row labels disappear. APIs, recovery behavior, and task conversation controls stay the same. ## What Changed - Remove the system interruption text from every shared task-row presentation. - Cover working, scheduled-retry, and failed interruptions in both row presentations, including a legacy recovery action that must stay hidden. - Record the expected row behavior and verification states in the task feature map. ## Verification - Passed: token gates, feature-map contract and reference checks, and Git diff whitespace checks. - Passed: 83 component tests in `IssueRow.test.tsx` and `Inbox.test.tsx`, including the new regression cases; full `pnpm -r typecheck`; full `pnpm build`. - The full local `pnpm test:run` was attempted and stopped after 16 database-dependent failures. An independent disposable `initdb` reproduced `FATAL: could not create semaphores: No space left on device`. The host has exhausted its macOS System V semaphore capacity. All required CI test lanes passed on the final commit and provide full database-suite coverage. An earlier attempt also found an unbuilt plugin SDK; building that existing dependency repaired collection before this database failure. - Review the working, scheduled-retry, and failed interruption branches. Each returns no system label. The existing guard still suppresses legacy recovery chips. - Independent source review found no actionable code issues. Regression cases check title navigation, removal of the three labels, and suppression of replacement recovery chips. - All required CI gates passed on commit `dd1d7b08dbba8a10a95c5d24c4aea64479befe95`: typecheck, general and serialized tests, browser suites, build, canary, and Runner checks. - Greptile reviewed that commit at 5/5 with no open findings. The optional Paperclip review also returned 5/5 with no findings; its check status is neutral. ## Risks - Low risk. This removes row text only. Task status, recovery records, other badges, and task-level retry actions keep their current behavior. ## Model Used OpenAI Codex, GPT-6 family. This session does not expose an exact model ID or context-window size. The assistant used reasoning, shell tools, and subagents for source review and validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8279c99008 |
fix(evals): clarify report details and link tested applications (#18009)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E reports help operators inspect real application journeys. > - A worker environment does not identify the application that a journey tested. > - Reports hid that URL and gave missing diagnostics too much space. > - Standalone reports also sent the logo link to a missing parent page. > - This pull request shows the recorded target and checks in the shared report. > - The benefit is a clear report with useful links and inspectable diagnostics. ## Linked Issues or Issue Description **What happened?** A standalone Product E2E report had no application link in test details or the screenshot gallery. Its details showed several empty diagnostic sections before useful information. The logo could resolve to the page host root and return 404. Campaign notes added outside the report shell could overlap the fixed sidebar. **Expected behavior** Show the recorded application URL beside each test and screenshot. Put recorded checks before optional diagnostics. Keep campaign context inside the shared layout. Link the logo to the evaluations directory. **Steps to reproduce** 1. Render a retained campaign at a standalone page path without a history index. 2. Open a test with screenshot evidence and missing startup timing. 3. Inspect test details and gallery information, then click the report logo. **Paperclip version or commit** Reproduced in the report renderer at `39e811c9c5`. This branch applies the fix to current master. **Deployment mode** Static Product E2E HTML report generated from retained normalized results. ## What Changed - Add optional recorded `targetUrl` metadata and resolve existing Product E2E and provider-connection targets. Only HTTP and HTTPS links without embedded credentials are shown. - Show the target URL in test details and screenshot gallery information. - Put recorded checks and execution facts first. Keep raw usage, timing, and startup details under Diagnostics. Preserve missing metadata as unknown. - Keep campaign titles, descriptions, notes, and evidence links inside the shared report layout during retained-evidence regeneration. - Fix standalone navigation and let the brand lockup wrap at narrow sidebar widths. - Match report profiles by suite and recorded provider/model before falling back to an inferred profile. - Add report regressions and document the retained metadata contract. ## Verification - Focused report tests: 86 passed across 8 files on the isolated branch. Command: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts report-target.test.ts report-navigation.test.ts report-catalog.test.ts first-task-report.test.ts report.test.ts dashboard-regenerate.test.ts`. - Browser verification covered dark desktop widths 1440 and 1000, light intermediate width 780, and dark mobile width 390. It covered the drawer, URL links, gallery keyboard navigation, focus return, and the logo destination. - The published retained report keeps all 3 original passing cases and 7 recorded checks. Its campaign and screenshot bytes remain unchanged. This presentation refresh made zero provider calls. - Full `pnpm build`, `pnpm -r typecheck`, and the eval TypeScript check passed after rebuilding the reused package dependencies. The first typecheck attempt had stale local dependency builds; those errors are resolved. - The local `pnpm test:run` attempt hit repeated embedded PostgreSQL bootstrap failures in server suites. I stopped that attempt and retained its log. It is not counted as a passing check. Current-head CI remains the full test gate. - Review follow-ups add provider/model and cross-suite profile regressions. Malformed campaign presentation metadata is rejected before any report or retained-result writes; ten malformed input cases verify that retained results stay unchanged. ## Risks - This changes presentation across Product E2E reports. Historical reports without a target show “Not recorded” rather than an inferred URL. - Unknown startup reuse data remains in raw metadata but no longer creates an empty table. - Standalone logo links open the shared public evaluations directory. Campaign history links still use a matching history index when present. - No application runtime, database, or measurement verdict changes are included. Private QA configuration and raw campaign evidence stay outside this PR. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bd0d81c7b |
fix(e2e): follow proven workspace deferrals during onboarding (#18008)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify the user outcome through the browser and scoped APIs. > - An onboarding child can wait while its parent owns their shared workspace. > - The server records that no provider work started and schedules a durable replacement. > - The journey rejected that expected wait, and feedback counted it as unknown legacy execution. > - This pull request follows the exact replacement and records the distinct no-provider evidence. > - The child must still finish through native execution and produce the independently checked download. ## Linked Issues or Issue Description Refs #17940. **What happened?** The fresh-workspace journey failed with `onboarding_first_task_failed` when a child encountered shared-workspace contention. The cancelled admission had an explicit `workspace_wait` receipt with `providerWorkStarted: false`. Its linked replacement was still scheduled. The runtime had not been selected. The default `legacy` value did not represent a provider execution. **Expected behavior** Keep both rows and their run-limit cost. Follow only the unique replacement for the same company, agent, and issue. A pending replacement must not pass. Real failures and contradictory execution evidence must still fail. **Steps to reproduce** 1. Start a native onboarding parent and accept its child-task proposal. 2. Observe the child admission while the parent still holds their workspace. 3. Present the server's no-provider cancellation and its durable scheduled replacement to the journey and feedback collectors. ## What Changed - Add a shared predicate for the explicit no-provider workspace-deferral receipt. Reject runtime, process, output, usage, and cost evidence that contradicts it. - Follow exact, unique retry lineage during onboarding. Wait for active or scheduled replacements without repeating browser actions. Preserve the campaign deadline, run cap, native artifact attribution, and download oracle. - Retain per-run feedback inspection classifications: `native_events`, `provider_not_started`, or `incomplete`. Require exhaustive event reads. Keep pending work and uncertain execution partial. - Add causal regressions and document the evidence contract. Historical failed attempts keep their original grades. ## Verification - Red: the new journey regression failed with the observed `onboarding_first_task_failed` before the repair. The new feedback cases also failed before the repair. - Green: 70 journey tests and 74 feedback-family tests passed. The final affected journey/feedback gate passed 275 tests plus 137 Node checks. Five retry-lane reset cases failed before the review correction and passed after it. - `pnpm test:e2e:runner:typecheck` passed after the retry-counter correction. - `pnpm -r typecheck` and `pnpm build` passed before the final rebase. - The full local `pnpm test:run` and harness unit gate encountered embedded PostgreSQL startup failures. The harness gate passed 4,574 tests and failed its database-startup test. The final focused journey/feedback gate and all 137 Node checks passed. CI will provide independent full-suite results before merge. - Current-head CI passed 55 checks; two checks skipped and one was neutral. Greptile is 5/5, with zero unresolved review threads. - Live Cloud verification will use the reviewed harness and Operations revisions. This PR does not claim that a fresh Cloud campaign has passed. ## Risks A broad cancellation exemption could hide real failures. The exemption requires the server's explicit receipt, unresolved runtime selection, no contradictory execution evidence, and exact durable lineage. Missing successors remain pending until the existing deadline. A successful native child and valid downloaded bytes remain required. No server behavior, approval policy, or budget policy changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e7ea2e2961 |
fix(evals): export grades for expanded qualification batches (#18010)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E campaigns retain each attempt and its independent grades. > - Braintrust receives product and cleanup grades for each retained attempt. > - A six-case qualification produces twelve scores. The export allowed only eight. > - Experiment creation succeeded, but grade insertion stopped before dispatch. > - This fix allows fourteen scores so the six- and seven-case batches retain every result. > - Operators can inspect failed and successful grades without running an agent again. ## Linked Issues or Issue Description Refs: #17044 **What happened?** A retained six-case Product E2E campaign created its Braintrust experiment, then failed with `Experiment ingestion allowance exhausted`. Its twelve deterministic scores exceeded the eight-score cap. No insert request was sent. Agent results and model accounting remained saved. **Expected behavior** The bounded qualification batches must export both grades for every selected attempt. Failed grades must remain visible. **Steps to reproduce** Create an experiment through `openExperimentIngestion`. Insert six or seven events with `product_pass` and `cleanup_pass` scores. The old limit rejects the insert before it reaches the transport. **Paperclip version or commit** Reproduced on master `cff331f6ca0a652b5e41641c68dfa5fa584c22ae`. **Deployment mode** Local Docker controller built from source. The failure is in the shared Product export path. ## What Changed - Raise the finite experiment score cap from eight to fourteen. Keep the three-request, 4 MiB and 60-second limits. - Add regressions for the six- and seven-case batches. Keep the failed product grade in each test batch. - Verify that the receipt collector accepts the old eight-score policy and the new fourteen-score policy. It rejects a fifteen-score policy. - Document the seven-attempt export ceiling. This change does not add agent calls or automatic retries. ## Verification - Before the fix: both new batch regressions failed with `Experiment ingestion allowance exhausted`. - After the fix: 67 tests pass across experiment ingestion, ingestion accounting, experiment projection and scheduled finalization. - Command: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/experiment-ingestion.test.ts tests/runner-e2e/ingestion-usage.test.ts tests/runner-e2e/scheduled-finalization.test.ts tests/runner-e2e/braintrust-experiment.test.ts --configLoader runner --cache=false`. - `git diff --check` passes. All 58 current-head checks are acceptable, including typecheck, build and tests. Both reviews are 5/5, with no unresolved threads. - Live agent attempts will not be repeated to verify this reporting fix. Recovery uses the saved results and a separate bounded ingestion receipt. ## Risks - New exports can submit at most six more deterministic scores per experiment. These are saved grades, not model judge calls. - Larger campaigns still fail closed. Existing receipt files keep their original policies and traffic counts. - This fix does not resolve the plan-lifecycle behavior failures observed in the qualification. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e93732c3d4 |
fix(server): deliver voice replies from commits and saved deadlines (#17578)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Phone calls receive approved answers from the existing chat publication queue. > - The chat maintenance timer checks for voice replies every second, including when no reply is ready. > - A reply can be saved before its publication commits. An early wake alone can miss that final transition. > - Existing session, reply, publication, and action rows already record delivery and recovery work. > - This pull request uses committed changes and saved deadlines to drive phone reply delivery. > - Empty reply queues become quiet while authorization, ordering, and uncertain-send protection remain in place. ## Linked Issues or Issue Description Stacked on #17576. Related delivery infrastructure: #15625. Related voice connection work: #11418. **What existing behavior does this improve?** Background phone reply delivery and recovery of abandoned dispatch intents. **Current behavior** Chat maintenance calls voice reply delivery every second. It also scans old dispatch intents. Publication transport starts a direct push before the final publication commit. The timer masks that timing gap. **Proposed behavior** Committed publication, reply, session, and configuration changes wake the shared coordinator. Ready replies drain in bounded passes. Saved rate-limit and dispatch-expiry deadlines retain recovery. An empty queue or a permanent delivery barrier has no recurring reply scan. **Reason and benefit** Remove recurring SQL from idle phone reply delivery without dropping replies, replaying uncertain sends, or adding a queue table. **Breaking changes** No API or schema change. Explicit manual recovery remains available. Inbound admission cleanup, history retention, and reports remain on their current paths for separate migrations. ## What Changed - Extract ordered reply state and multipart policy within the voice owner. Retain current authorization and atomic dispatch claims. - Register a scoped reply worker with startup reconstruction, fair work batches, failure backoff, drain/resume handling and shutdown ownership. - Extend the existing shared dispatcher with normalized partial-page results to preserve the bounded parent recovery contract. Existing complete-array consumers retain their calls. - Signal before relevant writes in their existing transactions. Both normal publication completion and operator resolution wake the actual conversation after commit. - Bound reply inventory to 100 total headers per refresh. Preserve partial scans, concurrent changes and newer denied-call restoration obligations across reconstruction. - Track abandoned dispatch actions independently of live calls. Quarantine expired intents without resending them. - Use existing active-call authorization checks to reactivate replies that were previously denied. Healthy call inspections do not rescan empty reply queues. - Remove the periodic reply scan and the early direct publication push. Keep provider requests outside database transactions. ## Verification Published head: `1435cab10be3f95c38ab6672d2a890ae49e9d300`. Voice recovery #17576 has landed at master `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. A normal merge reconciles squash ancestry while preserving the complete reviewed 31-file reply patch byte for byte and all unrelated master changes. The PR now targets master; fresh exact-head CI and Greptile are required. The prior 449-case/typecheck/build results below cover the identical owned source, not a new full-suite run. - **449 distinct targeted cases pass**: 111 complete voice-store/reply; 19 complete parent recovery; and 319 complete tool-gateway, secret-proposal, shared dispatcher/coordinator/batching/startup/hostname cases. Production server typecheck and build pass. The policy checker and its Node tests pass. Later edits record evidence only. - Two real normal-publication/operator-resolution tests seed 105 unrelated calls and 105 future dispatches. Five publications selected **1,065/1,060 inventory rows before repair**; both paths now pass the same **125-row total budget** and deliver every answer once. This counts selected SQL results, not physical tuple visits or host savings. - Publication commits now signal their actual conversation. The voice reader returns at most 100 total session/action headers per refresh, preserves phase/cursor state and retries, and resets on worker registration. Current-state checks and dispatch claims still own provider effects. The 105-action case measures the page cap, restarts the same owner during reconstruction, waits at saved future deadlines and quarantines every abandoned action without resending. - A real 403 fixture proves that an older broad scan cannot erase a denied call first observed behind its cursor. Its owner callback still emits the committed restoration wake after scan completion. The existing actual-service authority-restoration and final-publication timing tests remain in the full suite. They are not replaced by direct helper calls. - Guarded removal of conversation scoping, cursor reset or the denial revision fence fails the intended budget, 100-versus-5 restart or missing committed-wake assertion; source is restored. An initial shadowed fixture name and an inadequate due-action reset probe are retained as failed attempts in the [current audit](https://github.com/paperclipai/paperclip/blob/d2c0a7de8c3e9cac8382b1268f3f5d883a2067b3/doc/plans/2026-10-25-voice-reply-delivery-sleep-audit.md#publication-inventory-and-paging-acceptance-october-28). - Parent recovery still passes paging, restart, behind-cursor insertion, feature changes, actual disablement, quiet health observations, current authority, exact-scope failure fences and wildcard-failure isolation through the shared worker. Four existing array-based worker consumers retain their API and pass their full suites. - The two voice fixtures self-migrate separate empty databases. Legacy tool-review/secrets suites use separate migrated clones. Earlier full-local attempts remain limited by embedded PostgreSQL setup and are not passing full-matrix claims. Older counts apply only to their stated revisions. Fresh exact-head Greptile, Ponytail if present, security and all CI remain merge gates. No live-provider, multi-writer, external-wake, deployed-sleep or savings claim is made. ## Risks - Missing a supported producer can strand work. Tests cover real enqueue, retrieval, normal publication, and operator publication paths. Startup reconstructs existing records. - Signals cover the process-local database owner. Independent SQL writers or other replicas need their own wake contract. - Authority restoration relies on the existing live-call recovery check. This dependency must be preserved if that owner changes later. - A provider acceptance receipt does not prove speech. Unknown outcomes remain blocked from automatic resend. - This change does not extend Cloud alarms, allow sleep during live calls, or prove deployed suspension or cost savings. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e1ba77f067 |
fix(server): schedule live voice recovery from session changes and deadlines (#17576)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain durable session state and provider identities. > - The chat maintenance timer scans live voice sessions even when no call exists. > - Session writes, saved expiry and active-call inspection deadlines identify real work. > - This pull request moves live-session recovery to the shared coordinator. > - Idle session recovery becomes quiet without losing exact-call cleanup or current authorization checks. ## Linked Issues or Issue Description Refs #17572. This PR is stacked on receiver discovery. It owns only live voice session recovery. Reply delivery, inbound admission cleanup, inbound history retention and call reports remain separate migration slices. **What happened?** The one-second chat maintenance lane reads voice settings and scans live sessions even when an instance has no calls or only terminal history. **What did you expect?** Reconstruct live calls on startup. Wake on committed session or configuration changes. Inspect active calls and retry cleanup at actual deadlines. Stop querying when no live obligation remains. **Steps to reproduce** Start an instance with no live voice sessions. Let startup settle. The old maintenance method continues to read settings and session inventory every chat maintenance pass. ## What Changed - Add a voice-owned bounded deadline reader and scoped worker on the existing coordinator. Retain concurrent writes across pages and reset partial scans on restart. No tables or migrations are added. - Separate generic connection health observations from configuration changes. Existing receiver and command consumers retain their notifications. Voice eligibility and provider operations remain in the voice owner. - Signal existing session transactions and configuration/feature changes. Keep provider calls outside database transactions. Standalone session writes use one small transaction helper. - Preserve saved expiry, the two-minute unknown-creation grace, active-call authorization and missed-hangup inspection, and exact-provider cleanup. Recovery never creates a provider call. - Retry transient authority lookup failures instead of treating them as confirmed revocation. - Fence expiration against a provider identity bound after discovery. A late known call remains eligible for cleanup. - Preserve work through rollback, drain, restart, concurrent refresh, backoff and shutdown. Page inventory and rotate bounded batches across companies. - Leave the four other voice maintenance duties in the periodic lane. - Document producer coverage and the supported writer contract in `doc/idle-voice-session-recovery.md` and the dated audit. ## Verification Published head: `f0b2e3cc22a18e7d08b10d4fc02aae4a19a52f30`. Receiver #17572 has landed in master at `dce004259394486f6361d0c0dd5a33a5272acc0a`. This normal merge reconciles squash ancestry; the complete 22-file voice recovery patch is byte-for-byte identical to the reviewed and tested `ad79c7e805` patch, while all unrelated master changes are preserved. The PR now targets master. Fresh exact-head CI and Greptile are running; interrupted Ponytail turns are not completed verdicts under the maintainer’s clarified merge policy, and an actual failing verdict still blocks. - **210 distinct targeted cases pass on the current repair**: 19 complete recovery lifecycle cases and 191 complete voice-store, coordinator, batching, startup and hostname cases. Production server typecheck and build pass. Later edits record evidence only. - CI `40633667141` on prior head `1b6323c164` failed the feature-change assertion with `34050 < 25000`; its verification aggregate also failed. The test now disables automatically advancing fake time, drives 20 controlled scheduler turns and requires all 205 calls to close before an unrelated inspection deadline can mask a lost wake. Removing generation replay fails with missing closures. The repair does not benchmark database/host speed or raise the elapsed-time bound. - A real committed exact-call regression reproduces wildcard-backoff starvation. The worker keeps the broad retry but processes independent exact hints and cached due calls. Each check reloads current state and authority. Failed exact scopes remain fenced. Removing wildcard isolation fails the intended assertion; a successful broad retry then leaves the worker SQL-quiet. Full-row and lean queries now share the state/scope/cursor predicate. - Earlier acceptance on `712f3862b5` passed 725 distinct targeted cases. Its measurements remain useful history: five real metadata writes and one health check over 205 future calls selected 1,230 rows before the inventory repair and zero afterward, without changing the saved deadline. Those older test counts are not added to the current 210-case count. - Both external voice fixtures self-migrate; the current full voice-store run uses a separate initially empty database. Earlier fixture/method/observer failures, source mutations and partial edit attempts remain in the [current audit](https://github.com/paperclipai/paperclip/blob/ad79c7e805367fc7e2df3ea53c942ca6c7d27365/doc/plans/2026-10-25-voice-maintenance-sleep-audit.md#wildcard-failure-and-deterministic-generation-acceptance-october-28). Source is restored after guarded mutations. - Older CI `40632057899` failed because its trusted policy script was absent from the checkout, while build separately received a runner shutdown signal. E2E shards succeeded but their aggregate failed on policy. The current checkout includes the landed policy tooling and its checker/tests pass locally. The shutdown cause is unproven; neither old failed run is counted as passing. Earlier full-local attempts stopped on embedded PostgreSQL setup failures and did not pass. - Fresh exact-head Greptile, security and all CI must pass before merge. A completed failing Ponytail verdict blocks; an interrupted turn without a verdict does not. No full current local repository matrix, live provider acceptance, multi-writer support, Cloud wake, host suspension or savings is claimed. ## Risks - Process-local hints cover the existing active writer and shared clients. Independent SQL writers or another replica need explicit wake integration. - Membership, task binding, phone preferences and other authority inputs remain checked on the existing 30-second active-call inspection deadline and on use. This is not an instant cleanup guarantee for every authority mutation. - Unknown creation retains its existing maximum-duration grace. Cleanup failures retain retries. A provider request is never replayed merely because its creation response is unknown. - Genuine configuration changes retain bounded reconstruction. Metadata writes can probe parent eligibility, while health-only observations leave voice recovery quiet. Paging state is process-local and reconstructs from durable rows after restart. - Active calls, the other voice maintenance owners and connector transports remain real obligations. This does not expand Cloud alarm coverage, relax sleep eligibility or prove deployed suspension. - The receiver and Discord parents have landed. Current-master compatibility and exact-head CI and Greptile remain merge gates. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dce0042593 |
fix(server): wake chat receiver discovery from configuration and lease deadlines (#17572)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat receivers own persistent provider connections and fenced database leases. > - The chat maintenance timer repeatedly queries all eligible receivers, including when none exist. > - Configuration changes and known lease deadlines already identify when discovery is needed. > - This pull request moves receiver discovery to the shared delivery coordinator. > - Empty or settled discovery stops querying, while ownership and recovery remain explicit. ## Linked Issues or Issue Description Refs #17557. This is the next stacked slice after Discord command registration. The base is `codex/sleep-19-discord-command-registration`. **What happened?** The recurring chat maintenance lane scans receiver inventory and reconciles Discord and Photon runtimes even when configuration has not changed. **Expected behavior** Reconstruct receiver work at startup. React to committed configuration changes and local listener retirement. Retry foreign ownership at the saved lease expiry. Stop discovery when there is no remaining obligation. **Steps to reproduce** Start an instance with no chat endpoints. Let startup finish. Observe the recurring receiver inventory query from chat maintenance. With a configured receiver, remove its ownership token while listener shutdown is delayed and signal a configuration refresh: the old implementation can replace the listener before shutdown completes. ## What Changed - Preserve healthy receiver progress after a slow failed batch while retaining per-receiver retry backoff. - Register a receiver discovery worker with scoped refreshes, paged startup inventory, fair batches, and per-receiver or scope failure backoff. - Keep owned receiver identities cached without a discovery deadline, so deletion still stops their listeners. - Use existing transaction signals for configuration, credentials, health, access revocation, company deletion, and provider lifecycle changes. Rename the generic command-only helper to describe its broader configuration contract. - Keep lease-key selection with the receiver owner. Shared scheduling contains no provider protocol. - Retain stopping ownership until listener shutdown and lease release complete. Concurrent acquisition joins the existing stop promise. - Remove receiver discovery from the periodic chat lane. Preserve voice and the other unmigrated duties. - Record producer coverage, failure evidence, limits, and remaining work in `doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md`. ## Verification Published head: `ab39569250f4b699a0ab7a829e66dbe27c15020b`. Discord #17557 has landed at master `5bf0142d45`. A normal merge reconciles its squash ancestry by applying only the recorded 27-file receiver patch from parent `547fdf2139`. Every other path matches that master. Every server and package production/test file is byte-identical to the published `6e88ca0201` source that passed the 131-case run, typecheck and build. Unrelated upstream UI/eval/documentation changes are preserved. This PR targets master. Fresh exact-head reviews and CI are required. - **131 distinct targeted cases pass.** Five complete receiver, coordinator, batching, startup, and GitHub health error-projection suites pass 99 cases. Twelve real Discord automatic-registration/receiver and Slack held-transaction cases pass separately, with 1,581 other chat integration cases deselected. The complete Photon channel suite passes 20 cases. - The standalone receiver and Photon fixtures each reproduced missing-table failures on separate initially empty PostgreSQL 17 databases. Both now apply pending migrations before client use; their complete 20-case files pass on those same databases. The large chat producer suite uses a separate migrated clone. - The old review concerns already have source fixes: discovery stops suppress their own completion wake at `1604a9fcba`; slow failed batches supply `nextRunAtOnError` at `ab56393acc`. Guarded mutations prove the existing regressions: reenabling the stop wake yields two inventory reads instead of one, and removing the error deadline fails the 1,100 ms provider-delay next-wake assertion. Both original source files are restored byte for byte. Normal full worker and real producer suites pass with those fixes. - All 131 distinct targeted cases, production server typecheck and build pass on source `6e88ca0201`. Exact source comparison carries that evidence to this normal merge; the unchanged suites were not rerun just for unrelated upstream changes. Receiver and Photon fixtures self-migrate separate initially empty databases; the large producer fixture uses its own migrated clone. Source is unchanged after these checks. Recorded listener-count warnings and controlled provider failure logs remain; no assertion was weakened. An initial whole-file identity check rejected expected parent queue additions; precise patch comparison confirms those declarations are correctly preserved. - Old server shard 12 failed because GitHub reports that its self-hosted runner lost communication. The grouped-test step is cancelled, and the retained log has no failing assertion or completed summary. The underlying host/resource/network cause is unproven. That interrupted job is not a pass; the updated parent and fixture source require a fresh full CI run. - [Current acceptance and failure record](https://github.com/paperclipai/paperclip/blob/ab39569250f4b699a0ab7a829e66dbe27c15020b/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#compatibility-after-discord-squash-merge-october-27). Prior 218-case and full-workspace results describe earlier revisions. No full local repository-suite, multi-writer deployment, Cloud wake, host suspension, or fleet cost claim is made. ## Risks - Process-local transaction hints cover the active database owner and its shared clients. Arbitrary direct SQL or independent configuration writers need explicit wake integration. Startup/resume reconstruction and known foreign-lease takeover remain supported; this does not add general multi-writer discovery. - Active receiver sockets and lease renewals continue. This change does not prove host suspension or extend Cloud alarm coverage. - Configuration signals can arrive during drain, initialization, or shutdown. The worker preserves dirty scopes and the owner joins listener retirement before replacement. - Voice, Teams, milestone projection, publication maintenance, GitHub recovery, and the remaining stack inventory are separate work. - The Discord parent has landed and the receiver child now targets master. Fresh exact-head reviews and all CI must pass before merge. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, code execution, and test tools. The runtime does not expose a more specific deployment identifier or context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7d8bab773d |
perf: avoid full JIT and activity scans in Mine unread badges (#17973)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Mine includes tasks the person created, was assigned, or participated in. > - Its unread badge ranks the visible personal top 100 before checking unread state. > - Correlated participation and privacy predicates can trigger expensive PostgreSQL JIT work. > - This change joins native participant IDs before ranking a narrow candidate projection. > - A partial actor index avoids scanning company activity history for each badge. > - Local representative scoped reads improve while dense unrestricted overhead remains explicit. ## Linked Issues or Issue Description **What existing behavior does this improve?** The Mine unread badge in the company sidebar. **Current behavior:** A fresh five-minute window recorded 402 ranking calls and 31.7 seconds of SQL time. Correlated predicates can spend more time compiling than executing. **Proposed behavior:** Build the same company-scoped participant set, use native issue keys, apply fresh authority before the top-100 limit, then retain the original canonical rank and second unread query. **Reason and benefit:** Reduce full JIT compilation and avoid an unindexed company-history activity scan. Realistic activity-only participation is included in the measurement. **Breaking changes:** No API or policy change. This PR adds one concurrent partial index migration. Related Mine rules in #16356 and #17756 are preserved. ## What Changed - Join distinct native participant IDs; guard exact legacy text before UUID conversion. - Skip redundant activity participation already covered by creation/assignment. Sort a narrow eligible candidate projection before canonical activity ranking. - Share existing pending-request and user-activity predicates with ordinary Mine reads. - Add partial `activity_log(company_id, actor_id, entity_id)` for user-authored issue activity. Register its idempotent concurrent migration with the existing top-level/invalid-index-repair runner. - Add real PostgreSQL coverage for visible top 100, unread/archive/source semantics, exact IDs, company scope and fresh grant/private-project access. - Add a pinned service benchmark with separate pre-index/indexed databases, identical data, full positional equality, ordinary-call timings, plans and real migration/cancellation proof. ## Verification - Fourteen focused PostgreSQL 17 tests pass (16.37 s), including assembled badge routes and grant revocation. Full workspace typecheck and build pass on the final production sources. - The earlier full local suite is incomplete: native PostgreSQL startup exhausted system semaphores, independently reproduced with empty initdb. All sixteen failed cases passed with task-owned Docker PostgreSQL 17. This is not counted as a full local-suite pass. Full Linux CI passes on final head `f44e46ce580e8345209d7165f57bd8bf696c517c`. Greptile and Paperclip Review are both 5/5 with zero unresolved comments. - Baseline `70c864633744cd96c0dc7394d631fc7119cd17d2`; source and DDL hashes retained. Two warmups and five alternating samples per arm/cell. Capture and EXPLAIN occur outside timed ordinary service calls. - All twelve 300k-history cells match complete positional rows in both SQL phases and final returned IDs. Half the personal tasks qualify through activity alone; the actor index contains matching and other-human rows. | Local service case | Before median | After median | Before / after | | --- | ---: | ---: | ---: | | dense / unrestricted / JIT on | 18.33 ms | 25.06 ms | 0.73x | | dense / scoped / JIT on | 909.30 ms | 221.10 ms | 4.11x | | sparse / unrestricted / JIT on | 13.64 ms | 5.84 ms | 2.34x | | sparse / scoped / JIT on | 719.07 ms | 10.79 ms | 66.64x | Small paths add at most 0.6 ms at the median. Dense unrestricted paths regress by 6.7–7.7 ms; dense scoped JIT-off is approximately neutral (218.34→223.24 ms). Sparse JIT-off paths improve too. This is not a uniform speedup. The actual baseline fails the compiler-work assertion. The final query avoids full JIT optimization/inlining and exceeds 1.5x where the old plan triggers those passes. Small/sparse controls meet the 25% or 2 ms allowance. Dense no-JIT cases are explicit diagnostics. The optional 60 ms absolute goal still fails at 221.10 ms on dense scoped reads: substantial privacy-predicate execution remains after compilation is removed. The failure is retained rather than counted as green. The actual normal migration runner upgraded 300k activity rows in 668.79 ms including inspection, index build/validation and journal work. Rerun took 21.83 ms; a deliberately cancelled build left an invalid index, and retry repaired it in 187.71 ms. Concurrent reads/writes completed; the final index was 188416 bytes. No migration was run on the hosted database. The million-row mixed-participation sparse control also passes all four visibility/JIT pairs with complete equality: unrestricted JIT-on 13.44→5.76 ms; scoped JIT-on 807.76→11.59 ms. The actor branch visits 36 indexed rows rather than scanning company history. Actual migration+runner took 255.06 ms; rerun 20.32 ms; cancellation repair/retry 241.73 ms. Concurrent reads and writes completed without failures. Reproduce: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-mine-unread-badge.ts --baseline-ref=70c864633744cd96c0dc7394d631fc7119cd17d2 --history-decoys=240000 --verify-migration --assert-performance --evidence-dir=<directory> ``` Add `--fixture=sparse --history-decoys=940000` for the million-row control. The default harness starts disposable embedded PostgreSQL and ignores DATABASE_URL. Recorded evidence used PostgreSQL 17.11 through a scratch constructor provider because the host native runtime has semaphore limits. All normal database allocation/migration/client paths are retained. Source-only EXISTS/lateral alternatives were measured and rejected after company scans or sparse regressions. A time bound would lose historical participation, and existing indexes cannot select a user's activity. This justifies the exceptional partial index migration. The exact sample ranges, plans, failures, source hashes, migration footprint and feedback-loop timing are in `doc/plans/2026-10-27-mine-unread-badge-query.md`. ## Risks - Dense unrestricted reads are slightly slower, and dense scoped reads retain substantial authority work. The PR removes a compilation/lookup bottleneck rather than all badge latency. - The concurrent build scans activity history and can wait for long-lived transactions. Local build/cancellation timings do not guarantee the duration on larger or slower hosted databases. User-authored issue activity gains one narrow index to maintain. - Participation, visibility, ranking and legacy ID semantics must remain exact; the real database regressions exercise those boundaries. - Hosted improvement is unverified. The affected ranking SQL averaged 78.8 ms while HTTP badge requests averaged 752 ms without correlated traces. No merge or deployment is included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and local code execution. The runtime does not expose the exact model ID or context window to this agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a91f78e7e0 |
perf(chat): reduce inbound queue discovery cost (#17971)
## Thinking Path > - Paperclip helps people manage companies of AI agents. > - External chat messages enter a durable delivery queue. > - Queue reads must preserve pending wake ownership and conversation order. > - Retained processed history increases the estimated cost of a correlated pending-action test. > - PostgreSQL 17 enables JIT for these small selectors and reads extra JSON in deadline discovery. > - This pull request uses a scoped pending-action set for open-work discovery. > - The benefit is less database work with the same selected deliveries and deadlines. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound delivery deadline discovery and conversation-head selection. **Subsystem affected** Server chat queue discovery. **Current behavior** A correlated `EXISTS` inside an `OR` has a high cost estimate on retained completed history. PostgreSQL 17 already hashes its pending-action read, but the estimate still enables JIT. Deadline discovery also reads JSON from completed deliveries that have no pending wake. **Proposed behavior** Use `(company_id, endpoint_id, delivery_id)` membership in the pending-action set for the two open-work selectors. Keep ready-only evaluation on its original scoped `EXISTS`. Keep retry, lease, ordering, lifecycle and endpoint rules. **Reason and benefit** The local retained-history fixture reduces deadline selection time. The small control and the final head selection are neutral. Ready-only selection remains an unchanged control. **Breaking changes** None. No API, schema, migration, stored history, worker, timer or notification changes. Related work: #17647 and #15927 change inbound wake-notice scheduling and selection. This PR changes the separate conversation queue ownership predicate. ## What Changed - Use a fully scoped pending-action set for open-work discovery. Exclude null delivery IDs to preserve exact boolean results. Keep duplicate actions from multiplying deliveries. - Preserve the original ready-only query. - Add five real PostgreSQL tests and a reproducible baseline/treatment benchmark. Capture the complete real service SQL, parameters, positional rows and execution plans. - Document the query cause, rejected hypotheses, test results and limits in `doc/plans/2026-10-27-inbound-queue-query-proof.md`. ## Verification Pinned pre-fix baseline: `09ad9adce2cf4742944353a3707a2f908e9bb35e`. Local fixture: Linux ARM64 PostgreSQL 17.11, Node 26.4.0. It has 1500 completed deliveries and actions, two open deliveries, and 32 KiB of synthetic incompressible JSON per historical delivery. A separate control has 20 completed rows. The normal settings use JIT on with a 100000 cost threshold, 128 MiB shared buffers and 4 MiB work_mem. The final quiet retained-history pair measured these local SQL medians. Each arm has two warmups and five alternating samples. The small control is neutral. | Actual selector | Baseline | Treatment | Complete positional output | | --- | ---: | ---: | --- | | Next inbound deadline | 18.78 ms | 1.58 ms | Equal | | Conversation head | 5.71 ms | 6.18 ms | Equal | | Ordinary ready control | 9.23 ms | 8.33 ms | Equal ; unchanged | The deadline plan uses fewer shared buffers. Head physical reads remain unchanged. Deadline discovery avoids JIT on this fixture. The pristine head estimate is below the JIT threshold in both arms, and its timing is neutral. With JIT disabled, deadline discovery still improves; head and ready timings overlap. - Red: the unchanged baseline fails the explicit local fixture work budget for JIT and deadline buffer reads. Green: the treatment passes. There is no timing threshold in ordinary CI. - Five new PostgreSQL tests and six existing assembled chat tests pass. They cover scoped/null/duplicate pending actions, boolean results, payloads, head barriers, reactions, paused/attention/AgentMail exclusions, retry/processing deadlines, provider ordering, lease loss, lifecycle retry and edit/admission races. - Actual old/new ready selectors have equal error outcomes for unrelated malformed retry timestamps. An initial AND-only oracle comparison was inconclusive and is not defect evidence. - `pnpm -r typecheck` passes. - `pnpm build` passes. - Final head `f3cac298d8a8a5377127b0db7057c52019bbe426` passes the complete [CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40625611887). Greptile is 5/5 on that head with no unresolved review threads. The earlier end-to-end cleanup lock-timeout failure passed on this revision. - The complete local suite was not run. The host's known native PostgreSQL startup failure exhausts SysV semaphores. Focused integration tests use a disposable Docker PostgreSQL constructor through the existing test seam. Complete final-head CI is required. Reproduce the real SQL comparison: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-inbound-queue-discovery.ts --baseline-ref=09ad9adce2cf4742944353a3707a2f908e9bb35e --require-work-budget --retained-only ``` Use `--jit-off` for the diagnostic control. The default command uses the repository's bundled PostgreSQL version and records it in output. The reported 17.11 run used the existing test constructor seam with a disposable `postgres:17` container. Raw samples, complete queries, parameters and plans are emitted as JSON. The benchmark uses disposable synthetic data and does not read a live database URL. ## Risks - Query plans depend on data and PostgreSQL settings. These local measurements do not prove deployed speed or request latency. - Head selection still reads historical JSON. Action history remains stored and scanned. This is not a constant-time queue implementation. - The changed false branch has no retry timestamp cast. Ready-only parsing, claims and retries stay on their existing query. - Background SQL totals can overlap. Do not add these savings into an HTTP latency claim. The largest possible saving for an affected selection is bounded by its observed SQL time, about 49–53 ms on average in discovery. ## Model Used OpenAI Codex, GPT-6 family, with tool use and code execution. The runtime does not expose the exact model ID or context-window size. Three parallel agents own separate performance fixes; this PR contains only the inbound queue fix. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f69dfc0e23 |
perf: use indexed latest-run probes for task interruption lists (#17969)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task lists show the latest run's interruption state for each visible task. > - The old lookup applies a coalesced text expression to company run history. > - That lookup cannot use the separate binding indexes for a small task page. > - This pull request selects the newest run through those existing indexes. > - The benefit is less database work with the same interruption result. ## Linked Issues or Issue Description **What existing behavior does this improve?** The latest-run lookup used by task-list interruption enrichment. **Current behavior** The query scans company history for a coalesced native, first-class, or legacy task ID. A local two-task fixture inspects 1,506 runs to select six candidates. A recent hosted sample recorded about 110 ms per latest-run lookup. This is one part of the list handler, not its whole latency. **Proposed behavior** Probe each existing binding index for the newest candidate. Pick the newest candidate across the three binding branches. Read the same evidence fields in the same SQL statement. Preserve native precedence, first-class precedence, legacy text matching, company scope, and ordering ties. **Reason and benefit** The representative local 100-task page improves from 17.17 ms to 3.00 ms (5.72×). A modern 100-task page with 40 runs per task improves from 26.78 ms to 3.82 ms (7.02×). All positional SQL fields, row order, and full service results match. **Breaking changes** None. Stored history, API fields, visibility filtering, execution authority, and external-conversation state are unchanged. Related behavior: #16965, #17509, #17552. The duplicate search found no open PR for this query rewrite. ROADMAP.md has no duplicate planned feature. ## What Changed - Use three disjoint indexed candidate branches with a lateral newest-run selection. - Guard UUID parameters to preserve the old text matching for noncanonical input. - Keep final evidence projection in the same database snapshot. - Add real PostgreSQL work and contract regressions, a pinned-source benchmark, and a verification note. - Require the plan regression to find real run-table nodes. Record the deep-native-history follow-up without adding indexes or DDL. ## Verification - Baseline red: the real two-task lookup inspects 1,506 run rows and fails the work bound. The unchanged semantic oracle passes. - Treatment green: both PostgreSQL query regressions pass. They cover conflicting bindings, malformed JSON, empty values, noncanonical IDs, duplicates, foreign companies, missing runs, and timestamp ties across branches. - Seven paired PostgreSQL 17.11 cases pass complete positional SQL and full service equality. Each arm has two warmups and five alternating measured samples. Fixture payload sizes are random input bytes before Base64 encoding: 24 KiB becomes 32 KiB of stored text per noise field. - Representative 100 × 12 history: 17.17 ms (16.79–17.48) → 3.00 ms (2.64–4.28). Shared buffer hits: 30,033 → 3,400. - Modern 100 × 40 history: 26.78 ms (25.79–27.40) → 3.82 ms (3.47–4.78). - The benchmark command, all cases, plans, fixture limits, and feedback-loop timing are described in [the verification note](doc/plans/2026-10-27-task-interruption-query-performance.md). - Reproduce with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-task-interruption.ts --baseline-ref=09ad9adce2cf4742944353a3707a2f908e9bb35e`. - `pnpm -r typecheck` and `pnpm build` passed at `20d46e3a16d10cf3c5399fc1211b17db6a4afdc6`. The current head has the identical production runtime; it adds one test assertion and documentation. - The focused query, real project-access repair/retry, and presentation suites pass: 19 tests across three files. The earlier plugin-sdk loading failure is resolved by normal build setup. - `pnpm test:run` was attempted, then stopped after repeated native embedded PostgreSQL bootstrap failures. The full local suite is incomplete. All 16 affected real-DB executor cases pass when rerun with disposable PostgreSQL 17 in Docker. - Linux CI passes the full server suites on this head. One unchanged Runner close fixture failed on the first CI attempt; its source tree is identical to the passing upstream #17439 baseline. One targeted job retry passed on the original head. Fresh checks run again for the test/documentation review fix. A separate macOS diagnostic fails the unchanged close variants too; that close issue is not diagnosed or counted as a local pass here. - The original head passed 55 checks with one neutral result and two expected Storybook skips. Greptile gave 5/5. The current head adds one nonempty-plan assertion and documents the deep-history follow-up. Its focused PostgreSQL suite passes 2/2. Final head `c1db106d7ab56b530af82d2bb0556feb0eda290b` is mergeable with all CI checks complete and passing, Greptile 5/5, and no unresolved review findings; the documented per-task history limitation remains an explicit follow-up. ## Risks - More index probes add about 0.43 ms to the one-run tiny control. The maximum 1,000-task page with one modern run per task is 9.99 → 14.11 ms, a 4.12 ms regression. Dense history pages save substantially more in the local fixture. - Each branch can still walk one task's own history. The first-class and legacy probes can discard native-bound rows after index lookup. The plan note specifies a 10,000-run follow-up, migration-free alternatives, and a measured index-build budget if indexes prove necessary. This PR does not make work independent of history depth. - Local SQL results do not prove deployed HTTP gains. The affected query averaged about 110 ms in the hosted sample; its critical-path contribution can be smaller because other enrichment reads run concurrently. - The main issue-list SQL and external-conversation gates remain separate work. This PR has no migration or startup job. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tool use. The runtime does not expose the exact model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
356c7de9b7 |
fix(runner): bind Grok packs to current admission source (#17972)
## Thinking Path > - Paperclip manages agents and their work. > - Native agents use verified Runner and provider-pack artifacts. > - A protected installed pack can be reused when its selected provider profile matches. > - Grok kept profile 1 when its sidecar gained effort, search, and startup diagnostics. > - An older Grok sidecar can therefore pass compatibility and reject the new session fields before provider startup. > - This PR binds the current Grok profile to its admission source so a fresh run selects the current verified pack. ## Linked Issues or Issue Description Refs: #17922, #17942, #17962. Related installation contract: #14024. No duplicate repair was found. **What happened?** The selected Grok profile digest stayed unchanged across native session contract changes. The installed-pack verifier correctly compared that digest, but the unchanged value let a pre-effort sidecar satisfy a current controller. That old parser rejects `grokEffort` and `grokDisableWebSearch` before the new startup diagnostics run. **Expected behavior** A fresh Grok run must use a pack that supports its current session contract. Source-only app changes must still permit warm reuse when the selected Grok profile matches. Reattachment must keep its exact admitted artifact authority. **Steps to reproduce** 1. Retain a protected pack with published Grok profile 1 and ACPX interface 1. 2. Upgrade the controller to the native effort and search contract. 3. Start Grok 4.7 with High effort and disabled web search. 4. The old profile passes installed-pack compatibility, but its sidecar rejects the new fields. **Paperclip version or commit** Reproduced from master at `09ad9adce2cf4742944353a3707a2f908e9bb35e`. ## What Changed - Advance Grok to profile 7. Keep historical versions 1–6 decodable and keep the Grok 1.0.13 runtime pin. - Stamp the Grok admission source identity into a generated launcher comment. Keep the existing physical command-byte digest check unchanged. Bind effort/search/credential handling, sidecar admission, and Rust backend/session/diagnostic source. - Check the source attestation in the existing TypeScript/Rust profile build gate. Keep its required fixture in the Docker context. Refuse to overwrite a versioned identity after source changes. - Add regression coverage for rejecting the old pack, current-profile warm reuse, exact historical recovery authority, and actual sidecar dispatch of Grok 4.7 High with both search policies. - Re-attest Copilot as profile 71 because its bound shared recovery decoder now retains Grok 6. Preserve Copilot 70 and all earlier identities. Keep the compatibility algorithm, other provider identities, dependencies, lockfile, and workflows unchanged. ## Verification - The production installed-pack regression failed before the profile update: the old Grok pack was accepted. It now passes. - 105 server manifest, artifact, archive, and authority tests passed. - 246 Runner profile, physical launcher integrity, Grok effort/search/credential, and actual sidecar process tests passed; six existing platform cases were skipped. - 31 Node packaging and identity tests passed, including an actual generator CLI regression that preserves all authority files when a versioned identity would change. - 55 Rust ACPX unit tests and 17 real sidecar pipe tests passed. After the final profile declaration, all 55 unit checks passed again. - Final `pnpm -r typecheck` and `pnpm build` passed at the final source. - CI on the preceding head caught Copilot’s stale attestation for the shared recovery decoder. The same current-source test failed locally, then passed after profile 71 was generated with version 70 retained. All 106 affected Copilot/profile checks pass. Final head `78e6d76fc52f492371e15574482b3641735a6d89` passed the full CI workflow; the failed prior job log is retained. - Review follow-up: mutating the Rust backend’s actual Grok descriptor field left the prior identity unchanged. This regression failed before binding `acpx_provider_backend.rs` and passes afterward. Profile 7 retains the prior versioned fixture and recovery decoder. All 179 affected Runner checks and 75 focused server pack/manifest checks passed after this change. - `pnpm test:run` hit the recorded embedded PostgreSQL bootstrap failure after five attempts. The full run also exposed a current-release fixture with the historical Grok digest; it now reads the current profile. Its 11 focused manifest tests now pass. The failed local run log is retained; final exact-head CI passed. - The same filtered-context Docker build that failed CI now passes locally after adding its required source fixture. - A separate probe used the built sidecar in a private HOME with no credentials. It admitted the current profile and the new Grok fields, then returned the closed working-directory rejection before provider launch or inference. - A historical parser probe reproduced the pre-effort sidecar rejection. These probes establish the contract boundary; they do not establish native subscription task acceptance. - Initial restricted local ownership tests failed because loopback listeners were denied. The same tests passed with loopback access. One new dispatcher fixture first lacked required instructions; the completed request passed. These setup failures are retained in local evidence. - Live native task acceptance remains pending deployment. Earlier deployed attempts failed before tool output. No successful native shell proof is claimed. - Final exact-head merge gate: all 62 contexts are clean (57 success, four intentional skips, one informational neutral formal review). Greptile and Paperclip reviews are both 5/5, with no unresolved actionable threads. The formal review uses the existing report-only neutral policy. ## Risks - The first fresh Grok run after upgrade may need to stage the current provider pack. Matching current profiles keep their existing reuse path. - Fresh admission rejects older Grok profile identities. Historical decoding and exact reattachment authority remain separate; this change does not reinterpret an old identity or replace an active session's pack. - Future changes to bound source require a new Grok profile attestation. The build gate fails if the checked-in identity is stale. - Model, effort, search, credential, approval, and retry policy are unchanged. ## Model Used - OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks; full-suite database failure is retained above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.8 |
||
|
DottaandPaperclip
|
cff331f6ca |
fix(evals): retain local controller cost origin (#17990)
## Thinking Path > - Paperclip manages AI agents and their work. > - Its eval journals retain model costs and operating reservations under one cap. > - The operating policy records the controller's execution origin. > - The schema currently accepts only GitHub-hosted and AWS Fleet origins. > - A local controller must therefore use an inaccurate label or fail admission. > - This PR adds the local origin and keeps the hosted-only schedule gate. ## Linked Issues or Issue Description Refs #17986. No duplicate open PR or issue was found in the operating-overhead/local search. **What existing behavior does this improve?** The Product E2E operating allowance and its retained journal policy. **Current behavior** At master `40c1ab36cd`, an unscheduled local controller cannot set `runner: "local"`. The generic allowance validator rejects it. Local qualification can use the same accounting controls but cannot record its origin correctly. **Proposed behavior** Accept the local origin in the generic allowance. Keep all component, price, cap, expiry, and receipt rules. The scheduled validator still requires `github-hosted` and rejects a local allowance. **Reason and benefit** Local qualification can retain an accurate execution origin without claiming that its reservations are billed GitHub charges. ## What Changed - Add `local` to the operating allowance's runner type and validator. - Exercise local journal creation, model admission, retained overhead, restart, and reconciliation in the existing boundary test. - Check that a local allowance fails Product scheduled admission. - Pin new hosted Runner controller admission to `github-hosted`. Preserve original labels when reading historical manifests; those reads cannot authorize a new worker. - Exercise the hosted-origin rejection and historical/local reads in both source and copied portable runtimes. - Document the local label and its accounting limits. ## Verification - Before the fix: the new local persistence case and schedule-boundary assertion failed; eight tests passed. - Focused ledger, overhead, operations, and scheduled finalization tests: 71 passed across five files. - Controlled Runner source and portable tests: 86 passed. The new hosted-origin regression failed before the fix. - Full Product support suite at the first commit: 4,524 passed, three skipped, and one retained embedded PostgreSQL bootstrap failure. The same database failure was previously reproduced on a clean baseline. - Separate Node test phase on the host: 137 passed. - Actual Linux controller: database lifecycle tests pass. After installing its missing `zip` utility, the full Product suite has 4,527 passes and one Copilot parent-version coverage failure. That same failure reproduces on clean master `40c1ab36cd` with the same Linux filesystem and dependencies (14 passes, one failure). Copilot files are unchanged. The initial seven ZIP-related failures and the watcher failure remain retained. - The final fresh Linux semantic check passes: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=1 node node_modules/typescript/bin/tsc --singleThreaded -p tests/runner-e2e/tsconfig.json`. The parallel compiler OOM is retained. No semantic checks were omitted. - Both fresh reviews are 5/5, with the addressed thread resolved. Hosted Build first failed during unchanged Pi setup prefetch with an operation timeout. One job rerun recovered; the original failure remains retained. All 58 latest-head checks are now acceptable. - No paid model requests, sandbox creation, schedule changes, or historical journal edits. ## Risks - The caller supplies this origin. It must describe the actual controller. - `local` does not prove zero compute cost, provide a default allowance, or release a hold. - Both maintained hosted admission paths require the GitHub-hosted controller origin. A previously accepted hosted policy with an AWS/local label now fails new admission. Historical reads preserve the original evidence. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40c1ab36cd |
feat(evals): register explicit Daytona feedback persistence (#17986)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E evals check real server, browser, runner, and provider behavior. > - Feedback persistence already checks saved complaints and suggestions with exact bodies and run attribution. > - Its catalog registers only local cells, so a Daytona campaign cannot select the same workflow. > - This PR registers the existing workflow for the two native Daytona profiles and tests their gateway bindings. > - Remote cells stay explicit-only and unqualified until a bounded live run passes. ## Linked Issues or Issue Description Refs #17980 and #17975. No duplicate open PR or issue was found in the Daytona feedback search. **What existing behavior does this improve?** The `feedback-persistence` Product E2E suite and its explicit execution selectors. **Current behavior** At master `d54f776ca0`, the suite has only local Claude and Codex cells. Selecting the matching Daytona ID fails before admission. The gateway manifest already supports these native remote profiles. **Proposed behavior** Register four cells: local and Daytona for each native profile. Reuse the existing saved-body and completed-tool oracle. Require an immutable image and Daytona credentials. Keep one attempt, one expected work run, five-minute local and inherited 15-minute remote case timeouts, and the existing 60-minute provider creation TTL. **Reason and benefit** A bounded remote qualification can check both a simple task and real feedback persistence. Local evidence remains separate from remote acceptance. **Breaking changes** Explicit `--suite feedback-persistence` now lists four cells instead of two. `--all` still excludes the suite. Daily and weekly manifests still select their existing exact local IDs. ## What Changed - Register the two native Daytona feedback cells and increment the suite definition version. - Address the remote cold-start review: inherit the base task's 15-minute Daytona deadline and retain the five-minute local limit. The regression failed before the fix and now passes. - Check immutable-image requirements, credentials, lifetime, and shared gateway connections in existing contract tests. - Fix the config type in the TTL catalog test added by #17980. Keep its runtime lifetime assertions. - Document remote selection and its unqualified status in the existing eval guides. ## Verification - Before the catalog change, the registration regression failed: one failed and 16 passed. - Focused catalog, feedback, operations, ledger, manifest, and summary tests: 114 passed across six files. - Read-only discovery lists exactly the two Daytona feedback IDs. - The Node test phase passes separately: 137 tests. The retained Vitest database failure prevents the unit launcher from reaching that phase. - Full Product support suite: 4,523 passed, three skipped, and one embedded PostgreSQL bootstrap failure. The same database failure was reproduced on a clean baseline before this PR. - Fresh Linux Product typecheck passed at the review-fix commit: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=2 node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. The default run and the ineffective JavaScript-heap attempt exited 137 and remain retained. The compiler launches a native child process; its memory controls let the unchanged semantic check complete. Host typecheck still has 15 stale installed DB/Runner export errors; the two TTL test typing errors are fixed. All 58 latest-head checks are acceptable (success, neutral, or skipped). Both fresh reviews are 5/5, with no unresolved review threads. - The updated credential-free Linux controller discovers both planned remote IDs and launches Chromium 151.0.7922.34. Runtime build inputs are unchanged from the prior controller. The Daytona image content ID is still `b55f03dc9c633db300b70206a691ff45a8e779e56a3c546f9358e3282d3597c9`. - No model requests, sandbox allocation, or paid schedule changes. ## Risks - Selecting the whole explicit suite now includes remote cells and requires Daytona configuration. Use exact IDs for a bounded paid campaign. - Registration and contract tests do not prove live remote behavior or token accounting. Remote qualification remains pending. - The shared dependency checkout limits the local host typecheck. The fresh Linux semantic check passed. Earlier compiler OOM failures and the database bootstrap failure remain retained. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>nightly/v2026.1028.0-nightly.0 canary/v2026.1028.0-canary.7 |
||
|
DottaandPaperclip
|
d54f776ca0 |
fix(daytona): bound ephemeral eval lifetime at creation (#17980)
## Thinking Path
> - Paperclip manages agent work through Runner and sandbox providers.
> - Product E2E runs paid work in disposable Daytona sandboxes.
> - A controller deadline cannot stop remote compute after the
controller crashes.
> - Inactivity cleanup does not establish an independent wall-clock
lifetime.
> - Daytona supports a lifetime in the creation request.
> - This change exposes that option in the Daytona driver and verifies
the provider expiry.
> - Ephemeral evals can retain a finite lifetime even when their
controller is unavailable.
## Linked Issues or Issue Description
Refs #17947 and #17978. No duplicate creation-lifetime PR was found.
**What happened?**
The ephemeral Product E2E fixture configures inactivity stop, archive,
and delete intervals. Ordinary task leases do not supply a requested
expiry. A crashed controller therefore has no confirmed wall-clock
sandbox lifetime for its cost bound.
**Expected behavior**
Each ephemeral eval sandbox must carry a provider-enforced lifetime from
creation. The driver must reject invalid bounds and reject a sandbox
whose provider expiry is missing, expired, or later than the configured
remaining lifetime.
**Steps to reproduce**
Acquire a Daytona lease from the ephemeral fixture. Inspect the creation
request and returned expiry. Before this patch, the request has no
lifetime and the lease has no expiry unless its caller supplies one.
**Paperclip version or commit**
Current source based on master at
canary/v2026.1028.0-canary.6
|
||
|
DottaandPaperclip
|
5bf0142d45 |
fix(server): schedule Discord command registration from saved deadlines (#17557)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connections register and verify provider commands in the background. > - Discord command recovery ran inside the recurring Gateway runtime scan. > - Its existing durable records already store verification and retry deadlines. > - This pull request gives command registration its own shared-coordinator worker and scoped commit notifications. > - Idle command work stops querying between real deadlines. Existing command authority and recovery remain intact. ## Linked Issues or Issue Description Refs #17554 and #17561. Both foundations have merged. This slice targets master. Runtime discovery and the other chat maintenance duties remain separate work. **What happened?** The chat runtime maintenance loop called Discord command registration on every reconciliation. It repeatedly read durable command state even when the next remote verification or retry was minutes away. **Expected behavior** Use committed producer changes and saved deadlines. Keep the existing five-minute verification policy, Retry-After, uncertain-write quarantine, namespace ownership, and credential fences. Do not add a database table for the timer. **Steps to reproduce** Configure a Discord connection and let command registration settle. In the parent revision, repeated runtime reconciliation still invokes command recovery. This revision reconstructs command deadlines at startup and removes that call from runtime reconciliation. ## What Changed - Expose normalized deadlines from the existing registration store. Park indefinite retries and invalid or foreign-owned records. Signal before store-owned claims, receipts, and retry-result writes. - Discover eligible endpoints in bounded pages. Refresh only affected endpoint, connection, or company scopes after commits. Dispatch at most 25 due commands per pass with fair company rotation. - Wire configuration, credentials, provider lifecycle/failure paths, inbound activation, generic connection mutation/removal/health, access revocation, and company deletion. - Register the worker through the chat service and shared coordinator. Preserve direct foreground setup calls. Refresh an existing local Gateway through its current owner after command work. - Preserve dirty work during drain and concurrent changes, back off unresolved attempts, and join admitted provider work during shutdown. - Document producer coverage, retained deadlines, supported writers, verification failures, and remaining runtime/Cloud obligations in `doc/idle-discord-command-registration.md` and its dated audit. ## Verification Published head: `547fdf2139144e1e6484198278bf4bb0eade8794`. It normally integrates master at `f8563d25077b17681ab6ef5949f7b07415a58ede`, including landed health #17554 and company deletion #17561. Squash ancestry caused merge conflicts; resolution applied this slice's exact 26-file owned patch from its recorded parent to pinned master. Every other path matches master. Migration 0394 and the current lockfile are preserved. - **141 distinct targeted cases pass** on reconciled production source: 78 complete command-store, batching, coordinator and startup-timing cases; 8 real automatic Discord and held-transaction Slack cases; and 55 complete startup/export and company-deletion cases. The 1,581 other chat integration cases were deselected. - Server typecheck and build pass. The company-deletion fixture self-migrates an initially empty isolated database. The two Discord fixtures use separate migrated clones; this is not a fresh-schema claim for those fixtures. Signal-listener warnings remain recorded without assertion failures. - The parent integration preserves health and command notifications at shared producer sites. It changes no provider behavior, schema or coordinator contract. Prior producer, backlog, drain/resume, malformed-hint, transaction, namespace, shutdown and retry regressions remain in the complete command suite. - [Landing compatibility acceptance](https://github.com/paperclipai/paperclip/blob/547fdf2139144e1e6484198278bf4bb0eade8794/doc/idle-discord-command-registration.md#acceptance-after-the-company-deletion-parent-landed-october-27) records current scope and the earlier failures. Older full-matrix or combined-stack runs apply only to their stated revisions. - Fresh exact-head Greptile, Ponytail if present and all CI are required. No full local repository suite, live provider, Cloud alarm, host suspension or savings claim is made. ## Risks - Discord still has real five-minute verification and failure-retry obligations. This does not prove host sleep or external alarm coverage. - Runtime discovery, receipt-to-runtime projection, Gateway lease renewal and persistent sockets remain separate. The remaining runtime poll can still read command receipts; it no longer invokes the command protocol. - Process-local signals cover integrated writers. Independent SQL writers and other replicas require an explicit wake contract. Existing authorization and durable ownership checks remain authoritative. - Refreshes coalesce with a 30-second minimum spacing. Saved work is never discarded because a call returned without progress. - No schema migration, new table, workflow or lockfile change is introduced by this slice. Landed parent repairs are retained. The receiver child must receive this integration before landing. ## Model Used OpenAI Codex, GPT-6 family, with repository tools, code execution, and reasoning. The exact serving version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4c314117d2 |
fix(evals): bind active image identity and isolate routing checks (#17978)
## Thinking Path > - Paperclip manages agent work through Runner. > - Product E2E uses a content hash to identify its Daytona image. > - The hash must include the active provider identity declaration. > - The active Copilot fixture advanced past the manually listed image inputs. > - The runner routing test also captured later YAML steps as shell commands. > - This change binds the active fixture and stops shell extraction at the next step. > - Operators can verify image identity and routing against the current source. ## Linked Issues or Issue Description Refs #17937 and #17947. The related Copilot work includes #15563. This change repairs the shared image contract; it does not add a Copilot qualification claim. **What happened?** Two image tests fail because the active Copilot identity fixture is absent from the image hash inputs. Four routing tests execute later workflow YAML as shell commands after checkout and dependency steps were added to the authorization job. **Expected behavior** The image hash must change when the active identity fixture changes. The routing tests must execute only the selected routing step. **Steps to reproduce** Run the Daytona image and paid runner routing unit files on current master. Six assertions fail before this patch. The same failures occur on the clean |
||
|
|
39279aa34f |
feat(ui): shared error, connectivity and query foundation for disconnects (#15754)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board reads API data and keeps the current task on screen. > - A restart or temporary network failure can interrupt those reads. > - Separate page error handlers can hide cached data or expose raw server codes. > - The app needs one connection state, readable error copy and a consistent read policy. > - This change provides that foundation and moves affected read surfaces onto it. > - The dependent write layer in #17960 preserves edits and submissions across the same failures. ## Linked Issues or Issue Description **What happened?** Temporary API failures could replace loaded content with errors. Different surfaces handled the same outage differently. Some showed raw server codes. A successful query to an unrelated service could also falsely report that Paperclip had recovered. **Expected behavior** Keep cached content visible. Show one connection banner during sustained trouble. Confirm recovery through the Paperclip health probe or its WebSocket. Retry transient reads and show readable errors for permanent failures. **Steps to reproduce** 1. Open a populated task or workspace page. 2. Interrupt API requests with a 503 or a dropped connection. 3. Keep an unrelated external request in flight. 4. Confirm that cached content remains and that the external result cannot clear the outage. 5. Restore the API and check that reads refresh. The write layer first merged into this feature branch in #15769 now has its own dependent PR, #17960. This PR contains the shared foundation and read changes. Merge this PR first. ## What Changed - Classify API errors and provide readable error copy, including non-JSON proxy responses and Retry-After handling. - Use one connection store and health probe with bounded backoff. Confirm recovery from the server health probe or live-updates WebSocket. - Pause reads while disconnected, retry transient errors, and refresh active reads after recovery. - Keep cached data visible through shared query views and affected page, panel and plugin read surfaces. - Add one app-level connection banner and support the pending-write count consumed by the dependent write layer. - Add a query-error rendering guard. Call it directly from the existing CI step. Update development documentation. ## Verification - Historical-chat lookup now follows the same ready/stale roster state as rendering. Three regressions fail before the fix; the six new cases pass after it, including working Retry, fresh data, no-cache error, permanent 403 and company isolation. All six focused query/recovery suites pass 200 tests. Final-commit CI passes and Apex is 5/5 with no unresolved findings. Canary Dry Run initially refused a preview version that was already on npm. It passed after one rerun with fresh registry metadata; no check was disabled. - Integrated current master and retained both transcript interest registration and connectivity reporting on socket open. The seven focused recovery/query suites pass 288 tests, and UI typecheck and production build pass. All 9,451 UI tests across 799 files pass on the merge commit. Final-commit CI passes and Apex is 5/5 with no unresolved findings. - Current head `70d9335d732219352ef0f98414439766bbcb33df`: all hosted CI checks pass. Apex is **5/5** on that head with no unresolved findings. The branch has no merge conflict. GitHub still requires code-owner approval. - The foundation passes 9,430 UI tests in 804 files. After integrating current master page paths, 190 focused page tests pass. - Two unrelated-success regressions fail before the latest fix. With the fix, 62 focused UI tests, all six guard tests and UI TypeScript pass. - The composed foundation and write layer pass **9,505 UI tests in 808 files**, UI TypeScript, a production UI build, and query-error, token and map guards. - A real Chromium journey on the composed source injects a 503 during title save. The loaded task stays visible; one reconnect banner appears; the edit saves after recovery; another task stays unchanged; the saved title survives reload. - Full recursive local checks were not repeated because Runner compilation exhausted local disk during another repair. Hosted checks must pass before merge. No new staging deployment is claimed. - Latest review follow-up: a cached CaseDetail now survives a failed HTTP 429 refresh. Its page regression fails before the fix and passes after it. Eighty focused foundation tests, UI TypeScript, build and guards pass after that change. - Earlier CI evidence is retained: several foundation jobs received a runner shutdown signal together. The write-layer head also failed an unchanged Runner descriptor close test with `provider_transport_failed`. The same Runner source passed that suite in the foundation run. New CI runs cover the updated heads; no check or assertion was weakened. ## Risks - Shared retry defaults change the time before transient read errors appear. Permanent errors still fail promptly. - ApiError.message now contains readable copy. Callers that compare raw codes must use ApiError.code. - Cache success can come from another service or local data. It must not confirm Paperclip recovery. - Durable writes are reviewed separately in #17960. This PR does not change database or mutation contracts. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) produced the original work with tool use. OpenAI Codex, GPT-6 family, assisted with restoration, integration, tests and review fixes. The Codex session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Amy (Opus) <claude@users.noreply.github.com> |
||
|
DottaandPaperclip
|
9b7d5b35eb |
fix(evals): refresh forecasts from complete request receipts (#17975)
## Thinking Path
> - Paperclip coordinates agent work through Runner.
> - Scheduled Product E2E campaigns need measured cost and time
forecasts.
> - The saved forecast samples came from October 21.
> - The October 27 qualifications retain final usage for all 19 model
requests.
> - The Claude measurements are hosted; the Codex measurements are
local.
> - This change refreshes the samples and keeps measurement hosts
separate.
> - Operators can inspect the latest evidence without treating it as
current qualification or an invoice.
## Linked Issues or Issue Description
Refs #17127 and #17946.
**What happened?**
The scheduled forecast still uses the older pilot amounts. It reports no
hosted measurement and does not record the measurement host per sample.
**What did you expect to happen?**
The forecast must retain the latest complete model receipts and their
source hashes. It must keep local and hosted timing cohorts separate. A
changed definition must remain unqualified.
**Steps to reproduce**
Prepare the daily or weekly forecast from the default branch. Inspect
its saved measurement date, totals, and hosted fields.
## What Changed
- Refresh four samples from two intact request ledgers and four retained
Product results.
- Record measurement hosts, final request counts, and original campaign
outcomes.
- Add historical hosted totals. Current hosted totals still require
matching evidence for every selected cell.
- Keep hosts in separate per-cell cohorts. Preserve definition drift and
unknown slow-tail estimates.
- Document the model-cost scope and timing limits.
## Verification
- Forecast and scheduled-preflight tests: 33 passed.
- The extraction replays both hash-checked ledgers, requires final
receipts, and checks the retained pricecards. The four samples cover 19
final requests.
- Daily historical model subtotal: $0.428650100; summed attempt time:
107.286 seconds.
- Weekly historical model subtotal: $0.793676500; summed attempt time:
158.093 seconds. The Codex samples are local.
- Full Product support suite at the initial head: 4,510 passed, seven
failed, three skipped. A clean
canary/v2026.1028.0-canary.5
|
||
|
DottaandPaperclip
|
f8563d2507 |
fix(server): remove chat endpoints before company agents (#17561)
## Thinking Path > - Paperclip manages agent companies and their connected chat channels. > - A company deletion must remove dependent data in one transaction. > - Chat endpoints reference assigned agents with a restrictive foreign key. > - The current deletion order purges agents before those endpoints. > - This pull request removes company endpoint rows and chat history before issue and agent deletion. > - The existing lifecycle checks, company isolation, and rollback remain intact. ## Linked Issues or Issue Description **What happened?** Deleting a company with a chat endpoint fails on the endpoint's assigned-agent foreign key. The failure was found while verifying the company-deletion wake in #17557. **Expected behavior** Company deletion removes endpoint dependencies before their referenced issues and agents. It preserves other companies and global Discord ownership tombstones. A later failure rolls back the cleanup. **Steps to reproduce** Create a company, an agent, and a Discord endpoint assigned to that agent. Complete agent termination, then delete the company. The old order fails with the restrictive endpoint-to-agent foreign key. **Paperclip version or commit** This prerequisite includes merged #17554 at `94a780286f` and supports #17557. Only four cleanup-owned paths differ from that landed parent. Related #16465 covers the deletion feature flag, not dependency order. ## What Changed - Add a chat-owned company-deletion participant using the existing transaction contract. - Remove only the requested company's endpoints before issue deletion and the agent lifecycle participant. - Preserve namespace tombstones and existing restrictive foreign keys. - Add real PostgreSQL regressions for ordering, isolation, and rollback. - Migrate a validated disposable external database before fixture use, use the shared PostgreSQL bootstrap budget for setup, and remove fixture-owned Discord tombstones during test teardown. ## Verification Published head: `9766cc61c42fe0f73e241bbe99f9d9f145f4b11c`. Parent #17554 is merged. This branch reconciles the actual landed health squash `94a780286f` and targets master. Fresh reviews and CI are required for this exact head. - The latest review requested the repository's shared EMBEDDED_POSTGRES_TEST_TIMEOUT_MS (90 seconds) for bootstrap. This replaces the earlier suggested 30-second limit. All six existing cases pass on another initially empty database with that constant; no test assertion or production code changes. The documented contention budget is repository evidence, not a locally reproduced 40-second bootstrap failure. - All six complete cleanup-removal service cases pass on an initially empty dedicated PostgreSQL 17 database with the updated health parent. - The pre-fix real endpoint-deletion case failed with missing-table errors because external fixture setup skipped migrations. Applying pending migrations before client use fixes that failure; no assertion is removed or weakened. - An independent query after the complete suite finds zero fixture ownership tombstones. Production company deletion still preserves those records. - Cases cover endpoint removal before issue and agent references, other-company isolation, namespace ownership preservation, and rollback after a later failure. Both product regressions failed before the original ordering fix. - Server typecheck and build pass with the updated parent and fixture repair at `21be75d2b0`. The landed-parent reconciliation changes only the lockfile; all source and test bytes remain identical. All six cases pass again with the new installed dependency patch. The initial scope-proof command raced a shared `origin/master` update; a new proof pins the actual landed parent and confirms the four-file scope. A later native-checkpoint merge `09ad9adce2` has no overlap with these four files. - [Owning acceptance document](https://github.com/paperclipai/paperclip/blob/9766cc61c42fe0f73e241bbe99f9d9f145f4b11c/doc/company-chat-deletion.md#current-parent-acceptance-october-27) retains the failing baseline, parent scope, and limitations. No full local repository-suite or deployed suspension claim is made. Older inherited gateway/combined-stack evidence is separate from this four-file fix. ## Risks - The endpoint cleanup shares the caller's transaction. A later deletion failure must restore the endpoint; the regression verifies that boundary. - Existing retained voice-history restrictions and external provider teardown policy remain unchanged. This fix does not bypass restrictive foreign keys. - No migrations, lockfile changes, or workflow edits. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code edits, and tool-assisted tests. The exact runtime variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
SudheerandClaude Opus 5.5
|
70c8646337 |
fix(ui): drop top margin and border on first contextual sidebar section (#17412)
The first section in the contextual sidebar rendered a leading top margin and divider with nothing above it. Use first: variants so only sections that follow another section get the spacing and border. <!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI has contextual sidebars, such as Skills, Apps, and Agent > - These sidebars share one set of style constants in `contextual-sidebar-styles.ts` > - Each section in the sidebar has a top margin and a top border to separate it from the section above > - The first section has no section above it, but it still shows the margin and the border > - This pull request adds `first:` variants to the section style > - The first section now sits flush at the top, and the other sections keep their divider ## Linked Issues or Issue Description **What existing behavior does this improve?** The section style in the contextual sidebar. It adds a top margin and a top border to every section. **Subsystem affected** UI (`ui/src/components/contextual-sidebar-styles.ts`) **Current behavior** The first section in the sidebar shows a top margin and a divider line. Nothing is above it, so the divider looks wrong and wastes space. **Proposed behavior** The first section has no top margin and no top border. Each later section keeps its margin and divider. **Reason and benefit** The sidebar looks cleaner at the top. The change is small and uses only existing design tokens. **Breaking changes** None. ## What Changed - Added `first:mt-0 first:border-t-0` to `contextualSidebarStyles.section` in `ui/src/components/contextual-sidebar-styles.ts`. Before: <img width="585" height="410" alt="Screenshot 2026-10-24 at 10 14 36 AM" src="https://github.com/user-attachments/assets/12c35f5f-7993-4fdb-89b2-df3394859bf8" /> After: <img width="560" height="377" alt="Screenshot 2026-10-24 at 10 14 24 AM" src="https://github.com/user-attachments/assets/92e1f7ab-f4c0-48ac-832a-a3726770deac" /> ## Verification - Run `pnpm check:token-gates`. It passes. - Open a page that uses a contextual sidebar, such as Skills or Apps. - Check that the first section has no top gap and no divider. - Check that the later sections still show their divider. ## Risks - Low risk. The change affects only the spacing and the top border of the first sectio - Several sidebars share this style, so check each one: Apps, Skills, Agent, and Chat detail. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: Anthropic - Model: Claude Opus 5.5 (`claude-opus-5-5`) made the commit. Claude Sonnet 5.5 (`claude-sonnet-5-5`) helped review the change afterward. - Capabilities: tool use and code execution in Claude Code (CLI). - The one-line style change is human-authored. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only puub.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
c9213d1be4 |
fix: send saved queued messages after Stop (#17959)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People can save follow-up messages while an agent runs. > - Stop preserves those messages and stops the active execution. > - The queue removed its delivery controls when that execution ended. > - Native admission also refused an explicit resend of older queued input. > - This change adds Send for a stopped queue and records fresh board intent. > - The saved message can continue through normal cleanup and admission checks. ## Linked Issues or Issue Description **What happened?** A person queued a message with a screenshot while an agent was starting. Steering was not yet available. They stopped the run. The message remained visible, but no control could send it. Native queue delivery also rejected this transition. **Expected behavior** A stopped queue offers Send. The click preserves message IDs, content, attachments and original authors. It starts at most one eligible continuation. Stop remains effective until the person gives new direction. **Steps to reproduce** 1. Start a native task and queue a second message. 2. Stop the active run before it consumes that message. 3. Open the task after cleanup. Observe the saved message with no send control. Refs #16965 for the queue-control change and #15015 for stopped conversation recovery. No open repair for this exact transition was found. ## What Changed - Show Send for a saved deferred queue with no active run. - Accept the existing board queue action for a stopped native target. Keep live native interruption restricted. - Use the server-recorded click time to authorize older native messages. Reject stale, malformed and wrong-actor intents. - Reuse normal queue dispatch, revision checks, cleanup proofs and execution admission, including a failed native predecessor after a verified Send. - Ignore a fully committed, successful recovered export when admitting a fresh message, while retaining the wait for results still committing. - Honor an applied `/new` conversation boundary while retaining native execution holds until the existing stop and cleanup checks verify them. Workspace holds also survive reset. Check the newest failed or cancelled native execution even when older recovery actions exist. - Keep native automatic delivery separate from the explicit Send action. Update execution and access documentation. ## Verification - Meaningful regressions failed before the fixes: stopped queues lacked Send, native admission refused fresh intent, cleanup and dispatch receipts were missing, deferred rows disappeared, live plan Interrupt rejected its own receipt, and Agent Chat `/new` inherited an old non-workspace execution hold. - Six assembled-wakeup cases fail before the reset cleanup fix. All 11 reset/history cases pass after it: live processes, missing identity, quarantined cleanup, remote leases and live controllers remain held; verified cleanup proceeds and retires the hold. Two more cases fail before the fallback fix and pass after it: Send verifies a stopped or live native predecessor despite a later reset receipt and no recovery record. - **525 server tests pass** across seven queue admission, route, dispatch, envelope, provenance, blocker and conversation files. **343 queue/thread UI tests** also pass. Server/UI TypeScript, token and map guards pass. The final scoped server typecheck also builds native Runner successfully. - The latest regressions prove that completed recovered exports permit fresh input, and clearing conversation history still requires old native stop and cleanup verification; an unsafe workspace still blocks after `/new`; failed native Send reaches actual continuation dispatch with the original queued comment. A result still being committed remains deferred. History plus a newer Stop cannot hide a missing process/controller/lease receipt. - The real Chromium Stop/Send journey passes on current head `75fa5e7222c59f43ff7ac326e81093bc9da28330`: queue a follow-up, Stop, reload, explicitly Send, then verify exactly one useful response and no live run. Earlier browser journeys cover legacy Interrupt, interruption during a write and Stop followed by a new direction. These use an isolated app/database and controlled ACP fixture, not an external AI account or a staging deployment. - Negative cases retain actor, company, issue, assignee, queue and cancellation-receipt bindings; cleanup/process/workspace/controller holds; approvals; pause and budget gates. Typed responses keep their original resolver and immutable result. - Full local recursive typecheck stopped in unchanged Rust Runner compilation with ENOSPC, so the chained full tests/build did not run. All hosted CI gates pass on final commit `75fa5e7222c59f43ff7ac326e81093bc9da28330`, including typecheck, tests, build and E2E. Apex review on that commit is 5/5, with no unresolved findings. Earlier unrelated CI failures involved PostgreSQL fixture-cleanup deadlock and an unchanged decision-expiry deadline assertion; the latter's 20 cases pass locally without source changes. No assertions or limits were weakened. Local resource cleanup was limited to this task's verified completed test processes and rebuildable npm caches. ## Risks - Send can authorize a new turn for a saved message after Stop. A Send click must be newer than the stopped execution; a live plan Interrupt is bound to the cancellation receipt produced by that same click. It does not imply that cleanup finished or that an external action is safe to replay. - The existing endpoint and intent format are reused. No schema or credential change is required. Live native steering and legacy interruption keep their existing paths. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose the exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
6102f8e82c |
fix: pass healthy managed Runner connection tests (#17965)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board tests an agent's selected runtime and AI account. > - Cloud prepares and verifies the managed Runner runtime before execution. > - Its remote ACPX preflight reports that deferred step as a warning. > - The test keeps the warning after the selected account passes its hello probe. > - The board then asks the user to fix a healthy connection. > - This change makes that specific managed step informational after account verification. ## Linked Issues or Issue Description **What happened?** A Cloud agent's Claude connection test passes with Legacy. With Runner, the same account completes the Claude hello probe but shows "Connection needs attention." The only warning says that remote runtime preparation must occur before launch. **Expected behavior** A verified managed connection passes when its only warning concerns runtime preparation that Paperclip performs before execution. Failed authentication and actionable runtime checks retain their results. **Steps to reproduce** 1. Select a working Claude subscription on a Cloud-managed sandbox. 2. Select Runner and run the connection test. 3. Observe a successful Claude hello probe and a warning result. Related: #17961 covers managed OpenRouter model verification. #15880 adds persistent computer runtime interfaces. Neither changes this Cloud connection-test status. ## What Changed - Recognize the deferred ACPX runtime check after a provider hello or live API-key verification succeeds. - Report that check as informational for Cloud-managed sandbox connections. - Preserve every other warning and error, including unexplained warning results. - Add service regressions and document the test/launch boundary. ## Verification - Red: the healthy managed-Claude case returned `warn` after a successful hello. The regression fails before the change and passes after it. - **116 tests pass** across connection verdicts, environment-test HTTP routes and adapter registration. Scoped server TypeScript and the server build pass, including the native Runner build. - Negative cases cover rejected credentials, missing hello receipts, missing runtime packages, actionable or unexplained warnings, self-hosted instances, SSH targets and unexplained provider warning results. API-key success requires live provider re-verification. - Operator evidence confirms that the reported test lease was released with cleanup successful. The agent's saved adapter is Legacy. No saved account, agent or environment setting was changed. - A clean local install hit a frozen-lockfile `patchedDependencies` configuration mismatch. The scoped checks reuse installed workspace dependencies. The final commit has passing hosted CI, including clean-install typecheck, tests and build, and Apex 5/5 with no unresolved findings. - A cold HTTP-suite module load reached the 30-second setup limit while other checks ran. All 54 HTTP cases pass on retry, and the full focused run passes 116 tests without changing assertions or time limits. - No staging deployment is claimed. This change classifies the connection-test result; it does not claim that a native Runner hello turn was performed. ## Risks - This status change is limited to a verified Cloud-managed sandbox ACPX connection and the exact deferred check code. The provider's successful receipt is required. - Runtime package preparation and verification still execute before agent work. A connection test does not replace those launch checks. - There is no schema, permission or credential change. Local, SSH and plugin task probes keep their existing verdicts. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6cc765bd81 |
feat(live): deliver recoverable transcripts to interested views (#17456)
## Thinking Path > - Paperclip should show live agent output without repeating presentation work for every view. > - Execution, acknowledgements and durable checkpoints must remain independent of browser interest. > - The server already owns authorization and redaction at delivery. > - A shared browser connection can declare which visible runs need full transcripts. > - Catch-up must handle subscription races, disconnects, terminal output and slow consumers. > - This PR delivers the backend and UI contract together without a database migration. ## Linked Issues or Issue Description **What existing behavior does this improve?** Visible transcript consumers receive recoverable output through one shared browser connection. **Current behavior** Independent views can create duplicate connections, presentation work and recurring history reads. A subscription race or filtered native history can lose the saved tail. **Proposed behavior** Negotiate run interests, subscribe before catch-up and retain independent canonical recovery cursors. Keep lightweight lifecycle and attention events available without full transcript delivery. PR 4 of #17097 → #17439 → #17455 → #17456. The runtime feature is independent of hot-upgrade recovery; the stack order is for review convenience. ## What Changed - Add additive WebSocket capability negotiation, replacement interests, revision acknowledgement and resynchronization notices. - Reference-count visible consumer interests on one company-scoped browser connection. Preserve older-client/server behavior. - Reuse canonical structural/current-user/registered-secret redaction for plugin run notifications through the same bounded delivery queue. Keep producer and uninterested streams free of recipient lookups. - Refresh application queries after the matching recovery acknowledgement, including one follow-up after an older in-flight read. Ordinary interest acknowledgements add no refresh. - Keep authorization and redaction at delivery. Browser interest cannot change harness execution, PRP ACKs, checkpoints, accounting or canonical logs. - Deduplicate by sequence and log identity. Catch up terminal output and saved tails without recurring reads during healthy negotiated streaming. - Bound delivery to 1,000 items and 1 MiB. Overflow requires resynchronization; persistent congestion closes the connection. - Preserve compact-history expansion from current master across paginated catch-up. Snapshot the original read window before React can defer its update. ## Verification On current source `f6710f0b3c`, Linux [job 122604880503](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880503) executes all five run-read permission cases successfully (2.574 seconds), including grant and revocation on the same connection. Its shard passes all 2,430 cases. Current [server-shard job 122604880416](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880416) has 1,513 passes, one failure and three skips: an unchanged nested-clone expiration/restore test exceeds its 15-second deadline. The preceding `5efc6c48bf` source has identical non-document bytes; [its corresponding job 122601359272](https://github.com/paperclipai/paperclip/actions/runs/40623017917/job/122601359272) passes all 21 nested-clone cases, including that case in 9.505 seconds. The exact timing cause is unproven. The single diagnostic [retry job 122609332597](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122609332597) passes all 21 nested-clone cases (61.316 seconds), including the formerly timed-out case in 7.003 seconds, and all 1,514 shard cases (three skips). The failure remains recorded; no timeout or assertion was changed. After the server squash merge and qualification retarget, all 50 focused server cases and all 91 focused UI cases pass again, with both typechecks and token gates. Runner vendor preparation rebuilds current master inputs before those tests. This source needs fresh CI and reviews; prior CI and browser receipts do not qualify it by inference. Current candidate `f6710f0b3c` now targets master `6987c2f20b`, after merged #17097, #17439 and #17455. Its diff contains only transcript delivery and its documentation. It adds no database migration. Retargeting changes only the evidence document; all production and test code is byte-identical to `5efc6c48bf`. On the eighth integration, all 50 focused server delivery cases and all 91 focused UI cases pass, with both typechecks and token gates. The following 35-case receipt belongs to the earlier delivery correction. Four low-queue authorization/decoration stall variants and the late-rejection case fail before correction. They now verify the five-second read deadline, old and negotiated clients, revisions that cannot reset the deadline, unaffected sibling connections, rejected late results, and timer cleanup. The first combined toast run has four loopback EPERM failures and 31 passes; its permitted repeat passes all 35. Earlier 28-case evidence excluded the toast file. The permission correction requires the current HTTP company-scope transcript permission, plus task access, for both native events and legacy logs. Prior Linux source `49a8783e9b`, job 122549913643, executes all five run-read cases, including grant/revocation on the same connection, successfully. That source placed this file in server shard 11; later shard assignments can differ and must be checked against their actual inventories. The older `405bedd189` receipt retains a separate scheduled-retry teardown race and inherited fixture collision; neither is waived. The new head must pass again in Linux CI. The seven extracted stream-recovery tests preserve their assertions. All 39 related legacy/native cases, UI typecheck and token gates pass after restoring the initially missed fixture helper. Earlier full workspace build/typecheck and Chrome journeys at `445a76563c` cover two views, company switches, reconnect, terminal catch-up, reload and zero recurring healthy transcript reads. Those browser receipts are historical source evidence and do not qualify this new head by inference. The [qualification record](https://github.com/paperclipai/paperclip/blob/f6710f0b3c84c6fdb2021df67206261b446dfc97/doc/plans/2026-10-22-hot-upgrade-qualification.md) retains exact earlier UI/history, source-boundary and cleanup failures with their fixes and limits. The workload-cost test preserves bounds and authorization callback counts for zero interests, one viewer, multiple browsers and a slow viewer. Its socket congestion is simulated. It measures neither SQL nor network costs. Current permission checks and the final hosted active-agent/viewer matrix remain unmeasured; no production savings are claimed. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and ordinary merge eligibility remain required. The earlier recipient corrections pass all 47 registry/plugin/delivery/WebSocket/toast cases and all 174 UI provider/recovery/interest cases, both typechecks and token gates on source `b202e59a25`. A real bus/host-service test registers a synthetic run secret through the production registry and reproduces its raw notification before the fix; native/legacy/final notifications now redact it, disposal prevents late worker notification. A later P1 review reproduces lost final notifications on settings-read and secret-resolution failure. Both cases fail before correction. Any delivery failure or subscription timeout now sends one content-free stream error and removes the plugin subscription. It never changes the agent execution or run status. Later events start no new redaction reads. The final integrated six-file server lane passes all 50 cases, with server typecheck and token gates. Existing browser delivery behavior is unchanged. Storage and secret-provider seams are scripted. Two real React Query/provider regressions first miss a run during resync; both now refresh after the valid replacement ACK, including pending-read and stale-scope cases. The prerequisite qualification batch passes all 118 cases and E2E typecheck after master integration. Fresh CI, security scan and reviews remain pending on this head. The contribution-rule review correction removes 104 explanatory comments introduced by this slice and keeps the ordering, deadline and plugin-delivery explanations in the delivery document. Existing comments and functional test/lint directives remain. All edited source files have identical parsed non-comment tokens before and after the change. All 50 server delivery cases and all 91 cases in six affected UI suites pass, along with token gates. The new head requires fresh CI, security and review. ## Risks A slow or disconnected viewer must resynchronize from authorized canonical history. It may see a display gap until catch-up completes. Company changes and revoked access must remove delivery interests. This slice adds no database migration and cannot change execution or recovery authority. Full transcript delivery evaluates current company-scope permission per delivery. Active plugin recipients also perform canonical redaction reads; viewer-load database costs on this final source remain unmeasured. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.4 |
||
|
DottaandPaperclip
|
f52da40f08 |
fix: resume queued turns after maintenance (#17967)
Keep an accepted turn's starting input in task history, show its admission wait, and resume existing queued runs promptly when maintenance ends or expires. Verified with behavioral red/green regressions, real PostgreSQL and process execution, browser/native Runner acceptance with a deterministic provider, workspace typecheck, production build, and current-head CI/review. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6b9e5f3a80 |
fix(evals): enforce qualified ACPX profile admission before paid turns (#17964)
## Thinking Path
> - Paperclip coordinates agent work through Runner.
> - Controlled evals route model calls through a budget gateway.
> - The definitions pin each ACPX runtime identity.
> - A stale qualified profile currently reaches a paid turn before
scoring rejects it.
> - The Runner CLI already checks an exact expected profile before it
creates a provider.
> - This change forwards that existing admission argument through the
controller.
> - Profile drift can then stop before model spending.
## Linked Issues or Issue Description
Refs #17943 and #17929.
**What happened?**
The worker in run 40618022700 passed all four behavior checks and
retained three final request receipts. Its grader then rejected the old
Claude SDK pin. The controlled wrapper could not pass the existing
`--expected-acpx-profile` argument to the built CLI.
**What did you expect to happen?**
The built CLI must reject a stale expected profile before provider
creation.
**Steps to reproduce**
Run a controlled ACPX cell with a definition that pins a different SDK
from the built Runner. The old wrapper accepts only request and output
arguments. It cannot enforce the profile admission supplied by updated
definitions.
**Paperclip version**
The observed run used
|
||
|
DottaandPaperclip
|
6987c2f20b |
test(upgrades): qualify controller replacement with immutable artifacts and phase evidence (#17455)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/105/changes/paperclip/105 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip needs evidence that an admitted execution survives a main-server replacement. > - Runner mechanics and server adoption belong to #17097 and #17439. > - A healthy HTTP endpoint does not prove recovery of the original execution. > - Qualification must identify the serving artifact, natural ownership, authenticated attachment and resumed processing separately. > - It must also retain failed runs and measure normal-operation costs. > - This PR extends the existing qualification machinery and acceptance record. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operators can assess upgrade continuity from phase evidence tied to exact artifacts and profiles. **Current behavior** A same-artifact restart or synthetic executor pass cannot establish different-artifact upgrade and rollback support. Idle SQL gates also do not measure active-run capacity. **Proposed behavior** Use immutable A/B artifacts and strict phase receipts through the existing supervisor. Preserve failed evidence and add measured before/after workload costs. Builds on merged #17097 and #17439. The base is now `master`. #17456 follows for review convenience. This PR lands qualification machinery and the evidence record. The remaining hosted gates stay open for deployment qualification; merging this machinery does not claim those experiments passed. ## What Changed - Extend the main-server supervisor and installed-artifact verification for actual serving revisions, forward migrations and compatible rollback. - Distinguish HTTP readiness, natural ownership, authenticated attachment, resumed processing and backlog settlement. - Retain the original reconnect grace and execution identity checks. Never substitute a new task or edited lease for continuity. - Reuse real-harness and Daytona qualification paths, with exact profile and artifact provenance. - Bind current Copilot attestation inputs in the Daytona image fingerprint. - Keep one acceptance record, including both failed file-only stress phases and the final passing bounded-pool comparison. ## Verification After retargeting to master, all 158 focused qualification cases across 13 files pass, with E2E typecheck. The first 12-file batch has four report-lock test failures and 133 passes. Master #17811 had committed the exact already-approved output lock, while that fixture assumed HEAD must be the uncorrected input. The fixture now checks both pinned forms, executes the committed-lock path, and reconstructs the pinned input to retain all rejection tests. The production helper, policy hashes, patch and dependency resolution are unchanged. The corrected batch explicitly includes both process-preservation suites; the prior 146-case receipt has its own source and selection. Current candidate `8f1d6ec02f` integrates master `09ad9adce2`, including the merged server slice. Its diff contains only qualification code and documentation. The [qualification record](https://github.com/paperclipai/paperclip/blob/8f1d6ec02f5bf6c90e7b27da8156953cabd9a180/doc/plans/2026-10-22-hot-upgrade-qualification.md) is the acceptance ledger and retains source-attributed failures and hosted limits. On the eighth integration, all 146 cases across 12 focused qualification files pass, including report-lock preparation, workflow security and workflow targeting. E2E typecheck passes. The earlier 118-case result remains attributed to its prior source. The first sandbox invocation has 117 passes and one failure because process inspection was denied during inert fixture cleanup. The permitted repeat passes all 118 and a targeted process check finds no remaining fixture daemon. The earlier 96-case receipt remains source-attributed in the ledger. The latest two review regressions first fail: an intermediate replacement can cite another runner/session despite a valid final step, and malformed receipt JSON aborts evidence packaging. Each replacement now binds run/company/issue/runner/session, and corrupt receipts retain explicit invalid metadata plus sanitized diagnostics while other logs and receipts still package. These checks prove qualification machinery with injected supervisor operations, configuration subprocesses and real filesystem packaging. They do not qualify a hosted harness. Three request-reader cases, 14 artifact/configuration/cache/history cases, and three lost-partial-receipt cases fail before their respective corrections. The final collector preserves sanitized startup, readiness and recovery phase receipts through packaging and temporary-root deletion. Earlier embedded-Postgres initialization, report-catalog drift, CI scheduling/cleanup failures and fixture collisions remain recorded with their corrections or limits. The previous Greptile service error is retained as historical review evidence, not a current review pass. The record includes prior source `9bb9d3417e`'s actual Linux contention/fairness successes and the ordinary-warm regression proof from the server slice. The historical 300-session component campaign passes four after phases and retains earlier burst failures. Those component costs do not establish hosted capacity. The hosted API-only pair has zero agents/viewers, fails both absolute latency gates and the candidate memory gate, and does not complete the active-agent/viewer matrix. The strict ten-agent same-artifact restart remains attributed to its original artifact. It does not qualify this updated stack. Actual A→B/B→A, the normal drain-free Cloud rollout, and other hosted harnesses remain open. No new provider turn, deployment or hosted qualification was run during merge preparation. Prior source `57d3767eac` jobs 122590643944, 122590643983 and 122590644161 ended after runner shutdown/cancellation without final Vitest results. The shutdown cause is unproven. No retry was issued before this source was superseded by the required server recovery fix. These attempts remain failures/incomplete evidence. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and normal merge eligibility remain required. ## Risks Different-artifact A→B/B→A, the normal drain-free Cloud rollout path, other hosted harnesses and the complete concurrency/viewer matrix remain open. Frozen branch builds are development-build evidence. This PR owns qualification and evidence; production defects must be fixed in their earlier owning slice. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
09ad9adce2 |
feat(server): adopt running remote agents through durable controller checkpoints (#17439)
## Thinking Path > - Paperclip must keep admitted remote work alive while its main server is replaced. > - Runner owns recovery mechanics and exposes opaque checkpoints through #17097. > - The server owns company scope, encryption, Stop, tool authority and execution ownership. > - A successor must prove current ownership before it adopts the original execution. > - Checkpoint bursts must also leave capacity for normal requests and lease renewal. > - This PR persists and adopts that authority through the existing runtime boundaries. ## Linked Issues or Issue Description **What existing behavior does this improve?** Remote agents retain the original execution during a bounded main-server outage. **Current behavior** A server restart can lose the controller's local state and connection. An unsafe replacement could repeat a turn or an uncertain mutation. Checkpoint writes in the request pool can also delay ordinary requests during an aligned burst. **Proposed behavior** Restore encrypted control records from Postgres and adopt the original runner under a fenced lease. Recover known tool receipts through current tool authority. Give control checkpoints a shared pool capped at two connections per database owner. Depends on #17097. Successors are #17455 and #17456. Qualification and presentation remain separate review slices. ## What Changed - Add company/run-bound encrypted checkpoint records and migration 0394. Keep the SQL idempotent after renumbering against current master. - Recover startup and expired ownership through native runtime and heartbeat. Preserve the sandbox lease, admitted runner artifact, harness configuration, native session and active turn. - Keep Stop, finalization, preparation ownership and stale-writer fencing transactional. Preserve graceful detach and natural crash takeover. - Recover committed tool receipts through the existing authority. Reauthorize reads and hold uncertain mutations. - Bound startup retention with the existing coordinator and preserve zero idle scans. - Share at most two checkpoint connections per database client. Keep application and liveness capacity separate. Close the checkpoint pool during shutdown; late writes cannot reopen it. - Count SQL across both application and checkpoint clients. Prove request/liveness progress, command recovery and failure after pool shutdown with real Postgres. ## Verification Project creation recovery now uses the existing project activity receipt owner. It retains authenticated input normalization, fingerprint checks and current run authority after the receipt lock. An absent receipt returns no result before mutation. The new lost-HTTP-response and no-effect regression uses real API routes and Postgres. The seven transport cases pass locally. This Mac skips all 20 database cases. On current source `5de803c2cd`, Linux [job 122593415890](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415890) executes all 20 project-tool cases successfully (9.480 seconds), including the committed-project lost-response regression. Its shard passes 1,627 cases with three unrelated skips. Server typecheck passes. The transport regression initially has two failures; a subsequent incorrect expectation about schema defaults was corrected before the final pass. Ordinary execution adds no checkpoint writes or recurring work. Current candidate `5de803c2cd` includes the project-creation receipt recovery fix and master `2cd747d1b6`. Migration 0394 retains the prior idempotent checkpoint DDL. Every master SQL file and journal entry is preserved. Snapshots 0390–0394, migration checks and server typecheck pass. The ordinary warm-turn regression fails on its second turn before `a219ab51ad`. After correction, three completed warm turns select the correct prior checkpoint and preserve the original session. All 288 provider-wiring cases pass. The control plane, external persistence callback, identity rotation and filesystem are real; the backend and database-load seam are scripted. The 13 warm admission/configuration variants also pass after final master integration. This does not qualify an underlying harness. At current source `5de803c2cd`, Linux [job 122593415907](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415907) passes all 940 native-session-executor cases (61.431 seconds), including the warm external-checkpoint behavior; the shard passes 2,358 cases with one skip. Current [Runner job 122593415475](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415475) passes the native recovery lane: 115 passes and five container-only skips, including the encrypted restore guard, fresh preparation (63.189 seconds), blocked claim lock (1.226 seconds) and recovery beyond 101 blocked owners (3.897 seconds). The first lane passes 2,215 cases with one skip. The SQL cost test reports zero idle and streaming SQL for both output volumes; this is component evidence, not a production capacity claim. At prior candidate `1ff7bb44f4`, Linux job 122549307215 passes all 938 native-executor cases, including the three-turn external-checkpoint regression. Job 122549306469 passes the native recovery lane with 115 passes and five container-only skips, including the encrypted-restore session guard, fresh preparation (63.664 seconds), claim-lock contention (3.961 seconds), and 101-owner recovery (3.824 seconds). This Mac skips the real-Postgres suite. The earlier full local executor attempt has 921 passes and 16 failures during embedded-Postgres initialization before assertions. The original fixture collisions and their corrections remain recorded. At earlier candidate `8fbe62b41d`, server shard job 122535673841 has 3,876 passes, one failure and one skip in an unchanged expiry-worker retry/drain timing test. The same server/database tree on qualification source `b64cac6951` passes all 20 expiry cases and all 3,877 shard cases. The single diagnostic retry, job 122541891915, passes all 20 expiry cases and all 3,877 shard cases (one skip). No assertion was weakened. The original failure remains recorded and its exact timing race is not proven. All Actions checks and legacy status passed on `8fbe62b41d`, with completed Greptile 5/5 and no open feedback. The next candidate `1ff7bb44f4` also passed all Actions checks and Greptile 5/5. Its Paperclip Review remained queued before the new master conflict required integration. The new candidate needs fresh exact-head CI and reviews; old checks do not qualify it. Migration generation is checkpoint-only, all master SQL and journal entries remain unchanged, and migration checks, server typecheck and all 13 warm-owner variants pass after integration. The first migration check was blocked by TSX sandbox IPC; its permitted repeat passes. The first warm-test command used an incorrect path and ran no tests; the corrected command supplies the 13-case receipt. All 16 database pool-lifetime/options/teardown cases pass after granting loopback permission; the first batch had two EPERM failures and 14 passes. Earlier semantic-tool/HTTP recovery, adoption, encryption-batch, zero-streaming-write and idle-SQL receipts remain source-attributed in the [qualification record](https://github.com/paperclipai/paperclip/blob/2ff66f57da0da35bac1c19a2366ea4b84d87a986/doc/plans/2026-10-22-hot-upgrade-qualification.md). That record retains earlier assertion, setup and cleanup failures and their corrections. No failed gate is waived. The historical 300-session component campaign passes four after phases: steady read p99 is 107–108 ms at 500 reads/s and 100 writes/s; aligned-burst read p99 is 103–105 ms after the bounded pool, versus 584–689 ms before. This comparison isolates the persistence corrections. It does not measure the final stack's hosted active-agent capacity. The hosted zero-agent API pair leaves SQL and CPU essentially unchanged, but both arms fail absolute latency gates and candidate peak cgroup memory fails its gate. The active-agent/viewer matrix, different-artifact upgrade and rollback remain open. Run `PAPERCLIP_TEST_DATABASE_URL=<owned-disposable-postgres> pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-runner-restart-recovery.integration.test.ts`. Previous head `f2db8a6568` had one failed workspace setup gate: [job 122555672441](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122555672441) passes all 4,169 UI tests and 285 CLI cases (one skip), but the company import/export suite reaches its 30-second health deadline during doctor startup. Its output does not reach the server-start step. The CLI, server, database, shared, adapter and plugin trees are identical on qualification source `a866619d7b`, whose [job 122555886663](https://github.com/paperclipai/paperclip/actions/runs/40609057937/job/122555886663) passes that fixture and all 286 CLI cases. The one diagnostic retry, [job 122561057225](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122561057225), passes the formerly failing fixture and all 286 CLI cases. All Actions checks and legacy status on `f2db8a6568` pass. Its exact-head Greptile completed at 5/5 with no open feedback. The current integrated head requires fresh checks and a completed Paperclip Review before merge. The original failure remains recorded; its exact timing cause is unproven. A repeated failure requires investigation and repair rather than another retry. All current-head CI, legacy status, exact-head completed Greptile 5/5, resolved feedback and ordinary GitHub merge eligibility remain required before merge. The latest integration preserves master’s published Copilot v70 profiles and every published migration. Checkpoint migration `0394_conscious_whizzer.sql` was generated with `pnpm db:generate` and retains the prior idempotent DDL. The Runner vendor rebuild and replay parity checks pass. Server typecheck and all 13 warm-owner variants pass after that rebuild. The imported report-lock patch is byte-identical to master; its intentional patch-context whitespace is retained, and the feature delta passes whitespace checks. No hosted experiment was run. ## Risks The pool adds up to two active connections per instance, shared across agents. Checkpoint delivery can wait behind that bound; it still requires its commit. Steady component-test Postgres CPU rises by about 1.7 percentage points of a two-CPU allocation. These corrections do not remove the stack's control and renewal writes. The migration and key must survive replacement. Fresh-database tests and local load do not establish different-artifact hosted upgrade or rollback continuity. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
94a780286f |
fix(server): schedule tool health from committed changes and deadlines (#17554)
## Thinking Path > - Paperclip manages AI-agent work and its connected tools. > - Idle hosted instances need their database and network activity to stop. > - The maintenance tick queries tool connections even when no health check is due. > - Existing connection rows already contain eligibility and the last health observation. > - This pull request reconstructs health deadlines and refreshes them after committed changes. > - The shared coordinator preserves retries, drain, and shutdown without another private timer. ## Linked Issues or Issue Description **What happened?** The aggregate maintenance tick scans connection health every 30 seconds. Empty instances and connections with fresh health observations still cause SQL activity. **Expected behavior** No eligible connection means no recurring health SQL. Eligible connections retain the existing 15-minute proactive check. Configuration changes and health observations update the schedule after commit. **Steps to reproduce** Start a server with heartbeat scheduling enabled and no tool connections. Let startup finish and observe two maintenance ticks. The old code repeats the health inventory query. The new startup regression verifies that those ticks do not invoke the health inventory. **Paperclip version or commit** This branch includes merged parent #17518 and master `96727b3cfefb61f85ffe1fbc29a9b6dd6837efe8`. The 47 changed paths belong to the single tool-connection health duty. No database file differs from master. Related: #14638 covers runtime-auth exclusions. #15448, #15325, and #11910 concern MCP failure classification. This PR preserves those provider policies and does not replace their work. ## What Changed - Keep the GitHub error-projection unit fixture at the health-write boundary and verify the real transaction path in the PostgreSQL health-worker suite. - Add a health schedule owned by tool access. Read lean eligibility fields and saved observation timestamps. - Register health work with the existing coordinator. Refresh only dirty connection/application/company scopes using bounded inventory pages, check due connections in fair company batches, continue concrete due backlog after one second, and retain failures with backoff. - Notify before supported eligibility and observation writes in their owning transactions. Cover API/gallery setup, OAuth, catalog/gateway observations, email, voice, access removal, deletion, and settings. - Commit connection creation with grant, binding, and runtime-slot SQL before its wake. Keep provider calls outside the new transactions. - Keep provider-to-transport mapping in the chat catalog. SDK-only chat changes do not wake health work. - Replace aggregate health scans, retain the public manual sweep, and document the producer audit and remaining sleep limits. - Keep metadata-only example and Smoke Lab installs scoped. Lock and re-read application type before publishing a real eligibility change. - Keep failed inventory reads on their own retry deadline while healthy due batches continue. - Apply fixture migrations before using external disposable databases in all six producer suites. ## Verification Published head: `8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62`. Fresh CI and reviews are required for this exact head. - **623 distinct targeted cases pass.** The final four complete health, fair-batching, GitHub health recovery, and startup suites pass 80 cases after the retry fix. Five complete producer suites pass 543 cases: tool access 469, access 18, email 45, GitHub events four, and Smoke Lab seven. The earlier 26 health cases are part of this total, not additional cases. - The six producer suites first passed against separate, initially empty PostgreSQL 17 databases at `acbf262843`. They apply their own migrations. The subsequent repair changes only the health worker and its regression; its complete final consumer suites pass on that integrated parent. Server typecheck and build pass with the final repair. - **Installer amplification:** five real installs with 205 unrelated connections reproduced 1,035 example rows and 1,045 Smoke Lab rows. Repeated metadata installs now load only changed fixture connections. The same cases restore an excluded application through each installer and require all 205 unrelated IDs to re-enter the schedule. - **Concurrent eligibility:** a real PostgreSQL lock-wait case requires the installer to notify after a concurrent type change. A mutation that retains the lock but uses the old lookup fails the settled-hint assertion; original source was restored byte for byte. - **Retry isolation:** the failing baseline read one unavailable company three times while another company's 60 due checks drained. Final tests require one initial read, at least 30 seconds between failed reads, prompt healthy progress, eventual recovery, and SQL quiet after settlement. - Real-service producer coverage includes AgentMail activation/pause, GitHub installation changes, and member access removal. Lifecycle coverage includes outer commit/rollback, startup reconstruction, drain/resume, failure, and shutdown. - Failure history and exact evidence scope: [health acceptance](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/idle-tool-connection-health.md#acceptance-follow-up-after-the-plugin-parent-merged-october-27), [producer audit](https://github.com/paperclipai/paperclip/blob/8c1d293d1ae2b7db753ab20f5a2a2edec1fb0a62/doc/plans/2026-10-24-tool-connection-health-sleep-audit.md). Prior remote failures and earlier 706-case evidence remain historical. Startup tests emitted listener-count warnings; all assertions passed. No full local repository-suite, deployed sleep, fleet savings, or live provider qualification claim is made. ## Risks - Supported live writers must use the transaction signals. Independent SQL writers and other replicas need their own wake integration. - Configured eligible connections still have a real proactive health obligation. This does not prove Cloud alarm coverage or host/database suspension. - Existing manual probes and catalog operations can overlap. This migration does not introduce a credential-version CAS or change provider failure policy. - Voice remains eligible under the previous health policy. Its generic probe can still report an unsupported transport; changing that behavior is separate work. - No database migration, new table, workflow, or lockfile change. This PR includes its merged parent and current inspected master; later stack PRs and all current-head acceptance gates remain separate. ## Model Used OpenAI Codex, GPT-6 family, with repository tools, code execution, and reasoning. The exact serving version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
github-actions[bot]andlockfile-bot
|
5cfa2bf26d |
chore(lockfile): refresh pnpm-lock.yaml (#17811)
Refresh the CI-owned pnpm lockfile from current workspace manifests and patches. Frozen install and 64 focused Product Journey tests passed locally. Full hosted CI passed on the exact generated head. Earlier browser failures remain documented in the PR. Co-Authored-By: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
96727b3cfe |
fix(server): schedule plugin jobs from saved deadlines (#17518)
## Thinking Path > - Paperclip manages agents and plugin work. > - Plugins can declare cron jobs with saved deadlines. > - The existing scheduler queries for due jobs every 30 seconds, even when none exist. > - Empty queries keep an otherwise idle database active. > - This PR uses the shared scheduler and committed job changes to discover work. > - Existing job and run rows preserve deadlines and execution history. ## Linked Issues or Issue Description **What happened?** The plugin scheduler on the base of this stack runs empty periodic SQL. Its registration path also advances overdue timestamps. An empty replacement manifest can leave removed schedules active. **Expected behavior** No recurring SQL when no eligible jobs exist. Configured jobs wait for their saved deadline. Changes reactivate scheduling after commit. Work remains protected through drain and shutdown. **Steps to reproduce** Start the plugin scheduler with no jobs. After startup, observe queries during a three-minute idle window. The new PostgreSQL regression failed on the baseline and passes with this change. **Paperclip version or commit** Stacked on #17510 at `4d15c08428`. Deployment: source checkout, local PostgreSQL, controlled plugin RPC. Related: #7905 covers empty-manifest sync; this migration requires that behavior on both activation paths. #11571 proposes company-scoped jobs; this PR preserves the existing instance-scoped job contract. ## What Changed - Install the concurrency test barrier before due job notifications can dispatch. - Make dispatch the sole reservation owner and preserve retained or concurrently committed results when unregistering a plugin. - Replace the private interval with the existing shared deadline worker. - Signal job definition and run changes inside their owning transactions. No new tables. - Wake from plugin registration and worker status events. Retain explicit scheduler wakes received during drain. - Preserve overdue work and initialize only missing or changed schedules. - Recheck claims under a job-row lock. Keep plugin calls outside transactions and refill capacity on completion. - Commit completion and schedule advancement together. Retry failed persistence without repeating the plugin side effect. - Track admitted calls through shutdown and synchronize empty manifests. ## Verification This PR now targets master after #17510 merged as `4e27a82a4265c01f3ab4f05aaecc010a0026de75`. The normal landed-parent merge preserves the entire verified `df78ca3b5e` tree byte for byte; only final publication docs change afterwards. The PR still differs in its original seventeen paths and has no database schema or migration change. Primary test revision: `36be0bd73363680d7666d94ff5f1dd28b27f4235`, on parent `ba80060407`. Final integration of parent `51668b8240` preserves all plugin/runtime/shutdown sources from that proof and passes all 63 cases in complete plugin, approval-seeding and scoped-deadline suites at `56ce7c1d2a`, plus server types/build. Only six parent seeding/doc files changed; this is not a rerun of all eleven suites. Only the original seventeen plugin-scheduling paths differ from that parent; all other tracked files, including migration history, match it. - **247 cases pass in eleven complete suites:** plugin scheduler, shutdown, warm standby, plugin environment-driver readiness, delivery coordinator, both retention workers, approval seeding, decision expiry, scoped deadlines and startup. Server typecheck and production build pass. - The new real PostgreSQL lock test failed before the repair: shutdown remained pending while the completion update waited for a held run row. After repair, shutdown reports the statement timeout while the lock remains held; retry after release commits the saved result with exactly one plugin invocation. All 34 scheduler cases pass. - Existing regressions preserve real committed schedule changes, saved deadlines, manual/scheduled reservation, retained success and failure results during unregister, completion/unregister lock order, independent progress, rollback/outer commit, quiet configured states, drain/resume and shutdown cleanup. - The parent's complete seeding/producer run passed 522 cases at its stated revision. Earlier duplicate reservations, overwritten retained results, hourly-clock fixture failures and combined-stack CI/setup failures remain recorded in `doc/idle-plugin-jobs.md`, its dated audit and the stack ledger; they are not erased or recounted as this candidate's tests. - The completion timeout is transaction-local and defaults to thirty seconds per PostgreSQL statement. It does not bound every shutdown/network operation or make plugin side effects safe to replay. No full local repository-suite or deployed host-sleep claim is made. Commands: server Vitest for the eleven suites above; server typecheck/build. Fresh exact-head CI, Greptile and Ponytail reviews remain required before merge. ## Risks - This uses process-local notifications. Independent SQL writers and other replicas need their own wake integration. - A run left unresolved after process loss is not automatically replayed: the external effect may have occurred. Existing lifecycle/operator reconciliation must resolve it. Live-process settlement failures retry without replay. - Job definitions remain instance-scoped. This PR does not implement company-scoped job dispatch. - Cloud's plugin sleep blockers remain. Internal deadlines do not add durable external wake coverage or prove deployed sleep. - The shared deadline helper now retains direct lifecycle wakes through drain; existing coordinator and decision deadline consumers are tested. ## Model Used OpenAI Codex, GPT-6-based agent, with repository tools, code execution, and GitHub access. The exact serving model version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.3 |
||
|
DottaandPaperclip
|
39e811c9c5 |
fix(runner): prepare diagnostic proxy and reduce bundled skill queries (#17941)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native turns reconcile company skills and select verified remote runtime packs. > - Catalog imports repeat folder transactions for skills in the same bundled category. > - Diagnostic requests can also reuse an older image proxy that lacks the phase collector. > - This pull request reuses categories within one import and selects the current proxy for fresh diagnostics. > - Each import has its own category map, and recovery keeps its admitted runtime identity. > - The result is less preparation work and a runtime that can provide the requested measurements. ## Linked Issues or Issue Description Refs #17918. **What happened?** A hosted run retained `PAPERCLIP_OPENCODE_PERFORMANCE_DIAGNOSTICS=1` but emitted no provider phase batch. Its saved pack authority selected an older image proxy. The deployed server proxy contained the collector. The image proxy did not. Bundled inventory reconciliation also repeats folder transactions for skills in the same category. The paired local benchmark exposes that preparation cost. **Expected behavior** A fresh diagnostic request must use a verified proxy that supports the current collector. Bundled skill reconciliation should resolve each category once per import. **Steps to reproduce** 1. Use a protected image pack with the same OpenCode interface revision and an older proxy hash. 2. Enable `PAPERCLIP_OPENCODE_PERFORMANCE_DIAGNOSTICS=1` for a fresh remote run. 3. Observe that the current backend accepts the older proxy and cannot emit the requested phase batch. ## What Changed - Reuse each bundled category during one import. Every skill still runs its existing validation, access lookup, value comparison, and write path. - Compare the installed proxy hash with the current build when a fresh run requests diagnostics. - Keep compatible-pack reuse for ordinary runs and exact admitted identity for recovery. - Cover selection through the real backend, including recovery after diagnostics are enabled. - Document runtime preparation costs and admission limits for diagnostic requests. The skill importer owns category reuse. The provider verifier owns the artifact comparison. The native backend passes the resolved diagnostic requirement at the existing admission seam. The changes stay in those existing owners and share one latency investigation and rollout. ## Verification - Before the fix, the new verifier and backend cases failed because both accepted a stale proxy. - After the fix, 63 focused cases passed in 8.10 seconds. They cover normal reuse, current diagnostic proxy selection, stale proxy rejection, selected ACPX profiles, and admitted recovery identity. - 110 skill and folder service cases passed against isolated real PostgreSQL databases. Only cluster bootstrap used the existing local PostgreSQL service. Queries, migrations, authorization, file handling, and transactions ran on their real paths. - A paired benchmark used the same immutable skill snapshot, authenticated actor, 170-skill inventory, and 3.5 ms delay in each network direction. It checked every delivered file and executable bit. Warm median fell from 3.319 s to 2.482 s (25.2%). SQL calls fell by 81 in each matched sample, from 241–243 to 160–162. The bundled phase median fell from 2.596 s to 1.721 s. The final measurement uses commit `37eac8d8de`; only the skill importer differs from baseline `7951455073`. These local results are not hosted acceptance. - `pnpm -r --workspace-concurrency=1 typecheck` passed with the pinned pnpm 9.15.4 and a 6 GiB Node heap. Earlier attempts exposed a subprocess toolchain mismatch, a local IPC restriction, and a fixture type error. Each was corrected before the passing run. - Module boundary and whitespace checks passed. - `pnpm build` passed for all workspaces. - `pnpm test:run` did not complete. Embedded PostgreSQL initialization failed after repeated startup attempts. The owned test subtree was stopped after preserving the failure log. This is not a full local test pass. On final commit `37eac8d8de`, all 53 successful check contexts passed; two optional Storybook checks were skipped. Greptile is 5/5 with no unresolved findings. The draft awaits merge approval. - The catalog change has a measured local gain. The proxy change fixes missing phase capture. Real hosted response latency remains above the target and needs a new matched measurement after delivery. ## Risks The category map belongs to one import and one company. The importer still resolves current skill data on every invocation. Fresh diagnostic runs can prepare a current verified pack, which can add startup time. Admission can fail when matching runtime assets are unavailable. Ordinary runs keep the existing fast compatibility path. Recovery keeps the already admitted identity, even if diagnostics are enabled later. The operator endpoint gate, collector limits, and closed span fields are unchanged. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, shell execution, browser tools, and Honeycomb queries. The session does not expose a verifiable backend deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2cd747d1b6 |
perf(server): batch connection install capability checks (#17945)
## Thinking Path > - Paperclip manages AI agents and their access to external connections. > - The connection grants screen describes which agents can receive an installation. > - Each candidate currently repeats authorization for every retained installation. > - A company installation makes this work grow with the square of the roster size. > - This pull request checks retained targets once inside the existing service owner. > - The benefit is fewer authorization calls with the same scoped capability results. ## Linked Issues or Issue Description **What happened?** The grants GET repeats connection preparation and retained-target checks for each candidate agent. A local 100-agent company-install fixture made 50,300 SQL calls and 10,000 target lookups in one capability-description operation. Open PR searches found no matching capability fan-out fix. **Expected behavior** The description must preserve retained-installation obligations and each candidate's permission. It should evaluate retained targets once per description. Writes must still authorize fresh. **Steps to reproduce** Run the committed benchmark with `--baseline-ref=f6bc653f7f745adaafa26393852fb2f5a99256a4`. It creates disposable synthetic PostgreSQL fixtures and loads the actual pre-fix service from Git. It compares every positional capability, reason, and operation. **Paperclip version or commit** Baseline `f6bc653f7f745adaafa26393852fb2f5a99256a4`. **Deployment mode** Built from source. Local real PostgreSQL proof. No hosted rollout. ## What Changed - Add `describeAgentInstallCapabilities` to connection management. Ordinary authorization and the batch share preparation, target checks, and fallback completion. - Use the batch only for direct saved non-AI connections and board actors. Complex resources, delegated actors, and unusual saved target IDs keep the ordinary evaluator. Fallback consumes the first preparation once. - Replace only the grants route's editable-agent loop. Keep configure/use checks, identity filtering, audience disclosure, and write authority on their existing paths. - Add scoped-grant, retained-target, error-order, freshness, fallback, full HTTP response, and real mutation regression coverage. Add a reproducible benchmark and evidence notes. ## Verification Real PostgreSQL 18.6 Linux Docker, Node 26.4.0. Two warmups and five measured samples per arm, in alternating order. These are service-component measurements with the same SQL-count instrumentation in both arms. They do not measure the whole hosted endpoint. | Fixture | Old median | Batch median | SQL calls old → batch | Target lookups old → batch | | --- | ---: | ---: | ---: | ---: | | 100 agents, company install | 24.753 s | 246.39 ms | 50,300 → 503 | 10,000 → 100 | | 100 agents, partial install | 6.339 s | 236.37 ms | 13,075 → 502 | 2,575 → 100 | | 1 agent, company install | 5.13 ms | 4.82 ms | 8 → 8 | 1 → 1 | All eight fixture shapes and every measured iteration had complete positional equality. One-agent timing ranges overlap. Full ranges, database settings, and limits are in `doc/plans/2026-10-27-connection-grants-batch-verification.md`. - **Passed:** `pnpm -r typecheck` and `pnpm build`, using pinned pnpm 9.15.4. - **Passed:** 116 focused tests: all 83 connection-management integration cases use real PostgreSQL and HTTP; 33 batch regressions execute real policy over synthetic query transport. The integration cases keep existing database creation, migrations, clients, services, and assembled HTTP server, with disposable Docker PostgreSQL. - **Passed:** complete grants GET parity across six human-role fixtures; real install PUT succeeds before revocation, then fresh GET and actual PUT deny after revocation without changing saved installs. - **Passed:** a real PostgreSQL red/green regression for empty saved target selectors, including candidate-specific decisions and later reached infrastructure faults. - **Passed:** independent old/new evaluator comparisons, including recursive and delegated fallback, run-context reuse, continuation, failure order, and principal/company isolation. - **Local full-suite limit:** `pnpm test:run` was attempted and stopped after native PostgreSQL fixtures could not initialize. An isolated failed native case reproduced that startup error. The host cannot allocate PostgreSQL SysV semaphores, including outside the sandbox. The focused suites use an existing constructor seam to start disposable Docker PostgreSQL. This is not a complete local full-suite pass. - **Passed:** all 16 native-runtime cases that failed to start PostgreSQL on the host, rerun with the disposable Docker provider. The full local workflow remains incomplete. - `git diff --check` passes. Greptile is 5/5 with no actionable findings on final head `41dcf3ae029b6c60999ca6208136399b0af663d0`. All final-head CI gates passed, including the full PR workflow. The two Storybook checks are expected skips because this change has no UI changes. Normal reproduction: ```sh node --import ./server/node_modules/tsx/dist/esm/index.mjs scripts/benchmark-connection-install-capabilities.ts --baseline-ref=f6bc653f7f745adaafa26393852fb2f5a99256a4 cd server pnpm exec vitest run src/__tests__/connection-install-capabilities.test.ts src/__tests__/connection-management.integration.test.ts ``` ## Risks Retained facts are observed once during one descriptive call. A mid-call revoke or deletion can leave a stale displayed choice. The next description and every mutation or human continuation read fresh authority. The batch returns no reusable authorization context. The bound applies to target evaluations in the eligible shape. Scoped grant payloads and hierarchy reads can still grow inside each policy decision. Complex resources and agent callers keep existing costs. Hosted latency improvement remains unverified. There is no migration, provider protocol change, new worker, persistent cache, or UI contract change. ## Model Used OpenAI Codex with code editing, execution, and test tools. Design, test, and independent review agents used exact model IDs `gpt-6.1-sol` and `gpt-6-astra` with xhigh reasoning. The host did not expose the primary session's exact model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
38d08847b8 |
fix(runner): keep OpenRouter upstream keys in the parent proxy (#17949)
## Thinking Path > - Paperclip manages agents and their work. > - Paperclip Runner starts OpenCode with the selected provider and model. > - The native OpenRouter route passed the reusable upstream key into the child environment. > - A persisted-state scan found that key in an OpenCode database WAL. The exact write operation is not known. > - This pull request reuses the existing parent-held provider proxy for that route. > - The child receives a temporary loopback credential instead of the upstream key. ## Linked Issues or Issue Description **What happened?** The native OpenRouter path passed `OPENROUTER_API_KEY` into the OpenCode child. A real Product E2E persisted-state scan detected the key in the child database WAL. The scan does not establish which operation wrote it. **Expected behavior** The runner keeps the reusable upstream key in its own process. OpenCode can call only the selected model through the session proxy. **Steps to reproduce** 1. Start the OpenCode driver with an explicit or ambient synthetic OpenRouter key. 2. Inspect the fake child configuration, environment, and persisted provider values. 3. Send a request through the real loopback proxy and verify the upstream authorization and exact model. **Paperclip version or commit** The regression is present on merged master `813e808af462b6a6f9db38b6791e3bfe8465b51e`. This PR is rebased onto merged master `d40f9ac377`. **Deployment mode** Built from source. The finding came from an AWS Product E2E job. The regression tests use local synthetic servers and a fake child, with no provider calls. Related issue search found #16643 about general inherited environment scope. This change addresses the narrower native OpenRouter path. No duplicate custody PR was found. This PR must remain unmerged until separately approved; it does not change the eval baseline or qualify a new live result. ## What Changed - Reuse the existing parent-held proxy for native `openrouter/` models. - Remove the upstream key from the child environment and retain it in diagnostic redaction. - Keep the selected provider and exact model. Reject alternate-routing fields before the upstream request. Use a session-lived loopback token. - Preserve the effective explicit or ambient proxy bypass list when adding loopback entries. - Add explicit and ambient key regressions, upstream routing checks, and child persistence checks. - Document the boundary and its limits. ## Verification - Final rebased head: full OpenCode driver suite, 64 synthetic tests passed, and runner source TypeScript check passed. Original author run had 61 passing tests before the new master tests. - The two new explicit and ambient regressions fail on the old implementation. - Independent source review passed. The two new synthetic regressions also passed independently. A separately filtered legacy pair hit fixture selection/chmod ordering and is not claimed as a passing run. - Tests use the real driver and proxy with a fake OpenCode child and synthetic upstream transport. They check key exclusion, exact URL/model/authentication, wrong-model and alternate-routing rejection, proxy bypass preservation, redirect rejection, and proxy shutdown. - The two advisory regressions failed before the correction: fallback routing reached upstream, and ambient NO_PROXY was lost. The full 64-test driver suite and source TypeScript check pass after the correction. - No real OpenCode SDK/provider qualification was run. The original WAL write mechanism remains unknown. - Repo-wide typecheck, test:run, and build were not run locally because local native/DB/model execution is excluded. Normal PR CI passed on head `3e1903272ce65b92055ea70f8dc586d9aa5e0205`: typecheck/release registry, general and serialized suites, runner static/Rust/Vitest checks, E2E shards, build, and canary dry run. Greptile is 5/5 on that head. Paperclip Review is neutral with no blocking findings; its two earlier findings are fixed and resolved. ## Risks The native OpenRouter route now uses the existing OpenAI-compatible proxy transport. Synthetic tests do not prove every live SDK response shape. The loopback token remains a credential, is redacted, and expires with the session. This change does not claim isolation of every child secret or persisted value. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) assisted source investigation, implementation, synthetic tests, and review with tool use. No provider model call was used for verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e4b30163bb |
fix(evals): preserve the complete trusted controller artifact inventory (#17943)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/103/changes/paperclip/103 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages real agent work. > - Controlled Runner evaluations record model spend and traces. > - A trusted build creates one portable controller with a complete file inventory. > - GitHub artifact upload omitted one hidden dependency metadata file. > - The next job correctly refused the incomplete inventory before any paid worker started. > - This change preserves all inventoried bytes without relaxing verification. ## Linked Issues or Issue Description Refs #17889, #17929. **What happened?** [Run 40585138437](https://github.com/paperclipai/paperclip/actions/runs/40585138437) failed at copied controller verification. Its artifact contains 294 files; its signed inventory expects 295. The only missing file is `server/node_modules/protobufjs/node_modules/@protobufjs/pool/.npmignore`. No extra files exist. Every paid worker was skipped and no model request ran. **Expected behavior** The trusted controller artifact preserves its full inventory, including hidden package metadata. The strict verifier rejects any actual change or missing file. **Steps to reproduce** Build the portable runtime, upload the directory with the action's default hidden-file exclusion, then verify the downloaded artifact against the original receipt. ## What Changed - Enable hidden files only on the fresh trusted controller code upload. - Download that exact upload and run a real artifact-service round-trip test before any private preparation. The test fails if the upload drops hidden files again; it tests the actual transport instead of matching a YAML flag. - Keep the strict byte inventory and trusted source/receipt checks. - Document why this code artifact differs from private attempt evidence, which still uses the protected archive. ## Verification - Exact retained artifact comparison: one missing `.npmignore`, zero extra files. - `actionlint .github/workflows/runner-protocol-live-evals.yml` passed. - Protocol workflow security tests: 14 passed. - The artifact-service integration test is explicit-only locally. The controlled workflow supplies the real downloaded artifact, trusted source and receipt digest. - No product runtime, provider routing or grading code changed. CI runs broader checks. Fresh controlled workflow qualification follows merge. ## Risks Hidden files are allowed only in the code-only output of the provider-credential-free build. Raw private evidence and credentials do not enter that upload. The artifact verifier still requires every file and SHA-256 from the trusted receipt. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |