mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 04:25:52 +01:00
0e365768f032a472807b5720da8431f92d2f07ce
5861
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
0e365768f0 |
fix(server): preserve sandbox workers when task drain resumes (#18065)
## Thinking Path > - Paperclip coordinates agents and preserves queued work during maintenance. > - Stopping a task drain resumes that queued work. > - Sandbox runs need the configured plugin worker manager to acquire their environment. > - The instance-settings route received that manager but omitted it from its heartbeat service. > - The route could claim queued work and then fail before provider execution. > - This PR forwards the existing manager and tests the actual dispatch boundary. ## Linked Issues or Issue Description Follow-up to #17967, which resumes queued work after maintenance. #18037 adds readiness reporting to the same route but does not repair this dependency. **What happened?** After a task drain stopped, a queued Daytona task failed with `setup_failed`: the provider was installed, but sandbox plugin workers were unavailable in that server process. The application had supplied the manager to `instanceSettingsRoutes`. That route constructed `heartbeatService(db)` without it, then dispatched through that separate service. The environment runtime therefore reached its missing-manager guard before any provider work. **Expected behavior** Drain-release dispatch must use the manager supplied to the route. Queue eligibility, scope and ownership checks must still apply. **Steps to reproduce** 1. Configure a sandbox provider plugin and an agent that uses its environment. 2. Start a task drain and queue work for that agent. 3. Stop the drain through `DELETE /api/instance/task-drain`. 4. Observe dispatch fail before it calls the configured plugin manager. **Paperclip version or commit** Observed at `2e5bbde8cc7eba0eacce00649f17c331203e56d9`. Reproduced against `7f10cc058568e3c1ac133cf24f3b4000cbdfca0e`. **Deployment mode** Hosted Linux with a Daytona sandbox plugin. The local regression uses real PostgreSQL 18 and the actual route, heartbeat, queue, orchestrator and environment runtime. ## What Changed - Pass the supplied plugin worker manager to the route's heartbeat service. - Add a database regression that releases genuinely queued sandbox work through the real HTTP route and checks the scoped acquisition call. ## Verification - Red: the new test executed the route and recorded zero manager calls. The queued run failed with the same missing-manager error as the incident. - Green: the route reached exactly one `environmentAcquireLease` call bound to the expected plugin, company, task, agent, environment and queued run. The fixture then threw a unique sentinel before any external allocation; that sentinel was persisted on the same run. - All 8 database drain-release cases pass. All 101 existing instance-settings and task-drain unit checks pass. `git diff --check` passes. - Local database verification used an owned PostgreSQL 18 container with all 396 migrations. A temporary Vitest configuration replaced only embedded database startup/cleanup. Database queries and dispatch services remained real. The override and credentials are not in this PR. The owned container stopped after the tests. - Preserved setup failures include the initial temporary mock-bootstrap import, an unbuilt SDK entry and a reused synthetic sponsor row. A first fixed-path run also exposed an unnecessary expectation about an optional RPC timeout. That assumption was removed; the run identity, exact call count and scope assertions remain. These failures are not claimed as the behavioral red. - The initial frozen install found existing UI lockfile drift. Setup used the repository's CI resolution fallback. The original lockfile was restored and is unchanged in this PR. - Full `pnpm -r typecheck` and `pnpm build` pass at `be18d8ec1f84af20bad2b6bd97677b1e60c3626f`. Current-head CI passes: 53 successful checks and two intended Storybook skips, none pending or failing. Greptile is 5/5 on that exact head with no open review threads. CI ran the standard embedded PostgreSQL bootstrap and passed all 8 drain-release cases, including the new regression. The full standard local `pnpm test:run` is still in progress; no local full-suite pass is claimed. The earlier targeted database suite used the isolated external fixture. - No post-fix production drain cycle is claimed. The repair is not deployed. An ordinary retry of the observed legacy preparation failure was separately refused by the existing internal retry-safety classifier before admission. This PR does not broaden that classifier or replay the request. The initial CI server-7 shard failed the unchanged decision-expiry restart test: the global coordinator reported a wake at +30 seconds rather than the decision expiry at +120 seconds. The test failed before checking early expiration. A bounded probe of the real work signals, coordinator and decision workers reproduces that earlier wake after a failed transaction is reconciled during startup, without database or provider calls. The captured CI log does not establish that exact trigger. One diagnostic failed-job rerun passed all 1,625 shard tests and the dependent verification gate on unchanged source. The original failure, probe and retry receipts are retained; no scheduler fix or infrastructure cause is claimed. ## Risks The route can now reach the configured sandbox provider when it resumes eligible work. Existing queue claims, company scope, pause and new-drain gates remain in place. The change does not start a second manager, add a wake loop or change the successful drain-release response. Installations without a manager retain the existing missing-manager behavior. No schema, provider binary, credential or retry-authorization contract changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. The session exposes GPT-6 as its model version; a more specific model ID and context-window size are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
21137db805 |
fix: make Check now work for retained monitors on blocked tasks (#18061)
## Thinking Path > - Paperclip lets people manage agents and inspect their work. > - Task monitors schedule checks of work that is still pending. > - A task can retain a monitor after it becomes blocked. > - The UI still offered Check now, but the server rejected the task status. > - The UI also claimed that an automatic check would run on the next tick. > - This change admits an explicit human check through the normal execution gates and shows that automatic checks are paused. > - People can request the check without changing task status, and skipped checks no longer report success. ## Linked Issues or Issue Description **What happened?** A blocked task with a retained monitor showed Check now and an overdue countdown. Clicking the button failed because manual checks required an in-progress or in-review task. The endpoint also returned success when admission skipped a check. **Expected behavior** A human can request one check through normal admission. Automatic checks stay paused while the task is blocked. A check that cannot start reports a conflict. **Steps to reproduce** 1. Schedule a monitor on an agent-owned task. 2. Let the task become blocked while it retains the schedule. 3. Open the task and click Check now. Related: #14909 addresses scheduled-retry promotion. This PR addresses retained task monitors. **Paperclip version or commit** Base:canary/v2026.1028.0-canary.13 |
||
|
|
d30c8f42b8 |
feat(ui): keep writes durable through temporary disconnects (#17960)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/123/changes/paperclip/123 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People save tasks, write documents, make decisions and send board messages. > - A disconnect can hide a response after the server accepts a write. > - Recovery must keep the original account, task and request key. > - The merged connection foundation in #15754 handles reads and shared outage state. > - This change adds bounded write recovery and explicit actions when recovery stops. ## Linked Issues or Issue Description **What happened?** Temporary failures could lose edits, report accepted writes as failures, or require people to copy and send input again. A retry could target a newly selected task or account. Restoring attachments could create a second task. **Expected behavior** Keep the draft, its owner and its target. Retry only operations with a server contract that prevents duplicate writes. Verify uncertain results. Show conflicts and permanent failures, with an explicit resend or cancel action. **Steps to reproduce** 1. Save a task title during an outage, then open another task before recovery. 2. Restore a task creation that needs files, selecting those files in another order. 3. Lose a document or decision response, then interrupt the verification read. 4. Switch accounts while a create, upload or connected-channel publication is pending. 5. Type and erase a new-task draft before the first account lookup completes, then reopen it. This PR carries the write layer from #15769 and its review fixes. It targets master after #15754 merged. Related test cleanup #16952 already covers the unchanged copy-link toast test; this PR does not duplicate that work. ## What Changed - Add bounded autosave and submit jobs for supported writes, with pending, conflict and manual retry states. Retain admitted autosaves after navigation and preserve newer task revisions. - Keep each job with its original account, task and operation. First account resolution adopts admitted startup work; a later account change cancels old work. Remount inline and document editors when their owner or document subject changes. - Preserve task-creation keys and immutable request snapshots through file reattachment. Restore generated document keys before comparing reselected files. Reject changed requests instead of creating a second task. - Check the original account scope before creates and each upload. Save successful upload receipts before stopping on an account change. Identify attachments by content digest and metadata. - Verify uncertain document and decision results. Replay only plain, keyed comments automatically; combined reassignment, reopen and run-setting changes require receipt reconciliation or cancel. Settle each request once when a receipt races its retry. - Scope drafts and connected-channel publication reads to their owner. Keep unowned older records unread and unreplayed, with an explicit confirmed discard action after Activity inspection. Preserve typing through the first session lookup, keep erased text erased, and report browser persistence only when storage succeeds. - Wait for the first account lookup before sending startup status changes or decisions. Retry failed required lookups with a fixed limit and surface exhaustion to the original action. Preserve uncertain comment keys on account cancellation. Check the account generation before a comment retry calls its sender and again immediately before POST after asynchronous preparation. Retire only that request’s optimistic echo on completion. - Return an ordinary comment’s accepted same-company, same-task, same-user receipt before workspace reopening, task status changes or wakeups. Reused keys with changed text still conflict. Complete reference and activity records through the same finalizer for initial sends and accepted retries. It locks the task before its comment, matching task and company deletion, comment removal and tombstoning, rejects removed or tombstoned comments, repairs references and inserts a missing activity record in one transaction, and publishes after commit. - Remove reader-only record and comment-body size caps so accepted browser writes remain readable, including JSON-escaped comment requests in local and session storage. Update four existing Storybook fixtures to use owned draft keys. - Document the write contracts and add regression and real browser recovery coverage. - Add the exact reviewed-save recovery regression for both composer implementations and reloads. Align receipt completion, both comment deletion methods and company deletion with task deletion by locking tasks first. Cover real task and company deletion races. ## Verification - The exact reviewed-unknown-save regression fails in all four cases on original commit `1ed0af4186`. It passes with this PR for both composers, before and after reload. Cancel keeps the text and uploaded receipt. The retained draft sends, and the next message also sends. Run `pnpm exec vitest run --project @paperclipai/ui ui/src/components/composer-recovery.test.tsx`. - The new real PostgreSQL task-deletion race fails with SQLSTATE `40P01` on parent commit `f3253a00c4`. The company-deletion race also fails with `40P01` on `e8cf0e02dd`. All six receipt tests pass after the final lock-order fixes. They cover company deletion, task deletion, comment removal, tombstoning, concurrent cross-task references and interrupted reference repair. Run `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/issue-comment-receipt-postgres.test.ts`. - The full UI run passed 9,656 of 9,659 tests across 816 files. Three cases failed under that run: two large-tree timeouts and one plugin-panel assertion. All 67 tests in those two files passed in an isolated rerun. These failures remain recorded; this does not claim a clean full UI run. - `pnpm -r typecheck`, `pnpm build`, token gates and map checks pass on the final change. The local full repository test command was started, then stopped when the delivery moved to this existing PR. Fresh PR CI must provide the full-suite gate. - A local Storybook walkthrough used the real rich editor. Cancel kept the restored text, Send cleared it after the simulated save, and a new draft enabled Send again. Storybook simulates submission; it is not staging acceptance. - Earlier verification on `f3253a00c4` recorded five browser journeys for title saves, cached-task offline recovery, startup recovery, accepted comment replay and session lookup failure. Those results remain prior-head evidence. No new staging deployment is claimed. - On `e8cf0e02dd`, all CI checks passed after one browser-shard rerun. Its first failure copied page text instead of YAML in an unrelated agent-configuration test. On final commit `35131d4380`, Greptile is 5/5 with no unresolved review threads. The first CI attempt has two failures outside this diff: runner close-attachment suspension and scheduled-retry teardown. The scheduled-retry file passes all 14 tests in isolation; the isolated runner test still fails locally. The single standard CI rerun passed both jobs, and the aggregate verification gate passed. All CI checks are green on `35131d4380`; Apex is 5/5 on that same commit with no unresolved findings. ## Risks - Automatic replay requires server duplicate prevention or result verification. Uncertain combined mutations require receipt observation or cancel and a fresh intentional send. - Old unowned form drafts remain stored but are not assigned to an account. Unowned channel records can be retired after Activity inspection; this never cancels an accepted remote update. - Browser storage and hashing can be unavailable. Storage failure is reported honestly. Without a digest, replay can duplicate an upload rather than silently drop a selected file. - Detached autosaves remain bounded and check account ownership on each attempt. Conflicts or exhaustion retain the draft for explicit recovery. - No schema, credential or agent permission change is included. This PR does not claim staging deployment or original-task acceptance. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) produced the original implementation with tool use. OpenAI Codex, GPT-6 family, assisted with restoration, integration, verification and review fixes. This Codex session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Amy (Opus) <claude@users.noreply.github.com> Co-authored-by: Dotta <bippadotta@protonmail.com> |
||
|
Nicky LeachandPaperclip
|
a2fbce9305 |
Fix plugin session and directory reuse on follow-up turns (#18045)
## Thinking Path > - Paperclip preserves task state when agents receive follow-up work. > - Plugin computers keep their process state outside the controller. > - A local receipt proves that a previous plugin run suspended. > - A cold resume left that receipt in place after the session became live. > - The next warm follow-up rejected the stale receipt as an identity mismatch. > - A separate follow-up path also treated a fresh task directory as a Git worktree. > - This PR consumes the obsolete receipt after admission and preserves the bound directory on follow-up. ## Linked Issues or Issue Description Related: #17820. This change retains the controller restoration added by #18018. **What happened?** A plugin session resumed from a suspended checkpoint and completed a turn. Its next warm follow-up failed with `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate`. The journal had the completed run's identity. The local suspension receipt still had the preceding run's identity. Two separate task histories showed this sequence. After a fresh retry completed in a repository-free plugin directory, an ordinary follow-up could instead fail with `The selected plugin environment does not support Core Git worktree preparation.` The saved task settings contained only `mode: isolated_workspace`, which introduced the default Git strategy on the next admission. **Expected behavior** Keep the same admitted provider session during warm follow-ups. Reuse the saved plugin task directory without Core Git preparation. Retain ownership checks and explicit strategy settings. **Steps to reproduce** 1. Complete a warm plugin task and let its session suspend. 2. Resume the checkpoint and complete another turn. 3. Send a follow-up while that session remains warm. 4. Separately, complete a fresh retry in a repository-free plugin directory and send an ordinary follow-up. **Paperclip version or commit** Both regressions reproduce against base `820c45846` with the added tests. **Deployment mode** Hosted plugin execution. Local reproduction uses the real filesystem and a disposable PostgreSQL 18 database, with synthetic provider execution. ## What Changed - Remove the local plugin suspension receipt after provider recovery and durable run admission publish a live session. Admission still checks the receipt first. Normal suspension writes a new receipt. - Recognize a saved fresh plugin directory through its company, task, workspace, owner, attempt and path. Use the existing directory-selection path. Explicit strategy settings still apply. - Add cold-to-warm, database-backed workspace-selection and ownership regressions. Update execution semantics and session verification recipes. ## Verification - Red: the cold-to-warm regression failed on unchanged executor code with `prior_owner_terminal_state_indeterminate` on the follow-up. - Red: the PostgreSQL regression selected a Git strategy on unchanged environment-selection code. The explicit-strategy denial case already passed. - Green: all 95 tests in `workspace-preparation.test.ts` and `provider-fresh-workspace.test.ts` pass against a disposable local PostgreSQL 18 database, selected with the existing `PAPERCLIP_WORKSPACE_PREPARATION_TEST_DATABASE_URL` test seam. - Green: 92 focused executor cases pass, including plugin checkpoint recovery, warm rotation and denial cases. The full `plugin-runner-task.test.ts` suite also passed (38 tests). - Green: direct server check, `node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit`. - Green: `node scripts/check-maps.mjs`, `git diff --check`, and a local diff scan for secrets, personal data and live identifiers. - Full local native suites: 988 of 1,004 cases passed. Sixteen integration cases could not start their bundled PostgreSQL cluster. The failure persisted outside the sandbox. These cases are not claimed as passing. An earlier sandboxed attempt also denied process and loopback operations; those denials disappeared when run outside the sandbox. - The first broad local commands could not locate Cargo on `PATH`. Rust is installed under `~/.rustup`; its Cargo executable was verified. Per the requested scope, subsequent local verification stays targeted to changed TypeScript paths; no Rust source changed. CI Typecheck + Release Registry and Rust verification passed on the initial commit. The complete repository test suite was not run locally. - Review regression: an explicit agent-level Git strategy reproduced the incorrect directory fallback before correction. After including the raw agent strategy in the existing check, all three targeted cases (implicit, task-level explicit, agent-level explicit) pass. The server TypeScript check passes after this correction. - CI on `a81668c4b`: the first workspace-a 2/2 attempt failed in unchanged UI `AgentActionButtons.test.tsx` (`calls clearError and refreshes agent-related queries`, zero mock calls). [Original failed job](https://github.com/paperclipai/paperclip/actions/runs/40717870938/job/122921463517). That shard passed on `294c5dfc3`. The failure is retained here. The UI shard passed on final commit `687000287`; no unrelated UI edits or local UI suite runs were added. - CI also exposed three existing local-directory recovery failures on `a81668c4b`: the review correction applied the raw agent strategy guard to local directories. [Original failed server job](https://github.com/paperclipai/paperclip/actions/runs/40717870938/job/122921463229). Narrowed that guard to adapter-managed directories, preserving local recovery behavior. All 20 targeted cases (three plugin selection cases and 17 local recovery scenarios) pass afterward, and server TypeScript verification passes. - Final commit `687000287d3339b7e15e35d95515b3c75c273d7a`: [hosted CI is green](https://github.com/paperclipai/paperclip/actions/runs/40720169342), including the corrected server shard, all other server/workspace shards, Runner checks, typechecking and build. All 55 non-skipped PR checks pass. Greptile scores this exact commit 5/5, with no unresolved review threads. GitHub reports no merge conflicts. - Pre-merge hosted acceptance was requested. The maintained deployment CLI stopped during artifact discovery: no published private Cloud composition exists for `687000287`. The current private build requires canonical Core master readiness and standard-image provenance. The Core-only branch preview omits the private computer plugin needed for the journey. No campaign or tenant update was started; private-preview deployment support is a separate prerequisite. No live deployment or provider acceptance run is claimed. ## Risks The receipt is removed only after the existing runtime publishes an admitted session. Earlier admission failures keep their proof. A later normal suspension creates a new receipt. The change preserves existing identity, lifecycle and controller checks. Directory selection remains limited to the exact bound, active, repository-free plugin recovery directory. Foreign owners, changed paths, wrong attempts and explicit Git strategies do not gain this fallback. No schema, API, credential, permission or fleet-default change is required. Existing failed runs and recovery holds remain history. This PR does not edit quarantined state or replay an interrupted task. Rollback restores the previous code and can reintroduce the two admission failures. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. A more specific model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7f10cc0585 |
feat(evals): show full task conversations in E2E reports (#18055)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests verify tasks through the application and retain reports. > - A final screenshot cannot show the complete task conversation. > - Debugging a result requires the earlier requests, replies, and answers. > - This pull request captures task conversations before fixture cleanup and adds them to Test details. > - Reviewers can read each task thread without a running test instance. ## Linked Issues or Issue Description **What existing behavior does this improve?** The task evidence shown in Product E2E campaign reports. **Current behavior** Most task results show screenshots and checks. Screenshots show the final visible state. They omit earlier messages and long replies. Only the first-task suite has a conversation view. **Proposed behavior** Every standard runner attempt captures the saved conversation before cleanup. Test details show full task messages, requests, answers, and native agent progress. Related tasks have separate threads. Failed attempts also retain available history. Partial captures are labeled. **Reason and benefit** Readers can debug a task by reading its conversation. They do not need to reconstruct it from a screenshot or private run logs. GitHub searches found no duplicate PR or issue for this change. This is an improvement to test reporting and does not overlap a planned core feature in ROADMAP.md. **Breaking changes** None. The result adds an optional `taskChat` field. Old results still render. A result without a task has no chat section. Older reports need retained conversation evidence before they can gain this view. ## What Changed - Capture comments through all pages, task briefs, visible request copy, recorded answers, and native progress before fixture cleanup. - Include delegated tasks and standing chats. Preserve available history when a read fails. Report capture limits explicitly. - Add a closed, credential-redacted conversation format to normalized results. Exclude reasoning, raw tool payloads, and provider session metadata. - Render task conversations inside Test details. Retain them through report aggregation and dashboard regeneration. - Add capture, publication-format, regeneration, and browser tests. Update the evidence and security contracts. ## Verification - Report tests: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts task-chat.test.ts first-task-report.test.ts report.test.ts history.test.ts` — 103 tests passed after review fixes. - Browser tests: `node node_modules/@playwright/test/cli.js test --config tests/runner-e2e/report-playwright.config.ts` — 6 tests passed after rebase. - Targeted TypeScript check of the new capture, renderer, validator, report, and browser-test files passed. - Tests cover a 1,105-message thread, a long final reply, event pagination, partial capture, company access, credential redaction, HTML escaping, and report regeneration. - Review regressions reproduced two failures before repair: the task-run API uses `runId`, and item reviews save verdicts in `result.items`. Capture now follows these contracts. Both regression tests pass. - Greptile reviewed commit `471236438e03e54cc4ad85de6653adf45835063c` at 5/5 with no new findings. The separate Paperclip review execution stopped without an assessment and ended neutral. Its cause is not established; this is not a completed Paperclip assessment. - Execution discovery passed for `core-compatibility.runner-codex.local.message-marker`. No paid provider run was started. - Full local commands were attempted. `pnpm -r typecheck` and `pnpm build` stop because the DB package cannot find `tsx`. `pnpm test:run` stops because the general-server lane cannot find `vitest`. These package dependency links are absent in this checkout. - [Remote CI on the final commit](https://github.com/paperclipai/paperclip/actions/runs/40728689517) passed: typecheck and release registry, all general and serialized test shards, Runner static/Rust/Vitest checks, build, canary dry run, all eight browser E2E shards, and final verification. Security and contributor checks also passed. - Open a new task-based result and expand **Test details**. Read **Task chat** and expand each related task thread. Results without tasks keep their existing layout. ## Risks - Published reports now include fixture conversation text. Capture retains only reviewed display fields and redacts credentials. Fixture authors must keep this text suitable for public reports. - Long threads increase report size. Collection has a 30-second request-admission budget and page limits. It labels incomplete evidence and does not change the workflow grade. - Old reports cannot recover conversation text from screenshots. New attempts must retain the evidence. ## Model Used OpenAI Codex (GPT-6). The exact serving snapshot and context-window size are not exposed in this session. Used reasoning, code execution, repository tools, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b14e1ecaf8 |
feat: add a complete agent YAML configuration editor (#18036)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/175/changes/paperclip/175 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators configure each agent through several settings tabs. > - Operators also need one complete, editable configuration document. > - The document must retain each resource service's access and validation rules. > - This pull request defines the `paperclip/v1` Agent YAML format and adds an Advanced editor. > - The editor previews changes, rejects stale core writes, and explains partial saves. > - Operators can edit agent settings in one place and verify the saved state after reload. ## Linked Issues or Issue Description Refs #2219. Related settings work: #10974. **Problem or motivation** Agent configuration spans profile, runtime, instructions, skills, tools, channels, permissions, secrets, keys, and revision history. Operators cannot inspect and edit those settings as one versioned YAML document. **Proposed solution** Add an Advanced agent tab with a YAML editor. Define a shared document schema. Read and write resources through their existing APIs. Separate editable settings, explicit commands, and read-only observed state. **Alternatives considered** A raw agent-row editor would omit related resources. A generic database update would bypass resource ownership. The editor therefore uses the existing resource contracts and records partial results when a later request fails. **Roadmap alignment** This extends the shipped agent configuration capability. It relates to easy AGENTS.md configuration, multi-harness teams, governed tool access, and secrets management. Stored credentials remain redacted. This format is an editable settings view; it does not add company portability or provider consent. ## What Changed - Add the `paperclip/v1` Agent document schema and format documentation. - Add the Advanced route, sidebar entry, YAML editor, validation, change summary, save progress, and reload recovery. - Cover core settings, instruction files, skill pins, tool installs/profiles/policies, channels, permissions, secret operations, API keys, and revision restore. - Add private-keyed configuration revisions checked under the agent write lock. Read the current key through its owner so file-based rotation does not require restarting the server. Preserve unchanged secret markers at their original paths. - Retain drafts and returned credentials after partial saves. Require reload before retrying commands. Prevent duplicate submissions. Keep new key tokens visible through rename and disable saves with an empty change plan. - Send only changed channel settings. Require explicit intent for shared profile edits. Load destination instruction files when changing their location. - Update Storybook fixtures, verification maps, and regression tests. Add browser acceptance through real APIs and PostgreSQL. ## Verification - `pnpm -r typecheck` passes. - `pnpm build` passes. - The focused UI suite passes all 50 tests. The API/revision helper suites pass all 93 tests. - All 65 lifecycle tests pass with a dedicated PostgreSQL database. They exercise real row locks, redaction, revision history, audit, and company access. - All six browser acceptance cases pass. They check saved settings and clears, secret redaction, key creation/revocation, stale drafts, lost-response recovery, instruction relocation, revision restore, and key-token retention through rename. The tests use real resource APIs and storage. The MCP target uses the existing local provider fixture. No model run or provider login is claimed. - Signing-key regressions cover both environment changes and real in-place key-file rotation. The protected database save/fencing cases pass after removing the cache. A local measurement of 1,000 file-backed revision calculations took 48 ms. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and Storybook catalog checks pass. - The full [PR CI run](https://github.com/paperclipai/paperclip/actions/runs/40725425470) passes on `34ed97bbf9`: all 48 jobs, including all eight browser shards. Greptile is 5/5 on this commit, with no unresolved review threads. This final commit refreshes CI after a runner-routing fallback; its source tree matches `fb4cf488cf`. - Paperclip Review and Superagent Security Scan ended neutral because their external agents did not complete a valid assessment/receipt. Superagent passed on `fb4cf488cf`, which has the identical source tree. One request to retry the final scan returned HTTP 404. Neither check reports a code finding. - The local full-suite attempt was blocked by host System V semaphore exhaustion. Acceptance and lifecycle verification used an isolated PostgreSQL 18 container. Earlier CI runners were interrupted. A local combined run hit hook/test timeouts; the full 88-test route file passed after the builds finished. The final full CI run passes. Earlier failure logs and browser traces remain retained outside the repository. To run the browser regression: ```sh pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/agent-configuration.spec.ts ``` ## Risks - Multi-resource saves are sequential. Earlier operations can remain saved if a later operation fails. The editor reports the completed steps and requires reload. - Core settings and managed instruction files have atomic revision/hash checks. Other unversioned resource writes and external supporting files retain a race window after preflight. - Shared profile definitions affect other bindings. The editor requires `updateSharedDefinition: true` for those edits. - The editor follows native service normalization and experimental settings. Canonical reload can add generated access profiles or mask environment values. - New editor dependencies require the repository's normal automated lockfile update. This pull request does not edit the lockfile or add a database migration. ## Model Used OpenAI Codex, GPT-6 family. The exact current model version was not exposed. The work used reasoning, repository tools, code execution, and browser automation. Earlier work in this chat used the configured Codex model before a model switch; its exact model ID and context window were not exposed. The current context window size is also not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4808a3a2b1 |
fix(heartbeat): retry operator Stop after a transaction deadlock (#17895)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/69/changes/paperclip/69 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operator Stop must revoke task authority before it stops provider execution. > - Its transaction locks the task and then the run. A concurrent writer can hold the run while it waits for the task. > - PostgreSQL can abort the Stop transaction with a confirmed deadlock. The cancel API then returns 500. > - This pull request retries only that aborted database transaction, at most three times. > - Provider Stop and activity publication stay after commit, so the retry cannot repeat those effects. ## Linked Issues or Issue Description Refs #17639. Its agent-chat project browser test completed the product assertions, then failed during cleanup when the cancel API returned 500. A real PostgreSQL lock-cycle test reproduces the same `40P01` error in the Stop transaction. Related: #13911 addresses audit foreign-key lock contention. This change handles a different concurrent run-writer path without changing the schema or task/run lock order. Stack base: #17639. This is a separate correctness repair for the sleep migration stack, not another polling migration. ## What Changed - Control the runner timeout regression fixture's clock so it reaches the pending append before expiry; retain production cancellation and late-append assertions. - Retry the Stop authority transaction only after PostgreSQL reports `40P01`. Make at most three attempts, with short bounded backoff. - Discard rolled-back local run state, claim flags, and queued activity publications before each retry. Re-read current authority under fresh locks. - Keep provider cancellation and activity publication outside the retried transaction. Preserve other errors and the existing native lock-conflict response. - Add a real two-transaction deadlock test. Add server-side failure tests for retry exhaustion and errors that must not retry. - Let run-control tests use a validated disposable PostgreSQL database URL. ## Verification Published head: `60ba892ae9fc5faaf8cd0833694b4a66c13323c3`, targeting master after #17639 merged at `1ed0af41867ba40fd1d23292d395588d18c2fcbe`. The normal ancestry merge preserves the complete five-path tested cancellation patch byte-for-byte, including the review repair, and all four unrelated current master paths. The production retry logic remains unchanged from this slice's reviewed implementation; the follow-up adds coverage and its audit. - **228 distinct complete cases pass:** all 48 run-control cases and all 180 native-session runtime cases. The two new targeted cases are included in 48 and are not counted again. Server typechecking, build and module-boundary checks also pass. - The new real PostgreSQL cases reach a failed native run's correlated Stop claim, cancellation fence, board handoff retirement and activity write before raising `40P01`. One case preserves retry authority and commits exactly one cancellation activity/live notice. The other changes coordinator authority between attempts, returns 409, preserves the failed run and handoff, and performs no provider/workspace cleanup. - A negative control removes only the publication-list reset. It fails with two live notices for one durable activity. Production was restored byte-for-byte before final validation. The existing actual two-connection deadlock, bounded attempts, non-deadlock rejection and post-commit provider behavior remain covered. - Preserve the initial new-assertion typecheck failure (optional live-event payload) and the corrected final pass. Service-local test files are typechecked; `src/__tests__` is excluded. The negative-control harness initially expected an untruncated Vitest assertion line; the actual named 1-versus-2 failure and restoration were inspected before correcting its result matcher. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/60ba892ae9fc5faaf8cd0833694b4a66c13323c3/doc/plans/2026-10-28-cancellation-deadlock-acceptance.md) records current scope, commands, isolated fixtures, mutation evidence and limits. Earlier combined-stack counts retain their original revision scope. The existing controlled runner timeout/late-append fixture is retained and passes within the complete 180-case runtime suite. - This addresses the prior Ponytail request to exercise native claimed state before deadlock rollback and re-evaluate authority afterward. The advisory loop-invariant snapshot/named-attempt-bound note remains a maintainability suggestion, not a runtime defect. No production loop refactor or new worker duty is introduced. - **All 58 current checks are clean; Greptile and Ponytail both rate this exact head 5/5.** Ponytail explicitly confirms that the native-claim coverage request is resolved and says Ship. The formerly queued dependency/quality review workflow has completed successfully. Old `fffc`/parent checks and reviews are not current passes; the former pending Snyk status is not called successful. Actual failing Ponytail findings must be fixed. Missing/stopped/paused turns without a verdict are authorized nonblockers. Active unfinished reviews retain their original per-head timer, with a 25-minute monitor cutoff respecting the 30-minute maximum. - No full local repository matrix, live provider execution, external alarm, physical host/database suspension or savings claim is made. No schema, dependency, workflow, shared scheduler contract or shared OS resource changes are introduced. ## Risks - Persistent contention still returns an error after the third attempt. This change does not promise to eliminate every possible lock cycle. - Retrying a write after an ambiguous connection failure would be unsafe. The retry accepts only PostgreSQL's confirmed deadlock abort code; it does not accept transport errors. - No schema or public API contract changes. Current task/run authority and cancellation eligibility are checked again on each attempt. ## Model Used OpenAI GPT-6 via Codex, with shell, GitHub CLI, TypeScript, PostgreSQL, and Playwright tools. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4f8104dec6 |
fix(runner): keep human input tool visible to Claude (#18046)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native task agents must create durable human questions and approvals. > - The request_human_input schema has an object root with a conditional wrapped in allOf. > - The pinned Claude client omits that tool from the API request in an isolated gateway environment. > - An equivalent direct conditional keeps the tool available and preserves validation. > - This pull request fixes the schema and tests discovery through the real pinned client. ## Linked Issues or Issue Description **What happened?** A retained native Claude Plan attempt could save its Plan but received `No such tool available: mcp__paperclip__request_human_input`. A local comparison with Claude Code 2.1.286 reproduces the discovery failure without a provider: the client offers the control tool but omits the human-input tool. It offers both tools after this schema change. **Expected behavior** The client must offer the authorized human-input tool. Invalid question forms must still fail validation. **Steps to reproduce** 1. Serve the current request_human_input definition through Runner's local MCP bridge. 2. Start the pinned Claude client with a local mock Messages API and tool search disabled. 3. Inspect the API request's offered tools. Before the fix, request_human_input is absent. After the fix, it is present. Refs #18034 for the separate saved-Plan readback repair. This PR addresses tool discovery. It does not reclassify or rerun the original failed live attempts. ## What Changed - Put the existing question-payload condition directly on the live and scenario object schemas. A one-branch allOf wrapper is unnecessary. - Add a regression through the pinned native Claude executable, real Runner MCP bridge and loopback mock API. The test uses fake credentials and executes no tools. - Keep validation tests for mixed forms and malformed payloads. Add an explicit missing-payload rejection. - Refresh generated schema identity and the explicitly seeded catalog fixture with the maintained generator. Historical live evidence is unchanged. ## Verification - Native discovery regression red: the control tool is present and request_human_input is absent. - Green: 92 tests pass across native discovery, Runner tool bridge, core contracts and semantic tools. The final argument-visibility regression and core contracts also pass together (21 tests). - Runner package typecheck passes. - Runner package build passes. Its first run found stale generated contract identities; regeneration resolved that check. Both logs are retained. - No real provider request, paid eval, sandbox allocation or automatic retry. - Local verification is scoped to the Runner package. Full repository CI must pass on this PR's exact head before merge. New live Plan acceptance is still required. ## Risks - The conditional has the same JSON Schema validation meaning. The provider-facing object shape changes, so the generated catalog identity changes. - The native regression uses the optional platform CLI already required by the pinned ACP profile. It does not establish provider billing, model behavior, or full Plan acceptance. - Optional case/workflow tool discovery is a separate confirmed gap tracked in #18051. This human-input fix does not qualify those tools. - No database migration, permission, budget, workflow, lockfile or UI change. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.12 |
||
|
DottaandPaperclip
|
567c043b17 |
feat(server): expose bounded hot-upgrade readiness (#18037)
## Thinking Path > - Paperclip coordinates agent work and keeps its authority in the server. > - Remote native runners can preserve execution through a server replacement. > - Deployment operators need evidence that current runs support that recovery. > - Process-local drain counters do not establish this fact. > - This PR adds an optional, operator-only readiness observation from existing records. > - A deployment consumer can use that report while the existing runtime retains recovery authority. ## Linked Issues or Issue Description Refs #17439. **What happened?** A deployment cannot distinguish recoverable remote native executions from runs that require a task drain. The existing status endpoint reports only process-local counts. **Expected behavior** An instance operator can request a bounded readiness report without stopping admission or exposing checkpoint material. Missing ownership or persistence evidence must prevent readiness. Ordinary status reads must keep their current cost. **Steps to reproduce** Run a remote native task. Read the task-drain status. It reports active work but supplies no recovery eligibility. Deployment code therefore has no affirmative signal to select an active-run handover. ## What Changed - Add `GET /api/instance/task-drain?upgradeReadiness=1` for instance administrators and existing Cloud control callers. - Return aggregate eligibility from current run identities, controller leases and bound opaque checkpoint headers. Unknown work, pending admission and incomplete reads prevent a positive report. - Bound the explicit read to 1,001 rows, encrypted headers of at most 256 KiB each and 8 MiB total, and a two-second SQL statement timeout in a read-only transaction. Keep ordinary status reads in memory. - Add PostgreSQL, count-transition and route authorization tests. Update the API contract, operator docs and access maps. ## Verification - All seven readiness cases pass against an isolated PostgreSQL 18 fixture. They cover missing checkpoints, legacy adapters, missing process/session identities, foreign owners, expired leases, missing lease tokens, failure state and in-flight work. The positive SQL fixture uses the real Runner checkpoint writer and encryption. Regular sandbox workspace identity and plugin environment-lease identity are tested with independent IDs. Stopped/released/undispatched plugin tasks and every mismatched identity field are rejected. Local/legacy/malformed/oversized or mismatched headers are rejected. A held PostgreSQL table lock proves the two-second cancellation budget, rollback/connection release and unrelated query responsiveness. - All 123 focused readiness, instance-settings and Cloud control assertion tests pass, including operator-only access and no readiness read on ordinary status requests. Runner’s 168 checkpoint tests pass. Runner and server typechecks pass after review fixes. - Workspace typecheck, full build, module-boundary and map checks passed locally before the master rebase. Current-head GitHub CI is now green, including typecheck, build, all grouped/serialized server and Runner tests, browser shards, policy and review-dispatch checks. Greptile completed cleanly at 5/5 for `f15fb0c7e9dfb975090cd5d791a1c8e6701275f8`; all actionable threads are resolved. - The broad local test run was stopped after the macOS embedded PostgreSQL fixture failed to bootstrap. The SQL test was then run on the isolated PostgreSQL fixture. No assertion or timeout was weakened. One unrelated existing route test exceeded its 15-second limit under concurrent local builds; it passed on one isolated retry and the subsequent whole route suite passed. - Hosted active-run upgrade acceptance has not run. This PR supplies an observation; it does not change rollout policy or establish live continuity by itself. ## Risks This is an instance-wide observation, so company members and agent keys must not receive it. A readiness snapshot can age as work changes; shutdown and startup retain their existing execution authority and fencing. The report does not validate a replacement artifact or authorize a schema migration. It decrypts only bounded headers through the checkpoint owner and delegates format/identity inspection to Runner; full journal restoration and physical sandbox authentication remain with recovery. No migration, recurring query, new checkpoint write or wire protocol change is included. The additive Runner header inspection contract reuses its existing parser. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, source inspection, tool use and code execution. The host does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the affected contracts; the broad-run limitation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1ed0af4186 |
fix(server): wake native chat progress from committed changes (#17639)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat users receive coarse progress updates while native runs execute. > - A recurring chat pass scans saved run events even when no progress changes. > - This traffic prevents an idle database from staying idle. > - Events, lineage changes, and publication state already identify when projection can change. > - This PR gives native progress its own commit-driven worker and schedules only unfinished work. > - Idle progress stops scanning while restart and recovery preserve delivery. ## Linked Issues or Issue Description Refs #17606. This is slice 29 of the recurring-server-work stack, based on `codex/sleep-28-chat-run-milestones`. It owns native chat progress only. Related historical optimization: #16845. The parent and this PR must land in stack order. **What happened?** Native chat progress is still called by the one-second chat reconciliation loop. It repeatedly selects run events with no new work. Its cursor also truncates PostgreSQL timestamp precision, which can repeat a rejected full page forever. **Expected behavior** Committed events and eligibility changes should wake the projection owner. Existing rows should reconstruct work after restart. A settled worker should have no recurring database activity. Full batches, lock contention, and errors must retain recovery. **Steps to reproduce** Start an otherwise idle instance with chat history. Observe the native progress lane in the recurring chat coordinator. For the cursor defect, create a full page of suppressed candidates sharing a microsecond timestamp and one eligible later candidate. The old cursor fails to advance. ## What Changed - Extract native progress from lifecycle milestones into its owning service. Keep the public combined API and budget compatible. - Register a scoped worker with the existing delivery coordinator and scheduler. Remove native progress from the recurring chat coordinator. - Notify before allowlisted event writes in both persistence owners. Preserve outer commit, rollback, duplicate detection, and the ordinary non-progress log fast path. - Extend the existing lineage/authority notification boundary once. Failed-publication replay and operator retry carry their issue scope. First working publications signal progress so commit ordering cannot strand an already-inspected event. - Limit notification payloads to company/task IDs so wider caller inputs never serialize database handles or private fields. Load task-binding fields only for progress event kinds; other native/recovery events retain their narrow lock query. - Preserve current safe phrases, cadence, phase deduplication, provider policy, authorization, and final-output precedence. - Retry concrete issue/run lock contention. Disarm deleted or terminal candidates. Preserve scoped dirty work across batches, failures, drain and shutdown. - Preserve PostgreSQL timestamp precision in pagination. Add a bounded regression for the former infinite rejected-page scan. - Document the producer audit, lifecycle contract, failures, and remaining limits in `doc/plans/2026-10-25-native-chat-progress-sleep-audit.md`. ## Verification Published head: `b7ee820e49fd06c780b36b79d9b1c0883557fb73`, targeting master after #17606 merged at `5f0da562dbfdf3229f3cd4bb3dc742ab541986b4`. A normal ancestry merge preserves the complete 18-path native-progress patch byte-for-byte relative to its verified parent, with all six unrelated current master paths intact. No new production change was introduced during final reconciliation. - **163 distinct selected cases pass on the verified unchanged patch:** 137 across eight complete suites (native progress, atomic event append, all five milestone worker suites, and app admission wiring), plus 26 real chat integration cases selected by `native progress`. The other 1,595 chat integration cases were skipped, not passed. - The full worker suites cover restart, committed events, trace reprojection, duplicate/no-progress events, first-working publication ordering, concrete lock contention, issue scope, backlog, drain/resume, failures and shutdown. Settled history leaves no recurring SQL. The real chat cases verify operator retry and replay, microsecond pagination, authority, cadence, phase deduplication, final/question precedence and redaction. - The sole earlier parent conflict preserves both this slice's issue-scoped retry wake and the exact voice conversation scope. The current parent's milestone producers, onboarding completion guard, identity query/read-scope coverage, migration 0397 and voice cleanup/retention repairs remain present. - Server production typechecking, build and module-boundary checks pass. The selected server tests live under the excluded `src/__tests__` path and were executed separately. Each database owner uses a new isolated app-migrated PostgreSQL fixture. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/b7ee820e49fd06c780b36b79d9b1c0883557fb73/doc/plans/2026-10-25-native-chat-progress-sleep-audit.md#current-milestone-parent-acceptance-october-28) records current counts and scope. Older combined checks and failures retain their original revision scope and are not added to 163. No new table, dependency, workflow, migration, or scheduler contract is introduced. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing verdict is waived. Older head/parent reviews are not current passes. Actual failing Ponytail findings must be fixed. Missing/stopped/paused turns without a verdict are authorized nonblockers; active unfinished reviews retain the original per-head timer with a 25-minute monitor cutoff respecting the 30-minute maximum. - No complete local repository matrix, live provider tenant, external wake, physical host/database suspension or savings claim is made. Following stack duties remain separate. ## Risks - Signals are process-local under the existing single active writer contract. Independent SQL writers or another active process need an explicit wake integration. - Allowlisted direct log writes now use a transaction. Native bound runs register transaction intent; unrelated direct event types retain their atomic-statement path. - The shared eligibility boundary wakes both lifecycle and native projection. It does not decide provider policy or copy event payloads. - The existing 24-hour selection window, event-time cadence and authority checks remain. A suppressed event is not retried just because wall-clock time passes. - This establishes worker quiet, not whole-instance suspension. Cloud alarm coverage and conservative sleep gates do not change. Other chat maintenance remains on its existing cadence. - This actual-master reconciliation still requires fresh exact-head review and CI before protected merge. Subsequent slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code execution and browser-test tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0791da0cc5 |
fix(server): synchronize routine history during native status commit (#18033)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/173/changes/paperclip/173 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages work assigned to AI agents. > - A routine stores each invocation and its execution task. > - Native completion commits task status without using the public issue route. > - That route synchronizes routine history, but the native status transaction did not. > - A successful task could remain listed as an unfinished routine invocation on a local instance. > - This pull request uses the existing database projection in the native status transaction. > - Task status and routine history now commit or roll back together. ## Linked Issues or Issue Description **What happened?** A real routine execution completed successfully. Its task had status `done` and retained its routine origin and invocation identity. Public routine history still returned `issue_created`, with `completedAt: null` and a linked task whose status was `done`. The existing five-minute reconciliation runs only on cloud-managed instances. It does not close this path for a local instance. **Expected behavior** An authoritative native task status change must update the matching routine invocation through the existing routine status rules. It must not update another task, routine or company. **Steps to reproduce** 1. Create a routine on a local instance and run it through the public routine API. 2. Let its native execution finish and commit the task as done. 3. Read the routine run history. Before this change, the invocation can remain `issue_created` while its linked task is done. **Paperclip version or commit** The retained failure used merged production `39e811c9c5`. The missing hook is also present on this branch's base, `8049fbf51d`. Related: #16633 contains the existing cloud-only reconciliation. A bounded search found no duplicate native synchronization fix. ## What Changed - Synchronize routine history inside the native status transaction after the task status projection. - Require matching company, invocation and linked task before synchronization. Preserve supported custom display origins. - Add database-backed tests for terminal and active states, replay, invalid custody, rollback and a real plugin-managed routine dispatch with a custom origin. - Keep the routine status projection in a database-only helper. Both the routine service and native status committer use it without loading heartbeat execution from the committer. ## Verification - Own-source server TypeScript check passed after the helper extraction. It includes the new test source and production dependencies. Local package exports point at another checkout, so the check used explicit paths to this checkout's source and disabled declaration emit. - Existing pure `status-arbiter.test.ts`: 20 tests passed. `git diff --check` passed. - Initial required CI run 40711603938 built and typechecked successfully. It executed all ten new database tests: nine passed; the public-dispatch fixture failed because it assumed status version zero. The fixture now reads the saved version and decision binding. - That CI run also exposed a heartbeat mock dependency cycle. The database-only helper removes the committer-to-routine-service-to-heartbeat import. Required CI on the new head must confirm both repairs. - Initial browser shard 3/8 failed during cleanup: legacy run cancellation waited for a heartbeat row lock. Its cause is under investigation; the failure remains recorded. - A local runtime-execution check failed during module import from stale package exports, before any tests or database setup ran. It is not passing evidence. No local database, provider or live eval was run. - Latest-head required CI run 40717315212 executed all ten new database tests successfully in server shard 5/12 (2,163 passed; three unrelated skips). The heartbeat runtime suite executed 23 tests successfully in server shard 9/12 (1,704 passed). - Latest-head browser shard 3/8 passed all 33 tests. This does not establish the cause of the preserved earlier cleanup lock timeout. - Required CI run 40717315212 completed successfully on head `2d77f4302b36303741abad07266bbe9e376ba6a5`. All 48 workflow jobs are green after one failed-jobs-only infrastructure retry. The first attempt had seven explicit runner shutdowns and three lost-runner communication failures. The exact chat shard host was independently confirmed as an AWS Spot termination. Original logs and annotations remain retained. - Greptile completed at 5/5 on this exact head, with no outstanding findings. All review threads are resolved. The independent follow-up review found no issues. - No live qualification or corrected eval pass is claimed. The PR remains open and unmerged, with auto-merge off. ## Risks - Matching routine executions perform the existing bounded status synchronization queries inside the status transaction. A database failure rolls back both projections. - Inconsistent invocation or linked-task metadata now prevents synchronization. No unrelated invocation is inferred or repaired. - The existing scheduled repair remains unchanged. This PR does not change accounting, provider behavior or eval budgets. ## Model Used - OpenAI `gpt-6.1-sol`, with reasoning and tool-assisted source inspection and coding. The session did not supply a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run relevant credential-free tests locally and they pass; database coverage is pending CI - [x] I have added or updated tests where applicable - [x] I have checked documentation; the existing routine status contract is unchanged - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5f0da562db |
fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
143d20e466 |
fix(server): retain native Codex Plan write boundaries (#18034)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Codex task chat keeps a saved Plan inside the turn that wrote it. > - The compact conversation API omits ordinary tool history. > - That omission also removes the write receipt needed to place the Plan. > - This pull request retains a small successful write receipt in conversation reads. > - The UI can then keep the saved Plan at its write boundary after readback. ## Linked Issues or Issue Description **What happened?** A real native Codex Product E2E attempt saved its Plan and displayed it inside a settled turn. The Plan used the fallback position. The browser's final conversation-only API response contained no `write_document` receipt. The full event snapshot retained the successful write. The native placement assertion failed and remains failed in the original report. **Expected behavior** Conversation readback must retain enough evidence to place the saved Plan at its successful write boundary. It must keep arbitrary tool bodies and ordinary tool history out of the compact response. **Steps to reproduce** 1. Save a canonical Plan through a native `write_document` call. 2. Read the run with `view=conversation`. 3. Before this fix, the response omits the write receipt and the UI places the Plan at the end-of-turn fallback. Related eval evidence repair: Refs #18026. This is a separate product readback fix. It does not resolve the failed human-input call or the Claude tool-availability failure from those attempts. ## What Changed - Include only successful canonical `plan` document results, matched to a native `write_document` call in the same run, turn, session and source instance. Ordinary document writes stay excluded. - Build the small receipt inside PostgreSQL before its body crosses the database connection. Preserve only fixed event metadata, tool identity, the `plan` key and mutation disposition. Keep the existing payload decoder for ordinary conversation events. - Keep cursor paging, run scope, ordinary-tool exclusion, and full event reads unchanged. - Document the compact read contract in OpenAPI. - Scope this fix to the canonical native dynamic-tool call/result pair used by Codex. The retained Claude sample proves compatibility with that captured shape. It does not qualify every current ACPX Claude producer. Provider-activity-only paths and OpenCode keep the explicit fallback. ## Verification - Real PostgreSQL red: the new boundary test fails because the first selected event is assistant text rather than the saved write. The six existing tests pass. - Real PostgreSQL/API green: 36 tests pass across task cold-open and OpenAPI routes on an isolated Linux current-master checkout. - UI: 104 transcript tests pass, including completed native placement from the body-free receipt. - Review regression red: a later notes write incorrectly enters conversation readback (one failure, six passes). After the fix, the PostgreSQL/API suite passes all 36 tests. - Fixed the new UI fixture's string-to-Date type mismatch found by hosted CI. Original failed CI remains visible; fresh exact-head checks are pending. - Existing token/style gates pass. - Retained native Codex and Claude fixture shapes both pass. Each fixture includes the canonical item pair and parallel provider `tool.execution.*` activity. All identifiers and document bodies in the fixtures are synthetic. This captured-shape test is historical readback compatibility, not fresh current-source Claude acceptance. The Codex canonical pair is emitted by `packages/paperclip-runner/src/drivers/codex/codex-session-server-requests.ts` in the admitted dynamic-tool handler. - Projection-collision regression red: a non-Plan result selected by conversation identity was relabeled as a Plan. The SQL projection now uses the same full Plan predicate as selection. The regression and both native fixtures pass. - The database test checks cursor paging, a Plan followed by notes, failed/read/control-plane/unrelated-turn exclusion, schema and ID bounds, omission of a large tool body, unchanged full-history reads, and ordinary conversation payloads containing NUL. - Local server typecheck fails on 80 installed workspace export mismatches outside changed paths. No changed-path type error or local full build pass is claimed. Clean hosted checks must pass before merge. - Current-head hosted CI and legacy status pass. One failed-check retry followed recorded runner shutdowns and an unrelated native-distribution timeout; that test passed all 33 cases locally. Original failed logs remain retained. - Zero provider calls or paid attempt retries. Original failures and accounting journals remain unchanged. This is not a new live qualification result. ## Risks - Conversation pages include matching Plan results within the existing 1,000-row page limit. The query adds a run-scoped match against canonical tool calls and projects results inside SQL. PostgreSQL still reads the stored JSON to select receipts; private bodies are not transferred to the application. It adds no separate database round trip or background work. - Matching requires the existing canonical call/result identities. Missing or malformed historical receipts keep the explicit fallback. This fixes compact conversation readback; it does not redesign placement in an unrelated full-history multi-document UI flow. - This restores the existing UI write-boundary mechanism. It does not change Plan acceptance, permissions, or completion rules. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7d75ae0569 |
fix(server): recover Teams attempts from saved claim deadlines (#17592)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves a claim before each provider operation. > - A crashed operation can leave that claim unfinished. > - Periodic maintenance scans for abandoned claims even when none exist. > - This PR wakes recovery from committed changes and the saved claim deadline. > - Empty work becomes quiet while unknown effects remain safe to review. ## Linked Issues or Issue Description **What existing behavior does this improve?** Teams abandoned file-transfer recovery. This is slice 27, stacked on consent expiry #17590. **Current behavior** The remaining Teams maintenance callback scans every 30 seconds for expired transport claims. It runs from shared publication maintenance. **Proposed behavior** The Teams owner reconstructs existing claim deadlines at startup and reloads a transfer after its committed update. At expiry it clears the claim and preserves the existing unknown-outcome or conflict state. Normal receipt settlement removes the deadline. **Reason and benefit** Remove the last periodic Teams maintenance scan without resending a possibly delivered card or file. **Breaking changes** No schema or public API change. Explicit diagnostic recovery methods remain compatible. Consent expiry stays with its separate parent worker. ## What Changed - Add a Teams-owned worker for consent_sending, uploading, file_info_sending and conflict rows with a saved attempt expiry. - Notify from the existing versioned update before writes. Preserve transaction ownership and shared uncertain-write reconciliation. - Expose a narrow per-transfer recovery method that uses the existing lock order, integrity checks and atomic projection. - Remove the periodic Teams maintenance callback and its publication-maintenance invocation. - Add real claim, receipt, conflict, backlog, restart, precision, failure, drain and shutdown tests. Wire the owner through existing app admission gates. - Retain the existing signoff cancelled-run selection test and GitHub inline retry-window race assertion. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-attempt-recovery.md` and its linked audit. ## Verification Published head: `24c54399909105984e2a10f5869064984f69c472`, targeting master after consent #17590 merged at `4a89456bb3f9aef87cfe0e922e8b9d1a777a6c7e`. A normal ancestry merge reconciles the landed squash with the previously verified child. The complete 13-path binary patch is identical before and after reconciliation, and all 30 unrelated master paths are preserved. This adds no new worker behavior; the 239-case local evidence below belongs to the verified unchanged patch. All 58 current-head checks are now clean, including the complete required CI run. Greptile is 5/5 at this exact head. The current Ponytail turn is paused without a verdict, which is an explicitly authorized nonblocker; no failing review is waived. The complete 13-path slice preserves the reviewed attempt implementation and inherits the parent's report migration 0397, bounded consent worker and retention CI test repair. No new schema, queue table, shared scheduler contract, workflow or lockfile change is added by this duty. - **239 distinct selected cases pass:** 221 across five complete transfer/publication/consent/runtime/app-wiring suites, 16 selected real Board/Teams-authority/GitHub-retry integration cases, and two delegated Slack policy cases. Other chat integration cases (1,579) and connection-management cases (81) were skipped, not passed. Each database owner uses a new isolated PostgreSQL database. - Real Board publication saves an active attempt deadline; its confirmed receipt retires recovery, and SQL stays quiet afterward. The inherited consent producer and authority cases still pass. Lifecycle coverage exercises claim stages, unknown-outcome quarantine without repeating provider I/O, conflicts, rollback, restart, precision, backlog/fairness, failure recovery, drain and shutdown. The changed GitHub metadata/retry race passes without duplicate assessments. - The old server10 CI failure, run `40557802358` job `122390238707`, was a delegated Slack invitation rate-window assertion (2,122 passed/one failed/15 skipped in that shard). The verified parent already contains its Date-only fixture fix. Both linked and invitation cases now pass their full caller/target/shared-policy scenarios on a fresh server fixture; no duplicate policy change or new owned file was added. That old failure is preserved rather than called a pass. - Server production typecheck, build and module-boundary checks pass. Test files were executed; production typechecking does not statically check them. The existing test-only signoff cancelled-run selection fix is retained but was not executed locally in this acceptance; fresh remote E2E gates remain required. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-25-teams-attempt-recovery-sleep-audit.md#current-consent-parent-preparation-and-acceptance-october-28) records current scope, commands, counts and limits. Earlier 235-case and combined-stack counts belong to their stated revisions and are not added to 239. The [stack ledger](https://github.com/paperclipai/paperclip/blob/24c54399909105984e2a10f5869064984f69c472/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository suite, live Teams/Slack tenant, external wake, physical suspension or savings claim is made. - **Exact-head Greptile 5/5 and all 58 checks are clean.** Actual failing Ponytail findings must be repaired; a missing/stopped/paused turn without a verdict is an authorized nonblocker. An active no-verdict turn waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older reviews and parent checks are not treated as current passes. ## Risks - Recovery must not infer that an expired claim means a provider effect did not happen. It retains unknown outcomes and does not resend. - Late receipts must still match current claim identity. Current-state rechecks and existing protocol tests preserve this fence. - Supported deployment uses one active writer process through its owning DB client. Independent SQL and additional processes need a separate wake contract. - Cloud external alarm coverage and sleep eligibility are unchanged. Other chat duties still run periodically until their own migrations. - Voice-report parent review repairs are propagated: explicit single-active-writer scope and an indexed report window. Thirty report-worker/startup tests pass on the combined child. Fresh remote checks, older stack findings and final master reconciliation remain required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
SudheerandClaude Opus 5.5
|
2e5bbde8cc |
fix(ui): remove duplicate audit links from agent overview (#17867)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The agent detail page shows an overview of one agent in the board UI > - The streamlined shell shows the agent contextual sidebar beside this page > - The sidebar has an Audit group with Activity, Runs, Costs, and Budgets links > - The overview also ends with an Audit card grid that has the same four links > - The duplicate links add noise and make the overview longer with no benefit > - This pull request removes the Audit card grid from the agent overview > - The benefit is a shorter overview with one clear place for audit navigation ## Linked Issues or Issue Description **What existing behavior does this improve?** The agent overview page shows a section of Audit link cards at the end of the page. **Current behavior** The overview shows four cards: Activity, Runs, Costs, and Budgets. Each card opens the company audit page, filtered to the agent. The agent contextual sidebar shows the same four links. Both use `agentScopedAuditHref`. #12746 added the sidebar links. #12747 added the overview cards in the same PR stack. **Proposed behavior** Remove the overview cards. Keep the sidebar Audit group as the single entry point. **Reason and benefit** Operators see each audit link one time. The overview stays focused on agent status and recent tasks. ## What Changed - Remove the Audit card grid from the agent overview in `ui/src/pages/AgentDetail.tsx`. Before: <img width="1488" height="377" alt="Screenshot 2026-10-27 at 5 33 29 PM" src="https://github.com/user-attachments/assets/fd27fee2-63c4-420e-8b39-405e4a633f63" /> After: <img width="1526" height="271" alt="Screenshot 2026-10-27 at 5 33 41 PM" src="https://github.com/user-attachments/assets/bca0635d-3be6-42cd-b1d6-dc36b89ae1b0" /> ## Verification - Run `pnpm --filter ./ui exec tsc --noEmit`. The result is clean. - Run `pnpm --filter ./ui exec vitest run src/pages/AgentDetail`. All 7 files and 56 tests pass. - Run `pnpm check:token-gates`. All gates are clean. - Manual check: opened an agent overview page. Made sure that the Audit cards are not at the end of the page. Made sure that the sidebar Audit links still open the agent-filtered audit pages. ## Risks - Low risk. The change only removes UI markup. - The agent-scoped audit routes and the legacy URL redirects do not change. - Operators who used the overview cards must use the sidebar Audit links. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code. The model used tool use, code search, git history inspection, and local test execution. Reasoning mode: adaptive thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
4a89456bb3 |
fix(server): schedule Teams consent expiry from saved deadlines (#17590)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Teams file delivery saves consent and transfer state in the database. > - Unused consent must expire even when no new message arrives. > - The maintenance loop currently checks this state every 30 seconds. > - This PR schedules each existing consent deadline and wakes on committed changes. > - Empty or settled consent work stops querying while expiry remains recoverable. ## Linked Issues or Issue Description **What existing behavior does this improve?** Teams unused file-consent expiry. This is slice 26 of the recurring-server migration stack, following merged #17586 and targeting master. **Current behavior** The Teams maintenance callback scans for expired unused consent and abandoned transport attempts together, even when consent work is empty. **Proposed behavior** The Teams owner reconstructs unused consent at startup and reloads one transfer after a committed write. The shared scheduler runs its saved expiry. Abandoned attempts keep their current recovery until a separate slice migrates them. **Reason and benefit** Remove unnecessary consent queries without losing expiry, atomic publication projection, or recovery after restart. **Breaking changes** No schema or API change. The explicit combined recovery method remains available. Cloud sleep eligibility and external alarms remain unchanged. ## What Changed - Add a Teams-owned scoped deadline worker for the three unused consent phases. Registered reconstruction yields after one 100-row page per scope refresh and resets cursors on registration; the explicit diagnostic inventory still returns every eligible row. - Signal before insertion and the existing versioned update inside the owning transaction. Nested writes wait for outer settlement. - Extract the existing locked per-transfer recovery operation. Preserve authority checks, atomic projection, bounded lock waits and unknown-outcome quarantine. - Remove unused consent from periodic maintenance selection and execution. Keep abandoned attempts as a separate obligation. - Register the worker through existing app lifecycle gates. Add lifecycle, producer, production wiring and precision regressions. - Document supported writers, remaining sleep limits and the failure ledger in `doc/idle-teams-consent-expiry.md` and its linked audit. ## Verification Published head: `9e84cfab1f795279c8cec413a946631f1d781bd2`. This follow-up changes only the inherited voice-retention test and its CI failure ledger; all production behavior is unchanged from `36f43922`. It normally merges actual landed master/report parent `8049fbf51d0701ca6b242e4b1198e5449f7661b2`. The verified 12-path Teams patch was byte-identical across the actual-parent ancestry integration. This test-only CI follow-up adds one existing retention test path, bringing the final scope to 13 paths. Every unrelated master change, report documentation fix, and migration 0397 is preserved. No database, shared-worker, workflow or lockfile change belongs to this slice. - **223 distinct selected cases pass:** 209 across five complete transfer/publication/consent/runtime/app-wiring suites, plus 14 selected real Board-producer and Teams-authority integration cases. The other 1,580 integration cases were skipped, not passed. The protocol and production consumers use separate new PostgreSQL databases migrated through the application migrator. - A real 105-transfer regression failed against the original worker because one registered startup pass selected 105 rows instead of yielding at 100. The corrected worker uses the existing paged shared contract. Restarting the same service midway through discovery selects the first 100 again, restores all 105 saved deadlines and expires them at the original deadline. SQL is quiet before the deadline and after settlement, with no consent-card or upload requests. The one-shot inventory still returns all 105 rows. - A guarded mutation removing only cursor reset fails at five rows instead of 100 after restart. The original production source was restored before the complete suites. The initial regression fixture incorrectly faked network timers before its first SQL connection and timed out; that setup failure is retained separately from the corrected 105-versus-100 regression. No assertion or timeout was weakened. - The real Board producer registers the worker before publication, commits the transfer wake, preserves the saved expiry, remains SQL-quiet until due and atomically projects expiry to a cancelled publication without another provider request. Existing authority and consent transitions remain covered. - Complete lifecycle coverage retains all three consent phases, precision rounding, rollback and held outer commit, exact-row hints, expiry recheck, failed projection retry, corrupt-row fairness, drain/resume, shutdown and separation from abandoned-attempt maintenance. Production server typecheck, build and module boundaries pass. Server production typechecking does not statically check test files; those files were executed. - [Current bounded reconstruction and failure ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#bounded-reconstruction-and-current-parent-acceptance-october-28) records exact scope, counts, guarded mutation, corrected timer setup, earlier fixture failures and limitations. Earlier 222-case and combined-stack results belong to their stated historical heads; they are not added to 223 or claimed rerun here. The [stack ledger](https://github.com/paperclipai/paperclip/blob/36f43922b0a408f76adb7de31280812e9b752019/doc/plans/2026-10-24-recurring-server-sleep-stack.md) retains inherited failure history. No complete local repository-suite, live Teams tenant, Cloud wake, host/database suspension or savings claim is made. - **Fresh exact-head Greptile 5/5 and all CI are required.** Actual failing Ponytail findings must be repaired. A missing/stopped/paused turn without a verdict is an authorized nonblocker; an active no-verdict review waits at most 30 minutes from its original request/start (25-minute monitor cutoff). Older scores and parent checks are not accepted as current passes. - **Preserved inherited CI failure:** prior head `36f43922` had exact-head Greptile/Ponytail 5/5 but CI run `40708566195` server6 job `122887395220` failed the retention fairness assertion (2,291 passed/one failed/one skipped in that shard). A fixed final UUID reproduces the wrong expectation that an undiscovered page-two row is already deleted. The corrected test controls time, expects that row to remain before discovery and requires its deletion in the next fair batch, then all remaining history and idle SQL. The existing five-second wait budget and global timeout are unchanged. All 13 retention lifecycle cases pass; bypassing company rotation fails the corrected assertion, and the original shared helper is restored byte-for-byte. Earlier 223-case Teams acceptance remains valid for unchanged Teams source and is not relabeled as a new complete repository run. [Full CI repair ledger](https://github.com/paperclipai/paperclip/blob/9e84cfab1f795279c8cec413a946631f1d781bd2/doc/plans/2026-10-25-teams-consent-expiry-sleep-audit.md#inherited-retention-ci-fairness-repair-october-28). Fresh CI and exact-head reviews are required for this test-only follow-up; old scores are stale. - **Current merge gates:** all 58 checks are now clean for this exact head and Greptile is 5/5 with no actionable findings. The current Ponytail turn is paused without a verdict, an authorized nonblocking state. Workflow `40712464720` attempt 1 independently hit npm `ECONNRESET` during the serialized8 job's tool self-installer, before its tests ran. Exactly one targeted job retry was requested after the original workflow finished; attempt 2 completed successfully on the unchanged head. No code, dependencies, workflow, timeout or review timer changed for that retry. The original setup failure is retained in its [job log](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122901243926); the successful replacement is [job 122910350074](https://github.com/paperclipai/paperclip/actions/runs/40712464720/job/122910350074). ## Risks - Missed supported producer signals could delay expiry until restart. Real insertion, cancellation, consent transitions, caller-owned transactions and Board publication are covered. - Corrupt or locked rows must retain retry without starving healthy work. Paging, fairness, atomic rollback, redaction, drain/resume and shutdown tests cover these paths. - Independent SQL writers and other processes need their own notification contract. No persistent DB listener is added. - Attempt recovery still scans periodically until the next slice. A local deadline is not a durable Cloud alarm. This PR does not claim whole-instance sleep. - This slice is reconciled with its landed parent; current-head remote checks/review and live-master composition proof still gate merge. Later stack duties remain separate PRs. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.11 |
||
|
DottaandPaperclip
|
820c458465 |
fix(runner): restore per-turn controller checkpoints before resume (#18018)
## Thinking Path > - Paperclip coordinates agents and preserves their work between turns. > - Remote native sessions save controller checkpoints in the database. > - External persistence leaves a small identity marker on local disk. > - A later `per_turn` continuation must restore that checkpoint before it reads the controller identity. > - The restore path handled warm sessions only, so valid per-turn continuations failed before provider execution. > - This PR restores the referenced checkpoint and retains the existing ownership and lifecycle checks. ## Linked Issues or Issue Description Related: #17439 added external controller persistence and the warm restore path. #18011 repairs warm preparation and fresh bootstrap checks; it does not cover ordinary per-turn continuation. **What happened?** A remote OpenCode run completed with `per_turn` lifecycle and saved its controller checkpoint in the database. Its next turn reused the normalized session. Admission read the remaining external-persistence marker as a full controller file and failed with `runner_state_identity_mismatch: durable_identity_unreadable`. The prior run's complete checkpoint and harness backup still existed. The failure occurred before provider work started. **Expected behavior** Restore the exact prior controller checkpoint before identity inspection. Then apply the existing company, task, session, prior-owner and suspended-state checks before a provider can start. **Steps to reproduce** 1. Complete a remote native OpenCode task with `per_turn` lifecycle and external controller persistence. 2. Send a follow-up that resumes the same saved session without a session reset. 3. Observe admission reject the valid external controller marker before provider execution. **Paperclip version or commit** Observed at `40c1ab36cdac70c85dc8fe5e912431834fdb691b`. The same warm-only restore remains on current master. **Deployment mode** Hosted Linux with Daytona and native OpenCode. The original model was `openrouter/deepseek/deepseek-v4.1-flash-1017`. ## What Changed - Restore external controller state for ordinary remote per-turn continuations through the existing checkpoint loader. - Bind that read to the current company and task, the marker's run, and the expected normalized session. - Keep local, warm and restart-recovery paths, prior-owner checks and harness backup checks intact. - Refresh a prior-run projection from the latest checkpoint after a rejected active-owner attempt. Keep current-run full state and the existing legacy file-backed fallback intact. - Add restoration and denial regressions, execution documentation and session verification recipes. ## Verification - Red: the real controller generated an external marker; a valid per-turn OpenCode continuation failed with the exact `durable_identity_unreadable` error before tool-authority construction. The repair passed the same case and its ownership guards. - Review regression red/green: a real controller advanced its checkpoint after an active-owner rejection. Before the follow-up fix, a later valid continuation retained a null attach seed. After the fix, it restored the newer attach template and prepare command. Legacy full-file roots with no database checkpoint retain their existing behavior. - At current head `1f6589233e6121d082a0c0c4444ea895b6e8cd7c`, 86 focused executor cases and all 14 execution-driver cases pass. Coverage includes checkpoint restoration, direct/backup suspension, Codex, legacy files, warm admission and canonical bootstrap, missing or malformed evidence, wrong bindings, active/foreign owners and unsuspended runners. Tests stop before provider launch. - The real PostgreSQL encrypted-checkpoint integration passes, including normalized-session refusal before write and stale-writer fencing. Independent source review found no actionable safety defect. - `node scripts/check-maps.mjs`, `git diff --check`, full `pnpm -r typecheck` and `pnpm build` pass on current head. - The local full suite did not complete: native PostgreSQL startup failed under the host sandbox, an initial Docker rerun lacked migrations, and the migrated retry crossed the review-fix revision change. Logs are retained; that mixed-revision run was stopped and its owned database removed. Full test evidence comes from current-head CI. - Current-head CI passes: 53 successful checks, two intended Storybook skips, none pending or failing. Greptile is 5/5 on that exact head with no open review threads. The only manual merge conflict with #18011 was in the verification recipes; both recipes and the unchanged per-turn restore helper are preserved. **Live continuation acceptance remains unverified, so this PR stays draft.** Two disposable OpenCode/Daytona attempts never reached turns two and three: - At `72b49208b1`, the first turn executed tools and produced the exact 33-byte workspace note, but exceeded its four-minute settlement deadline. The first completion lacked a registered attachment; the second decorated a typed attachment reference with filename/size/hash text. The strict UUID parser rejected that reference before checking a valid criterion reference. The scoped attachment, asset and publication receipt exist. Replaying the actual guard rejects the original input before a database lookup; correcting only that reference passes against the recovered row and receipt. This did not show a publication-persistence or controller defect. The task remained unsettled. - At `1f6589233e`, a separately approved replacement used a temporary prompt-only fixture change requiring exact `deliverable:<attachmentId>` references, with metadata in `verification.detail`. Production code, grading and deadlines were unchanged; the fixture patch was removed afterward. This attempt had no recorded provider authentication, harness-ready, model, tool or result event. Ingress acquisition took 84.072 seconds; transport connection took 205.835 seconds within the 239.832-second failed native execution. A local settlement GET also timed out after 30 seconds. The underlying startup stall is unresolved; these observations do not establish a remote provider outage. Final recovery was blocked with `ambiguous_remote_recovery_evidence`; no successful settlement or continuation is claimed. Both attempts used a Darwin arm64 source host with Node 26.4.0, isolated PostgreSQL 18, a Linux x64 runner and OpenCode 1.18.34. The representative qualified model was `openrouter/deepseek/deepseek-v4-flash-0731`; the original incident used `openrouter/deepseek/deepseek-v4.1-flash-1017`. The controller defect occurs before provider work. Runtime artifacts came from immutable build `e7ea2e2961`, whose Runner package source is identical to the candidate. The current-source manifest, artifact, distribution, asset and all 110 archive-hash checks passed. Each attempt had $10 company/agent model budgets and zero automatic workflow/task retries. Accounting was incomplete; zero recorded spend is not proof of zero cost. Earlier setup failures are also retained: host PostgreSQL bootstrap denial, cached-pack compatibility admission before provider work, and an [official artifact build](https://github.com/paperclipai/paperclip/actions/runs/40696838122) that stopped during an unrelated Pi npm install at its 300-second bound. No manifest or integrity guard was weakened. Owned local processes and databases are stopped, and the temporary fixture patch is removed. Standard fixture cleanup had left one replacement sandbox started; it was stopped and deleted through the SDK. Every owned sandbox was then verified absent with a final 404 readback. Resource cleanup does not establish successful task finalization. An earlier head retained one [voice-recovery timing failure](https://github.com/paperclipai/paperclip/actions/runs/40695892339/job/122843005098): a deadline test observed two failed-provider inspections instead of one. A single isolated shard rerun passed all 1,579 tests without a source change. Its timing explanation remains an inference. The current reconciled head passed without a retry. ## Risks Restoration writes a local projection before the existing admission checks. The encrypted checkpoint loader validates company/task/run and normalized session before writing. A later ordinary continuation refreshes a prior-run projection so rejected admission cannot freeze an active owner's older checkpoint. Active, foreign or unsuspended prior owners still cannot start a successor. Missing or malformed evidence remains blocked. The change adds no retry, credential, provider binary or lifecycle-policy change; it preserves the existing legacy fallback. This PR does not deploy the repair or recover the original hosted tasks. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. The session exposes GPT-6 as its model version; a more specific model ID and context-window size are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d8bd8fe9fb |
feat(evals): browse E2E suites across targets and models (#18027)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - E2E tests show whether people can complete real app and agent journeys. > - Operators need to choose a suite and inspect its evidence across deployments and models. > - The history page listed campaigns without a suite browser or clear coverage filters. > - QA smoke also appeared separate from the other retained E2E measurements. > - This pull request puts suites, deployment targets, and named selections in one history browser. > - The benefit is that a passing subset stays distinct from a full suite run, with direct links to evidence. ## Linked Issues or Issue Description Related: #18009 added clear per-test details and links to the tested application. This PR extends the shared E2E history browser. **What happened?** The public directory called full app testing “Product E2E Evals” and made users choose campaigns before they could find a suite. The history list could not filter independently by tested app, agent execution, model, or run selection. A recent passing subset could be mistaken for a full suite result. **Expected behavior** Browse current and retained suites in one E2E category. Keep QA smoke in normal campaign history. Show the latest matching run and latest full suite run. Preserve the recorded definition and denominator when filters change. **Steps to reproduce** 1. Open the E2E history directory. 2. Find a suite, then compare a narrow passing run with a full run across execution environments and models. 3. Find the QA smoke measurement and open its retained screenshots. **Paperclip version or commit** The previous report UI was at `8279c99008`. This branch is rebased on current master. **Deployment mode** Static E2E history and campaign reports generated from retained measurements. ## What Changed - Call full-product testing “E2E tests.” Keep Runner protocol evals separate because they use a mock control plane. - Add a searchable suite sidebar that includes current suites and suites found only in retained history. - Add correlated suite, coverage, deployment target, execution, provider, model, profile, runtime, result, date, and named-selection filters. Keep filters in the URL for reload and back navigation. Sidebar suite links clear other filters; the Suite dropdown keeps deliberate intersections. Incompatible suite/preset URLs explain the empty selection and link to the suite runs. - Show latest matching and latest full run links. Distinguish full, subset, unfinished, and unknown coverage from the recorded suite definition and execution counts. Keep failed full runs visible. - Retain the tested application URL and runtime mode in new history records. Older records show “Not recorded.” Browser-only journeys do not gain an inferred agent or model. - Add an index-only refresh command that can import a reviewed public campaign once. It rejects private Cloud bundles and does not replace retained campaigns or upload evidence. - Keep existing report URLs, test IDs, commands, and measurement timestamps stable. Campaign links carry suite/profile/environment; target/model/provider/runtime remain history-level filters. - Add unit and synthetic browser regressions, including mobile navigation, long source refs, sidebar filter reset, and empty-selection repair. Inset dropdown carets in the shared report styles. Document direct Node entry points without adding package.json shortcuts. - Add a prominent Reset all filters button, blue active-filter borders and tint, and visible active counts for the main and advanced filters. Reset clears all 13 history filters, sidebar search, URL state, and the attention view. Campaign-page filters use the same active styling. ## Verification - Post-rebase focused Vitest: 127 tests passed across 10 files. The latest follow-up selection passed 100 tests across 9 files. E2E support TypeScript passed after both changes. - The synthetic Chromium journey covers full and subset coverage, unfinished selections, correlated filters, browser-only QA selection, URL/back/reload, and desktop and touch navigation. The expanded journey verifies 13 active history fields, the collapsed advanced-filter count, reset of every field and sidebar search, and active border color changes that revert after reset. It makes zero provider calls. - Hub Python tests: 7 passed. The browser harness also exits promptly when Chromium is unavailable; its HTTP server closes on launch failure. - Real history verification covered 419 campaigns and 59 suites. All 418 previous campaign records remained deeply equal after importing the reviewed QA campaign. Publication used the authoritative history ETag. Only mutable index/history/pointer files were written; immutable campaign bundles stayed unchanged. - A live browser walkthrough passed from the two-family eval hub through QA smoke, Basic app journeys, target filtering, the retained report, screenshot gallery, history, and the logo link. The QA URL was clickable and the image loaded. No browser JavaScript errors occurred. The original QA normalized results and all three PNG hashes remained unchanged. - Live [E2E history](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history), [core compatibility suite](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history&suite=core-compatibility), and [QA smoke runs](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/?view=history&preset=qa-smoke). - The live follow-up verified the incompatible Stock harness/QA-smoke URL, the repair link to 21 Stock harness campaigns, and sidebar navigation that clears preset/model filters. Dropdown carets were inspected in dark, light, and mobile views. No JavaScript errors occurred. - The final live follow-up verified reset of history controls, URL state, sidebar search, and the attention view, plus reload/back restoration and mobile touch reset. Active filter borders and tint were inspected in dark and light views. Campaign-page reset restored all three QA cases. All 419 history records and original QA measurement/evidence hashes stayed unchanged. - Full repository typecheck, test, build, and E2E CI gates passed on final head `1334cb79a5`. Both enforced aggregates, `ci / verify` and `ci / e2e`, completed successfully. Fresh Greptile review scored that exact head 5/5 with no unresolved review threads. ## Risks - Historical metadata can be missing. Unknown targets and runtime details stay explicit instead of being inferred from a worker environment. - Full suite coverage follows the saved definition and selection. Filtering to one model does not redefine that denominator. The latest full-run shortcut can cross coverage selections and says so in its detail text. - This changes shared report presentation. It adds no application runtime or database changes. The browser harness and index refresh do not invoke models. - Public history publication must keep its existing conditional-write and retained-evidence boundaries. Private bundles, raw logs, credentials, and screenshots are not part of this PR. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
b25f3e2af8 |
fix(onboarding): show native proposal confirmation recipes (#18025)
## Thinking Path > - Paperclip lets people manage agent work through tasks. > - First-task onboarding asks a new agent to propose work before it starts. > - The user must accept a saved proposal before the agent can create its child task. > - A native agent can ask for approval in a question card, which does not record proposal acceptance. > - The skill names confirmation cards but does not show their native tool arguments. > - This change adds executable native proposal recipes and checks their real approval path. > - The task admission rules and Product E2E assertions stay in place. ## Linked Issues or Issue Description **What happened?** A fresh onboarding agent asked the user to approve a child task through an Approve/Revise question. The question looked like a decision, but its answer cannot authorize child creation. The first-task skill teaches native question calls for interviews and names proposal confirmation cards without showing the native mapping. **Expected behavior** The agent should create a confirmation card for a single-task proposal, or a checkbox card bound to the saved plan revision. Only the saved human acceptance should admit the child task. **Steps to reproduce** 1. Start first-task onboarding with a native agent. 2. Choose one task and ask the agent to propose a child before it starts work. 3. Have the agent use a question card with Approve/Revise options. 4. Answer Approve. Child creation remains blocked because no proposal confirmation was accepted. Related public context: #17940, #13517. ## What Changed - Show the native `request_human_input` arguments for confirmation and plan-bound checkbox proposals. - Explain that question answers supply scope and cannot grant proposal authority. - Resume the agent after either acceptance or rejection, and use new proposal keys for revisions. - Parse the shipped recipes and execute them through the real native tool authority and interaction service. - Verify that pending proposals, agent-only acceptance, and an answered Approve/Revise question cannot admit child work. - Use the existing isolated PostgreSQL test helper when a local test database is supplied. - Fix the required clean Grok npm-install check to prefix synthetic revision identifiers with `g`. The former all-numeric, leading-zero identifier was invalid SemVer and caused a reproduced `ETARGET` failure. Customer release versions are unchanged. ## Verification - Red: both new executable recipe checks fail against the preceding skill. - Green: 44 focused tests pass with zero skips against a task-owned PostgreSQL server. The actual tool authority and approval service are exercised. - Human acceptance of either recipe admits the child. Agent-only acceptance does not. Answering the question leaves the issue set unchanged. - Red/green real HTTP checks prove rejection saves the assignee continuation, starts no extra model run, and still denies child creation. - `git diff --check` passes. - Full repository `pnpm -r typecheck` and `pnpm build` passed on the final approval-guidance revision. - The first full local test attempt used an unmigrated external fixture and was stopped after the cause was confirmed. A new migrated fixture resolved that setup error; the second full attempt reached its explicit 10-minute limit before finishing. Both attempts are preserved and are not claimed as passing. The temporary owned PostgreSQL container was removed. The complete Linux CI workflow passed on the final PR head. - The exact synthetic-version failure was reproduced with real npm and an empty local packed registry. The new maintained regression keeps the invalid-version negative control and installs corrected numeric-hash and normal-hash controls. All 27 existing Grok helper tests and three offline registry-boundary checks pass. - Final head `52d0b92a00dcb0653b6fb48415cfaea979b8f21f`: [CI 40706933035, attempt 1](https://github.com/paperclipai/paperclip/actions/runs/40706933035) passed; 55 successful checks, two skipped and one neutral. Greptile is 5/5 for this exact head with no unresolved threads. The original failed build remains preserved. - Live fresh-workspace acceptance has not passed yet. It follows the reviewed testing rollout; this source test does not prove model compliance. ## Risks The change guides model tool selection. An agent can still ignore the skill. Product admission remains the authority, and the live journey must still verify human approval, native child execution, downloaded artifact bytes, context reuse, and cleanup. There is no schema or API change. The CI helper changes only temporary verifier versions and preserves the existing registry integrity, install, lifecycle, and isolation checks. ## Model Used OpenAI Codex `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
8049fbf51d |
fix(server): schedule voice call reports from saved deadlines (#17586)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain reports and project guest transcripts into intake tasks. > - Chat maintenance repeatedly scans recent calls even when no report is due. > - Empty scans keep the tenant database active. > - Existing session and report rows already contain the work and its deadlines. > - This pull request moves report refresh to the shared coordinator and removes the final voice maintenance lane. > - Idle voice configuration and old call history no longer cause report scans. ## Linked Issues or Issue Description Refs #17583, #17580, #17578, #17576. This is slice 25 of the recurring-server sleep stack. It now follows merged #17583 and targets master. **What existing behavior does this improve?** Voice call-report refresh and guest transcript projection. **Current behavior** The chat interval scans terminal calls for reports. It repeats the scan even when there are no eligible calls or all saved deadlines are in the future. **Proposed behavior** Committed session transitions and signed callbacks wake a scoped worker. Startup reconstructs existing rows. Later refreshes use saved deadlines. The worker becomes quiet after all eligible windows expire. **Reason and benefit** Stop empty database scans without losing late transcript and billing updates. Keep the existing 24-hour refresh policy, provider response ordering, and guest-task trust restrictions. ## What Changed - Add a voice-owned report worker using the existing scoped deadline helper. No queue table or field is added; the existing session table receives a report-window index. - Save each attempt and its retry before provider I/O. Compare that ownership on settlement so an older response cannot overwrite a later callback or claim. - Preserve available reports when a response is incomplete or stale. Preserve their finite refresh window. - Route voice-change and reply call-ended writes through transaction-aware report signals. - Preserve guest transcript locks and use the observed document revision when updating an existing transcript. - Register the worker in app startup. Remove voice reconciliation from the chat interval. Keep explicit one-shot diagnostics. - Document producer scope, lifecycle behavior, tests, and external wake limits. ## Verification Published head: `758cfe15c3e6283d38ba512c8416ccce4d200862`. The follow-up after `c28de398` changes only two documentation files; the verified production, migration and test source is unchanged. It normally merges landed retention/master `980775e0e956eecdc28c73c9b4aca0407ad45e1a`. The 20 owned paths remain the complete slice, including officially regenerated migration 0397, its snapshot and required pruning of snapshot 0392. The non-database reviewed patch was preserved during ancestry integration; the only subsequent source edit makes one existing report test’s provider responses call-specific. All production report behavior and unrelated master changes are preserved. No shared worker, workflow or lockfile change belongs to reports. All 58 current-head checks are now clean, including the previously failing approval shard; exact-head Greptile is 5/5 with no new actionable issues. Ponytail’s current turn stopped without a verdict, an authorized nonblocking state. The prior conditional documentation requirement was fixed; old CI failures remain recorded below. - **Prior-head evidence at `9bf20467df`: 303 distinct targeted cases passed**, not rerun wholesale for this integration: all 132 voice integration cases; 170 cases across 11 complete report, retention, cleanup, recovery, company-removal, shared-worker and startup/hostname suites; and one complete snapshot/schema drift case. This includes 15 report, 13 retention and 14 cleanup lifecycle cases. Voice integration and the four other database owners use separate initially empty databases and migrate themselves. Report cases use their dedicated database after the setup repair. - Five real connection saves over 205 sessions with saved future report deadlines selected 1,025 session rows before the notification repair. They now select zero and preserve the original deadline. The report notification helper accepts only exact session scopes. Configuration, authority and connection hints retain their existing live recovery/reply/cleanup behavior but do not rebuild report history. Real session terminal writes, callback hints, report claims and receipts still reactivate reports. - Database/server typechecks and builds, migration numbering/safety, and module-boundary checks pass. Final server types/build also pass after integrating the retention review repairs. The parent 205-admission, five-configuration-save test still observes zero retained-history rows and preserves its deadline. - The old eager reader selected all 105 saved-report headers before yielding and failed the 100-row budget. Registered reconstruction now uses the existing paged contract and yields at 100 rows per scope refresh. A partial scan can restart on the same owner and reconstruct every saved deadline. At due time all 105 report IDs are fetched exactly once. Future deadlines remain quiet between actual retries. - The company-fairness case places a small company deterministically on page two and requires it in the next 25-work batch after discovery, with all 105 reports processed before the failed provider's retry. Guarded mutations catch a missing cursor reset and bypassed company rotation. Original source was restored before final acceptance. - At prior head `9bf20467df`, migration 0396 was generated with `pnpm db:generate` on the parent 0395 lineage. It creates the partial `(ended_at, id)` report-window index concurrently and replayably through the existing migration runner. All 394 historical SQL files and the journal prefix are unchanged; snapshots 0392 through 0396 remain. Fresh application migration and two replay invocations succeed with a valid index. The real plan case retains 20,001 old sessions, 205 recent sessions, precision-safe keysets, and fewer than 1,000 rows removed by filtering on each page. - Existing real hangup/callback/reply-error producers, callback ownership fencing, monotonic report results, saved retry before provider contact, guest transcript projection after a document lock, transaction rollback/nesting, drain and shutdown remain covered. No table, field or provider protocol is added. Available reports retain the existing finite 24-hour refresh policy. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/9bf20467dfd50b81848e1c8dd7ddc905d206418f/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#current-parent-reconstruction-and-migration-acceptance-october-28) records the failed fresh fixture, eager read, fairness-clock attempts, guards, commands, counts and limits. Older 174/138-case and workspace-wide results belong to their stated heads. No new complete local repository suite, live provider, Cloud wake, host/database suspension or savings claim is made. - [Report configuration-scope repair](https://github.com/paperclipai/paperclip/blob/9bf20467dfd50b81848e1c8dd7ddc905d206418f/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#configuration-scope-review-repair-october-28) records the current 303 cases and measured rescan repair. The prior `80207ba6bb` head had Ponytail 5/5; that review is stale after this change. Fresh exact-head Greptile 5/5 and all CI remain required. Actual failing Ponytail findings must be fixed; a stopped or paused turn without a verdict is an authorized nonblocker. Active no-verdict review waits are capped at 30 minutes from the original request/start (25-minute monitor cutoff). Older scores and parent checks are not inferred as current passes. - **Current-master verification: 185 distinct cases pass**: final complete voice integration 132, complete report/cleanup/retention 15+24+13, and snapshot/schema drift 1. Each database owner used its own empty fixture database. The first combined run had 183 passing cases and one report-test failure; it is preserved in the audit. A deliberately earlier-ended competing call reproduces the old one-shot mock consuming the intended response. The corrected test selects responses by provider session ID, verifies independent costs for both calls, then rejects stale target data. The isolated repair and full 132-case suite pass; production behavior and timeouts are unchanged. These 185 cases are not added to the older 303 count. - **Current migration lineage:** official `pnpm db:generate` after actual master creates `0397_known_madrox.sql`; the reviewed concurrent/idempotent report-index SQL is retained byte-for-byte. All 395 existing SQL files and journal entries are unchanged; snapshots 0393–0397 remain and the new snapshot links parent 0396. The application migrator boots a fresh database, repairs a deliberately invalid index during two concurrent replay invocations, records one history row, and replays again successfully. Database build, server build/typecheck, migration checks and module boundaries pass. [Full current-master acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/c28de398fef074f650eff532952ab6f5563739a8/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#landed-master-migration-and-acceptance-october-28). - **Review follow-up and preserved CI timeout:** Ponytail’s prior `c28de398` review was 4/5 advisory, with no must-fix production defect and a request to fix the stale design-doc migration reference before shipping. That document now names migration 0397 and links current 185-case acceptance; older 302/303-case evidence is explicitly historical. The optional suggestions about one-shot diagnostic inventory cost and inherited test-clock helpers remain disclosed. CI run `40703490340` failed the unchanged approval-queue backoff test on its existing 15-second timeout and then an activity-log FK in teardown (2,097 pass/one fail/one skip in that shard). The complete unchanged 28-case approval suite passes on a new isolated database; the exact CI timing cause remains unproven. No timeout, workflow or production logic was changed, and no old-head rerun or failure waiver is claimed. This documentation head requires fresh CI and exact-head Greptile. [Review and timeout ledger](https://github.com/paperclipai/paperclip/blob/758cfe15c3e6283d38ba512c8416ccce4d200862/doc/plans/2026-10-25-voice-call-report-sleep-audit.md#review-documentation-and-preserved-remote-timeout-october-28). ## Risks - Process-local commit signals do not observe independent SQL writers or another server process. - Available reports still refresh for 24 hours. The provider's complete flag does not promise immutable final data. - Transcript projection retries remain within the existing report window. This change does not add indefinite projection repair. - One-shot diagnostic `reconcile(limit)` inventories the eligible window before slicing; it remains a manual diagnostic cost, not recurring idle work. Some inherited test helpers advance fake time during assertion retries, an optional reliability follow-up noted by review. - A local deadline cannot wake a stopped host. Cloud alarms and sleep eligibility remain unchanged. No deployed sleep or cost saving is claimed. - This PR depends on the preceding stack. Final master reconciliation and remote review gates remain required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code editing, and local test execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8fe5d5defc |
fix(evals): honor the shared feedback read deadline (#18026)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E evaluations must retain feedback from each paid attempt. > - The collector reads saved feedback and native tool events before teardown. > - A five-second request limit can stop a valid read before the shared read deadline. > - This pull request gives event reads the remaining shared read budget. > - The total deadline and reserved checkpoint time stay the same. ## Linked Issues or Issue Description **What happened?** A paid Ask attempt passed the product grade but retained partial feedback invocation coverage. The collector recorded `events_read_failed`. Its event endpoint later returned HTTP 200. The retained timing is consistent with a request that exceeded the five-second request limit. This change fixes that deterministic failure path; it does not prove the cause of every historical partial read. **Expected behavior** A valid read that takes eight seconds can complete within the 30-second collection deadline. A stalled read must still time out and leave time to save the available feedback. No read or paid attempt is retried. **Steps to reproduce** 1. Return a terminal native run and an empty saved-feedback list. 2. Make the event read take eight seconds and respect the supplied request timeout. 3. Run `collectAttemptFeedback` with a 30-second deadline. 4. Before this fix, the read times out after five seconds and invocation coverage stays partial. Related accounting and reporting work: Refs #18010. ## What Changed - Use the remaining shared read budget for event API requests. Keep the five-second cap for other reads. - Add regressions for an eight-second event response, a stalled event response that preserves saved feedback, and a stalled early feedback read that still permits the event scan. - Document the read budget, reserved persistence time, and no-retry behavior. ## Verification - Red: the initial two regression tests fail before the fix. The other 42 feedback tests pass. The early-read starvation review regression then fails against the first implementation (44 pass), and passes after narrowing the change. - Green: 58 tests pass across feedback collection, interrupted recovery, process shutdown, and scheduled finalization. - Full Product support run: 4,609 pass, three skip, and the retained embedded PostgreSQL bootstrap test fails on this host. The log remains available. This is not a full local pass. - Local Product typecheck fails on installed workspace export mismatches outside the changed files. Hosted CI must check the exact commit with clean dependencies. No local full repository build pass is claimed. - Original paid grades, partial feedback receipts, and accounting journals remain unchanged. This repair makes zero provider calls. ## Risks - A stalled event read can consume the remaining read budget. Other endpoints keep their five-second cap. The shared deadline still reserves up to five seconds for the final checkpoint. - A slow event page can leave later runs uninspected. Those runs remain partial or unavailable. The change does not treat missing evidence as success. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
25f1070fbd |
fix(runner): preserve checkpoint proof across warm preparation (#18011)
## Thinking Path > - Paperclip coordinates agent work and preserves its execution state. > - Native Runner checkpoints preserve authority across controller changes. > - Warm sessions also need checkpoint writes while their successor prepares. > - External persistence replaces the local state file with a marker. > - Two existing checks treated those valid states as lost ownership or prior activity. > - This PR uses the existing preparation lease and canonical controller state to keep those checks accurate. > - The result is safe warm-session retirement and fresh-session startup on reused sandboxes. ## Linked Issues or Issue Description Related: #17439 and #17320. The open #17401 changes the recovery welcome window and does not cover these paths. **What happened?** A configuration change could fail before the next provider turn with `runner did not durably suspend before checkpoint`. The task already pointed to its preparing successor, but that successor did not yet have a native coordinator. The old completed runner's maintenance checkpoint was rejected. A separate fresh session on a reused sandbox could fail with `resumed_unavailable`. Checkpoint registration had replaced the full local state file with an external-persistence marker. The bootstrap check still required the old full-file format. Process supervision then reported unconfirmed termination before launch. **Expected behavior** Retire the completed warm runner under the same process's live preparation lease. Admit a fresh remote session only when the registered controller proves exact identity, no prior activity and pending bootstrap commands, and the remote session directory can be claimed atomically. **Steps to reproduce** 1. Complete a native warm turn with external checkpoint persistence. 2. Change configuration so instruction preparation must retire that session, then submit a follow-up. 3. Separately, admit a fresh native session on a reused sandbox lease after external checkpoint registration. **Paperclip version or commit** Reproduced in the checkpoint implementation introduced by #17439. Fixed against current master. **Deployment mode** Source checkout tests and a hosted Docker/Daytona incident. No deployment or original-task retry is part of this PR. ## What Changed - Accept completed-run maintenance under the exact successor's live, same-process preparation lease only before a native coordinator exists. - Preserve original controller generation, accepted-result, no-new-work, company and task checks. - Pass the registered controller through the existing connection callback and read its current canonical state for untouched-session proof. - Keep marker-only, identity, activity, backup and atomic remote-directory guards. - Add regressions and update execution semantics and session verification recipes. ## Verification - Warm retirement red: the real Postgres writer rejected the preparing successor's `runner.suspend` checkpoint with `native_runner_checkpoint_owner_lost`. - Warm retirement green: 14 Postgres ownership cases pass, including stale/foreign/expired owners, conflicting native authority and new provider work. - Bootstrap red: real controller registration and external persistence caused the resumed and replacement positives to fail. Six external negative cases passed. - Bootstrap green: all 34 bootstrap cases, the bootstrap-state predicate and 13 connection-driver cases pass (48 total). - `node scripts/check-maps.mjs` and `git diff --check` pass. - Full `pnpm -r typecheck` and `pnpm build` pass. - Full local `pnpm test:run` remains partial/interrupted. The default attempt could not initialize embedded Postgres on this Mac. A dedicated Postgres 18 attempt ran for 17 minutes without a completed file result before interruption; the generic helper separately confirmed unsupported embedded Postgres (`initdb` failed after five attempts) and ignores the external database URL. The generic probe skipped one test. Setup and interrupted-run logs are retained; neither is claimed as a behavioral red or a full local pass. - Real source-hosted Codex/Daytona acceptance was attempted at `6ce7fcacc4` with `gpt-6.1-sol`, `xhigh`. The actual task stopped before provider work at `runner_ingress_auth_failed`, with zero authenticated connections and zero source events. A separate environment-qualification handshake reached challenge/response; that does not prove task authentication. The three-turn warm reuse, configuration retirement, useful output and reload workflow remains unverified, so this PR stays draft. - The live attempt used the cached Linux amd64 Runner binary `7437a4e32a099180889be534fcfb2a086046514490c1448aba46c55848983ed8` from the immutable Daytona image. The current staging binary could not be copied because SSH timed out, and the configured current EXE image returned registry 403. No compatible current local or CI-downloadable artifact was available. Earlier setup failures and cleanup receipts are retained; no provider success is claimed. Disposable test resources were cleaned up. - Exact-head Greptile completed at 5/5 with no open feedback: https://github.com/paperclipai/paperclip/pull/18011#issuecomment-6469515952. All current-head CI checks pass: 53 successful checks and two intended Storybook skips. ## Risks The preparation fallback affects maintenance checkpoints only after an accepted result. An existing native coordinator prevents that fallback. Stale controller generations, expired or foreign preparation leases, wrong tasks and new provider work remain denied. The bootstrap check retains the live controller object and reads current state each time. It does not trust a marker or restore local state beneath a live controller. Pending turns, prior events/connections, backups and existing remote directories retain recovery gates. No schema, provider binary, credential, permission or lifecycle-policy change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. This session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.10 |
||
|
DottaandPaperclip
|
c08da3e0a7 |
perf(server): narrow skill list reconciliation reads (#18023)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The skill library lists current company skills and their agent attachments. > - Each list request also reconciles bundled skills, folders, and local files. > - These reads return full agent records and large metadata that their callers do not use. > - Conditional requests do this work before they return an unchanged response. > - This pull request selects the fields each caller needs and avoids unused folder lookups. > - The result is less database transfer and fewer round trips while reconciliation stays current. ## Linked Issues or Issue Description Refs #17941. That change reuses bundled categories within an import. This change removes other repeated work in the same list path. **What happened?** Warm skill-list requests still made 119 SQL calls in a local HTTP fixture. The folder pass returned full skill metadata. The local-source pass also returned remote skills. Attachment counts loaded full agent configurations and monthly spend. Bundled reconciliation resolved folder paths that its release check did not use. **Expected behavior** The list must preserve its current result, attachment counts, access rules, local-file reconciliation, and bundled release checks. Internal reads should return only the data their callers need. **Steps to reproduce** 1. Use a company with many remote skills and agents with large saved configurations. 2. Read its skill list, then repeat the read with its ETag. 3. Measure the queries and response values before the server returns `200` or `304`. 4. Run `pnpm --filter @paperclipai/server exec tsx scripts/benchmark-skill-list.ts --samples=5 --network-delay-ms=3.5` for a synthetic reproduction. **Paperclip version or commit** Baseline service: `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. The branch is based on current master. The paired mode changes only the skill-service module; both versions use the same imported helpers. **Deployment mode** Built from source. Local HTTP server and disposable PostgreSQL 17. No hosted deployment is part of this PR's local proof. ## What Changed - Read only the desired-skill selections of nonterminated company agents for attachment counts. Resolve each agent's references once against the full visible skill list. Preserve IDs, keys, aliases, duplicate handling, and ambiguous slugs. - Read only `sourceKind` from metadata for bundled folder classification. - Limit local-source reconciliation to local-path rows. - Skip folder-path enrichment only for bundled reconciliation. Normal imports and list responses keep their folder paths. - Add real-database byte regressions and a local HTTP benchmark. The benchmark can alternate the old and new services against one database, actor, and process. It checks complete response equality and conditional responses. The company-skill service owns these reads. The query predicates and freshness checks remain at their existing owners. This change adds no cache, schema change, or new background work. ## Verification - The attachment byte regression failed before the fix: 601,418 captured response-value bytes exceeded its 2,000-byte bound. The same case passed after the fix. It checks fresh configuration edits, IDs and keys, duplicate aliases, ambiguous slugs, paused agents, terminated agents, and company scope. - 111 focused service and authorization tests passed against isolated real PostgreSQL 17 databases. They include full result equality, retained stored Markdown and metadata, local-file updates, release checks, project visibility, and import authorization. - The default embedded PostgreSQL probe skipped the first local attempt on this Mac. The completed run used a local-only bootstrap adapter for disposable databases. Queries, migrations, services, authorization, and filesystem work used their real implementations. The adapter is not part of this PR. - In repeated synthetic HTTP byte runs, calls fell from 119 to 98 per request. Result-value bytes fell from about 7.75 MB to 1.64 MB per request, about 79%. These values are measured before Drizzle decoding. They exclude protocol framing and do not measure provider billing. - A separate-process run without the byte observer did not show a latency gain. The paired experiment removes that comparison gap: it alternates both services against the same database and checks identical complete responses. On final commit `3076fe1402085f7286ff58e642d7caca892a5f80`, five observer-free pairs had median times of 2.582 s before and 1.850 s after (28.3% lower). The candidate was faster in all five pairs. Its range was 1.802–2.199 s; the baseline range was 2.327–2.725 s. Every normal response had the same complete body hash, and both versions returned the expected unchanged conditional responses. These are local service/HTTP timings with synthetic data and 3.5 ms delay per network direction. They are not hosted acceptance. - Final-head workspace typecheck and build passed. The full local test command did not complete because embedded PostgreSQL bootstrap failed on this Mac; I stopped that owned runner after 16 native-runtime cases failed. All 16 cases then passed when rerun alone against disposable PostgreSQL 17 using their existing external-database path. The 111 focused PostgreSQL 17 tests above also completed successfully. This does not constitute a completed full local suite; all current-head remote CI gates passed (55 successful checks and 2 skipped). Module-boundary and whitespace checks pass. An initial nested package command used the global pnpm; subsequent commands were pinned to pnpm 9.15.4. - The first canary dry run failed while installing an unrelated standalone package: npm returned 404 for the newly published Rollup 4.66.0 tarball. The exact download subsequently returned HTTP 200. The one retry passed without code changes. - Benchmark commands: add `--capture-values=false` for observer-free timings. Add `--baseline-ref=e1ba77f06771a27114b3999004b0ca5e3c172dd4 --capture-values=false` for paired service acceptance. The normal mode uses the production router. The paired mode uses equivalent default-list HTTP handlers and a synthetic authenticated board actor; it does not test login. ## Risks The SQL selection must stay in sync if the desired-skill format or folder classifier needs new fields. Existing parsing and reference resolution still interpret the selected data. The synthetic fixture has large unused payloads. It proves the mechanism and preserves required content. Its percentage is not a projection of whole-instance transfer or billed savings. Hosted latency still needs verification after merge and deployment. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, shell execution, real PostgreSQL, and local HTTP testing. The host does not expose a precise backend revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jamie Buchanan
|
3339582810 |
fix(adapter-utils): stop asking the GitHub broker on every local git call (#17886)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs use `git` and `gh` through the managed GitHub launcher. > - The launcher asks the broker for credentials before every `git` and `gh` call. It also asks for local commands that do not use the network, such as `git status`. > - Each request runs a transaction and an UPDATE in the database. On a busy install this gives hundreds of thousands of writes each day and long COMMIT waits. > - This pull request stops the request for local `git` commands and writes the identity summary only when it changes. > - The benefit is fewer broker requests and fewer database writes. The credential rules do not change. ## Linked Issues or Issue Description - Fixes #17879 ## What Changed - The managed `git` launcher does not ask the broker for local subcommands (`status`, `diff`, `log`, `rev-parse`, `add`, `config`, `branch`, `checkout` and others). It reads past global options such as `-C` and `-c`. An unknown subcommand or option still asks. `commit` and `var` still ask, because the managed author comes from the broker answer. - The launcher keeps no broker answer between invocations, so no GitHub credential is written to a file. The suggestion in the issue to reuse one answer per run is not part of this pull request, because the reused answer would have to rest on disk. - `resolveGitHubOperationCredentials` writes `run_identity_contexts.github` only when the summary changes. - `doc/execution-github-identity.md` describes the behavior. ## Verification - New launcher tests run real `git` against a fake broker that counts POSTs. A 12-command sequence (init, status, config, rev-parse, add, diff, commit, log, branch, show, status, diff) made 12 POSTs before and 1 POST after. The one POST is `commit`. - The same tests check that `var`, `remote`, `fetch`, an alias, and an unknown option still ask, and that `git var GIT_AUTHOR_IDENT` returns the managed author. - A new test covers the unchanged-summary write. - `vitest run` on the launcher, launcher environment, callback bridge, github-operation-credentials, run-identity, heartbeat-github-launchers, native-github-access, openapi-routes, run-trust-preset, http-log-redaction, skill-source-github-access and chat-channels tests: 12 files, 1901 passed, 1 skipped. - `vitest run --project @paperclipai/adapter-utils` on `src`: 1903 passed, 12 skipped, 1 failed. The failure is `sandbox-callback-bridge.test.ts`, a file this PR does not change. It passes alone (89 of 89, three runs). - `tsc --noEmit` passes in `packages/adapter-utils` and in `server`. - Three deliberate breakages each made a test fail: `var` back in the local list, the broker always asked, and the summary write skip removed. - Not run: the full server suite and `pnpm build` (the local machine has no `cargo` for the runner binary). ## Risks - In a partial clone or an LFS repository, a local command that must fetch a missing object runs without a token. - A `git` command that is not local (`commit`, `fetch`, `push`) and every `gh` command still asks the broker once. - No migration. No credential is stored at rest. ## Model Used - Claude Sonnet 5.5 (`claude-sonnet-5-5`, Anthropic) through Claude Code, with tool use (shell, file edits, test runs), wrote the code and tests. - Claude Opus 5.5 (`claude-opus-5-5`, Anthropic) through Claude Code wrote the issue text and this description. - A person asked for the change and submits it. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
980775e0e9 |
fix(server): schedule inbound voice history retention by deadline (#17583)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls leave history after provider cleanup is confirmed. > - The chat maintenance tick repeatedly deletes old admission history. > - Each eligible row already contains the dates needed for its retention deadline. > - This pull request schedules that deadline and wakes on committed changes. > - Idle retention stops querying while the existing seven-day policy remains enforced. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound voice history retention. This PR follows merged #17580, which owns physical call cleanup, and now targets master. Call reports remain a separate migration. **Current behavior** Voice maintenance scans admission history repeatedly. It deletes confirmed, terminal, unbound admissions when both expiry and last update are at least seven days old. **Proposed behavior** Load existing retention deadlines at startup. Wake from the existing admission transaction notifications. Delete at the later timestamp plus seven days. Lock and recheck current eligibility before deletion. **Reason and benefit** Configured idle endpoints and future history do not cause recurring retention SQL. Failed deletion retains a retry. Restart reconstructs the same saved deadlines. ## What Changed - Add a voice-owned retention worker using the existing scoped deadline helper and coordinator. Yield after 100 history rows per scope refresh and reset reconstruction on registration. Keep complete one-shot inventory reads. No table, field, or migration is added. - Extend `notifyVoiceInbound` to notify retention before the same committed writes that already notify cleanup. Callers need no new recovery wrapper. - Preserve confirmation, terminal-state, session, task, and seven-day eligibility conditions. Round PostgreSQL submillisecond deadlines up. - Lock and recheck the current row before deletion. New bindings or moved dates invalidate stale candidates. - Remove history pruning from recurring voice maintenance. Keep the explicit one-shot diagnostic path. Leave call reports for their own slice. - Document producer ownership, retry/drain behavior, tests, and external wake limitations. ## Verification Published head: `53f2fafcb64d104887eb4455c292b48e6dd10253`. It includes the normal merge of actual landed master `71b448b9cb841e6fa18fe883585430c5081245da` after cleanup #17580. That ancestry integration at `768c650365` preserved the complete binary diff of all 13 reviewed retention paths. This follow-up changes only the existing cleanup fairness test and its failure ledger, bringing the final scope to 14 paths; all production source is unchanged from that integration. All unrelated master changes, cleanup repair, migration `0396_tan_paladin.sql`, and gateway clock repair are preserved. Database files, shared worker, workflows and lockfile match actual master. The earlier 282-case/type/build evidence below belongs to unchanged retention source at its stated head; current repair acceptance is 37 distinct complete cleanup and retention lifecycle cases. All 58 current-head checks are now clean, including the Canary Dry Run; Greptile is exact-head 5/5 with no new actionable issues. The current Ponytail turn stopped without a verdict, which is an authorized nonblocking state. The earlier failing CI remains preserved below. - **282 distinct targeted cases pass**: all 127 voice integration cases and all 155 cases in ten complete retention, cleanup, recovery, company-removal, shared-worker, startup and hostname suites. These include 13 retention and 14 cleanup lifecycle cases. Voice integration, parent recovery and company removal start on separate empty databases and migrate themselves. Retention uses its dedicated database after the setup repair; cleanup uses its already-migrated dedicated database. - Ponytail's prior exact-head 4/5 review found configuration-driven history rescans. Five real connection saves selected 1,025 rows over 205 future admissions before repair. Retention now receives only admission hints and startup reconstruction; the same saves select zero admission rows and retain the original deadline. Cleanup still receives its full configuration and authority scopes. Restrictive binding foreign keys prevent implicit unbinding outside the admission-write boundary. - Successful deletion now returns its confirmed result without reloading the deleted row. A real SQL-count case and a guarded negative probe cover that path. Changed or not-yet-due history still reloads current eligibility and its deadline. - Server typecheck, build and module-boundary checks pass. No schema, driver, shared scheduler contract or provider protocol changes are added by this slice. - A real 105-row future-history case failed before the reader repair because startup selected 105 rows rather than yielding at 100. The existing paged shared-worker contract now bounds each scope refresh. The same-owner restart case reconstructs every deadline and deletes all 105 eligible rows at the original deadline. It verifies SQL quiet before that deadline and after settlement. Removing only cursor reset fails at five rows instead of 100 after restart; guarded source was restored before final acceptance. - Existing cases retain the later-of-two-timestamps seven-day policy, submillisecond rounding, current-state confirmation and binding checks, concurrent binding lock, rollback/outer commit, first write during drain, failure retry and shutdown. Real signed ingress, cleanup confirmation and maintenance-lane removal remain covered by the complete voice suite. - The fresh retention fixture first failed on missing schema and now applies the application migrator. The initial parent merge duplicated a test import and failed to parse before any voice case ran. Both failures are retained in the audit; the corrected complete suites pass. - Prior head `daca270a99` had a failed server (4/12) check, run 40557790710/job 122386307340. Its actual log reports a runner shutdown and cancellation before completion. This is incomplete acceptance, not a pass or a proven product flake. New-head CI must pass independently. - [Current-parent acceptance and failure ledger](https://github.com/paperclipai/paperclip/blob/68b62b6d13f28ac5f17f35b98adf5488752f9b97/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#current-parent-paging-and-restart-acceptance-october-28) records the exact scope, commands, counts and preserved failures. Earlier 218/133/134-case and full-workspace results belong to their stated heads. No new full local repository suite, live provider, Cloud wake, host/database suspension or savings claim is made. - [Notification review repair and current 282-case evidence](https://github.com/paperclipai/paperclip/blob/68b62b6d13f28ac5f17f35b98adf5488752f9b97/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#configuration-rescan-review-repair-october-28) records the new source. The prior head had green CI and a 4/5 Ponytail review; neither is accepted for this new commit. Fresh exact-head Greptile 5/5 and all CI remain required. Completed actual failing Ponytail findings must be repaired; a stopped or paused turn without a verdict is not a merge blocker under the user’s authorization. An active Ponytail review waits at most 30 minutes from its original request (25-minute monitor cutoff). - **Preserved current-master CI failures:** run `40697141057` failed server shard 7/12 in cleanup’s fairness test and Canary Dry Run on an npm `rollup-4.66.0.tgz` 404. A fixed final admission UUID reproduces the same fairness assertion: the second company was sometimes beyond the first 100-row page, so expecting it among the first 25 calls was invalid. The corrected test controls time and proves service within the next 25-call batch after discovery, all 105 successful calls before the failed provider’s saved retry, exactly one retry at its deadline, and idle SQL after settlement. Bypassing company rotation fails the corrected assertion; the shared helper was restored byte-for-byte. Both complete cleanup (24) and retention (13) suites pass after restoration. An intermediate 23/24 run incorrectly demanded idle before reconstruction finished; the audit preserves that failure and its corrected pending-intent wait. No production behavior, timeouts, dependencies or workflow gates changed. The exact npm metadata and tarball now return 200; new-head canary CI must pass independently. Prior Greptile/Ponytail 5/5 on `768c650365` are stale for this new head. [Full CI repair ledger](https://github.com/paperclipai/paperclip/blob/53f2fafcb64d104887eb4455c292b48e6dd10253/doc/plans/2026-10-25-voice-inbound-retention-sleep-audit.md#current-master-ci-repair-october-28). ## Risks - Supported admission writers must use the existing transaction notification boundary. Independent SQL writers and additional processes need a separate wake contract. - Retention never deletes unconfirmed cleanup or session/task-bound history. Those records remain with their owners. - The in-process deadline does not wake stopped compute. This PR does not extend Cloud alarms or relax sleep eligibility, and does not claim deployed host/database suspension. - Call-report reconciliation and other chat maintenance still need separate migrations. ## Model Used OpenAI Codex with reasoning, shell, and code-editing tools. The exact model version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green for this new head - [x] Greptile is 5/5 on this new head with no unresolved actionable findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
71b448b9cb |
fix(server): schedule inbound voice cleanup from durable obligations (#17580)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls need approval, expiry, and safe provider cleanup. > - The remaining voice maintenance tick scans admissions every second, including idle endpoints. > - Admission status does not prove that the provider call has ended. > - Existing admission rows can retain the cleanup owner, confirmation, and retry deadline. > - This pull request replaces the cleanup scan with committed changes and actual deadlines. > - Idle cleanup stops querying while unfinished calls keep a recoverable path to termination. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound voice admission cleanup. Its reply-delivery parent #17578 has landed; this PR targets master. Live-session recovery is in #17576. It preserves separate ownership for those services. **Current behavior** The voice maintenance tick scans expired admissions even when no cleanup is due. A session can commit before the separate approved-admission write. Expiry can then terminate that authorized call. A rejected guest call can be labeled ended before provider termination succeeds. Seven-day retention can delete that unfinished cleanup. **Proposed behavior** Wake cleanup from committed admission and session changes. Schedule the original expiry or saved retry. Claim current state under the admission lock before provider I/O. Commit approval with session reservation. Keep unconfirmed termination durable and transfer bound calls to live-session recovery. **Reason and benefit** Configured idle endpoints and confirmed history cause no recurring admission cleanup SQL or provider requests. Approval, retries, crash recovery, and cleanup remain supported. History retention and call reports stay separate migration items. ## What Changed - Recover terminal and rejected legacy admissions whose session binding was lost. Copy known termination or bind a proven matching live session for cleanup without granting approval. - Register inbound cleanup with the shared coordinator and scoped deadline worker. Yield after each 100-row scope page and reset cursors on registration. Remove only cleanup from periodic voice maintenance. Keep complete manual reconciliation. - Add commit-aware notifications at admission, approval, rejection, guest binding, pending-tool expiry, lifecycle confirmation, and session termination writes. - Commit authorized session binding and admission approval together. Recheck cleanup ownership under the same admission lock. Recover older interrupted bindings only from matching session evidence. - Add nullable cleanup confirmation, next-attempt, and provider-account fields to the existing admission table. No new queue table is added. Generated migration 0396 backfills only provable historical termination in committed 1,000-row keyset batches and adds a concurrent partial index. - Save the retry before provider contact. An unconfirmed end receipt retains cleanup. Bound calls remain owned by live-session recovery. - Verify the signed provider account before using a rotated key for unbound cleanup. A different account cannot inspect or end the saved call. Legacy admissions require their original credential fingerprint. - Prevent history pruning and company deletion from erasing unfinished cleanup. Company deletion locks endpoints before the voice owner's check, matching ingress serialization. - Document producer coverage, ownership, deadlines, verification, and remaining sleep limits. ## Verification Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply parent #17578 has landed. A normal merge integrates actual master `e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated changes, and regenerates this slice’s migration after landed migration 0395. Fresh exact-head reviews and CI are required. - The previous Ponytail 3/5 finding is repaired: legacy terminal admissions no longer enter approval-only repair, and rejected admissions with valid matching live sessions retain their rejection and persist the binding needed for session-scoped cleanup confirmation. Five regressions failed with ownership conflicts before the repair. Five conflicting-session regressions prevent an unrelated leg from being claimed. No cleanup operation grants approval or starts a provider call. - **24 complete cleanup lifecycle cases and 122 complete voice-store integration cases pass** on the repair. After actual-master integration and migration regeneration, the complete 24-case cleanup suite passes again on a new empty PostgreSQL 17 database, and the database migration integration test passes separately. The cleanup owner, voice-store test and admission schema are byte-identical to that tested repair; the cleanup fixture only changes the migration filename. Migration numbering/safety, database build, production server typecheck and build pass. - Migration `0396_tan_paladin.sql` was generated with `pnpm db:generate`. Every landed SQL migration and the complete parent journal prefix are unchanged. Generated snapshots retain exactly 0392–0396, pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and committed 1,000-row historical backfill are retained byte for byte under the new generated filename, registered for top-level execution. The real application migrator boots a fresh database, repairs an interrupted concurrent index, serializes replays and verifies eligible-history backfill across actual transactions. - The preceding published head `48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but failed CI server shard 9/12: one existing connected MCP rate-limit test expected two real-clock calls to share a fixed 60-second bucket. A controlled window rollover reproduced its exact failure. That existing fixture now freezes only `Date` around the same-window 429 assertion, verifies acceptance after advancing one window, and restores the real clock in `finally`. All 92 gateway acceptance cases pass. Production rate limiting, timeouts and assertions are unchanged. This adds one existing CI fixture to the slice; the original failed run remains recorded. - The first regression invocation skipped the suite because it used the wrong database variable; that is not a pass. The corrected invocation reproduced five ownership conflicts. The first fixed cleanup run passed 22 and failed two new low-level deletion test calls on a retained-session foreign key; the regressions now check the cleanup deletion guard at its owning boundary, with production deletion ordering unchanged. The final full suite passes all 24. - The first complete voice-store invocation passed 121 and timed out on one transcript case at the unchanged 15-second limit. That case passed in isolation on a fresh database. One subsequent complete invocation, with builds finished, passed all 122 without source, assertion or timeout changes. The underlying timeout cause is unproven; the original failure remains recorded. A new PostgreSQL startup also failed from exhausted host semaphore capacity; fresh databases on the already-owned server were used without removing shared IPC resources. - Prior source `b8b12199b9` passed 266 distinct cases and carried three guarded mutation proofs; those older results are not added to this repair’s case count. Its CI run 40641168596 failed E2E shard 3 when legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub initially rejected a retry while the run was active, then accepted one targeted retry after completion; attempt 2 passed on that same old head. The lock holder and underlying cause remain unproven. The unchanged Node policy checker’s older failure on ignored generated Copilot manifests and passing tracked-input mirror are also retained in the audit; no workflow, checker, vendor manifest or ignore rule was modified. - [Current repair and failure record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28). Current-head CI and Greptile 5/5 gate merge. A completed failing Ponytail verdict must be fixed; missing or interrupted reviews do not block, and active pending review waits are capped at 30 minutes under the maintainer’s explicit instruction. No new full local repository suite, real provider, Cloud alarm, host/database suspension or savings claim is made. ## Risks - The migration distinguishes admission status from physical termination. Legacy ended guest rejections without proof remain cleanup obligations. Missing original credentials can keep those old obligations unresolved. - Future writers must use the existing notification boundary. Independent SQL writers and multiple processes need their own wake contract. - Pending inbound cleanup now blocks company deletion instead of allowing cascade deletion to lose the provider obligation. Existing voice-history deletion restrictions remain. This deliberately includes an active anonymous inbound ring and an old call whose termination cannot be verified. An operator must stop ingress and settle the call before deletion; this slice adds no abandonment override and never treats missing credentials as proof of termination. This is the retained cleanup ownership policy, not a claim that every deletion attempt can complete immediately. - History retention and report processing still run in the remaining maintenance lane. This PR does not extend Cloud wake coverage or claim measured host/database suspension. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.9 |
||
|
DottaandPaperclip
|
6ce0909db4 |
fix(agents): skip runtime probes for unchanged connections (#18016)
## Thinking Path > - Paperclip manages AI agents and their runtime settings. > - Operators save identity, model, connection, and run-policy settings from the agent page. > - PostgreSQL stores connection bindings as JSONB and can return keys in a different order. > - The save route compared JSON strings and treated an unchanged binding as a new connection. > - That false change started a remote runtime and credential probe during ordinary saves. > - This pull request compares binding values and checks real execution selection changes separately. > - Ordinary edits save without a probe. New connections and execution selections retain validation. ## Linked Issues or Issue Description Fixes #14518. Related work: #17760 also addresses prospective environment resolution. #14027 also adds explicit connection removal. This change is limited to the save validation gate and its regressions. **What happened?** A title-only save took 9.8–16.0 seconds in a hosted reproduction. JSONB key order made the unchanged AI connection look different. The route then waited for a remote runtime probe. **Expected behavior** Identity, thinking-effort, and run-policy edits skip the probe when the connection and execution selection stay the same. A changed connection, harness, model, provider, engine, or environment still validates adoption. **Steps to reproduce** 1. Save an agent with a managed AI connection in PostgreSQL. 2. Edit its title or thinking effort without changing the connection or execution selection. 3. Observe that the old route starts the adapter environment test. The fixed route saves without that test. ## What Changed - Use value equality for the stored and validated AI connection bindings. - Compare the effective execution selection separately. Preserve runtime probes for real selection changes and existing Copilot checks. - Add PostgreSQL route regressions for shared, delegated, default-user, and pooled bindings. Cover failed connection, model, engine, environment, and harness changes. - Keep the existing credential rejection and reconnect tests. Document the save behavior. ## Verification - `pnpm -r typecheck` passed on the rebased implementation. - Focused PostgreSQL 17 route selection: 21 passed, 58 excluded by the name filter. This covers ordinary saves, selection-change rejection, and credential health. - Full PostgreSQL 17 connection suite: 79 passed, zero skipped. Final `pnpm build` passed at `8fde75fa1a8fe63c71b05a3db149edf289295672`. - Before the fix, the PostgreSQL regression reached the unintended probe and failed for unchanged shared, delegated, and default-user bindings. The pool control passed. - An early controlled HTTP comparison of the binding fix used a synthetic 500 ms provider probe: five measured saves had a median of 798 ms before and 56 ms after. Probe count fell from five to zero. This is local supporting evidence, not a hosted latency result. - Earlier adjacent-route run: 159 of 160 tests passed. One existing roster-edit case exceeded its 15-second timeout. That case passed alone with the normal timeout. The cause of the earlier timeout remains unproven. - Local full-repository tests were not run. Native PostgreSQL startup fails because the host has exhausted System V semaphores. The focused tests use an isolated PostgreSQL 17 Docker fixture. - `git diff --check` passed. Hosted improvement remains unverified until deployment. - Greptile and Ponytail Paperclip Review both rated the exact head `8fde75fa1a8fe63c71b05a3db149edf289295672` 5/5, with no actionable defects. - CI attempt 1 passed 51 checks. Three self-hosted runners lost communication, interrupting the chat shard, serialized server shard, and canary dry run. GitHub annotations identify the runner communication failures; the serialized shard log also records a shutdown signal. Only those failed jobs were retried, once, on the unchanged commit. The [full CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40692641103/attempts/2) then passed on attempt 2 at the same commit. All 55 passing checks and the legacy Snyk status are green; no checks remain pending. ## Risks This changes when connection adoption validation runs. Separate selection checks preserve validation for real runtime changes. Regression tests verify that rejected selections do not replace the saved configuration. Company access, connection authorization, activity records, and Copilot checks retain their existing paths. There is no schema or migration change. ## Model Used OpenAI Codex, configured model `gpt-6.1-sol` with ultra reasoning. Work used source inspection, shell tools, browser network timing, and PostgreSQL route tests. The context window size is not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
83709f75eb |
fix(ui): hide task row retry status labels (#18017)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task lists and the inbox help people scan their work. > - Shared task rows show system interruption messages beside task titles. > - These messages add detail that the person cannot act on from the row. > - This pull request removes the three system interruption labels from those rows. > - People can still open the task to inspect progress and use Retry. ## Linked Issues or Issue Description Refs #16965, which introduced these labels. Related recovery report: #17386. This change only removes row text and does not resolve that report's retry behavior. **What existing behavior does this improve?** The shared task rows in task lists and the inbox. **Current behavior** Rows show “Retry available,” “Thinking,” or “Trying again soon” beside the title for unresolved system interruptions. **Proposed behavior** Omit all three labels from shared task rows. Keep the existing system-interruption guard so the row does not show a legacy recovery chip in their place. Keep task status, other badges, and retry controls inside the task. **Reason and benefit** People can scan task titles with less incidental execution text. **Breaking changes** The three row labels disappear. APIs, recovery behavior, and task conversation controls stay the same. ## What Changed - Remove the system interruption text from every shared task-row presentation. - Cover working, scheduled-retry, and failed interruptions in both row presentations, including a legacy recovery action that must stay hidden. - Record the expected row behavior and verification states in the task feature map. ## Verification - Passed: token gates, feature-map contract and reference checks, and Git diff whitespace checks. - Passed: 83 component tests in `IssueRow.test.tsx` and `Inbox.test.tsx`, including the new regression cases; full `pnpm -r typecheck`; full `pnpm build`. - The full local `pnpm test:run` was attempted and stopped after 16 database-dependent failures. An independent disposable `initdb` reproduced `FATAL: could not create semaphores: No space left on device`. The host has exhausted its macOS System V semaphore capacity. All required CI test lanes passed on the final commit and provide full database-suite coverage. An earlier attempt also found an unbuilt plugin SDK; building that existing dependency repaired collection before this database failure. - Review the working, scheduled-retry, and failed interruption branches. Each returns no system label. The existing guard still suppresses legacy recovery chips. - Independent source review found no actionable code issues. Regression cases check title navigation, removal of the three labels, and suppression of replacement recovery chips. - All required CI gates passed on commit `dd1d7b08dbba8a10a95c5d24c4aea64479befe95`: typecheck, general and serialized tests, browser suites, build, canary, and Runner checks. - Greptile reviewed that commit at 5/5 with no open findings. The optional Paperclip review also returned 5/5 with no findings; its check status is neutral. ## Risks - Low risk. This removes row text only. Task status, recovery records, other badges, and task-level retry actions keep their current behavior. ## Model Used OpenAI Codex, GPT-6 family. This session does not expose an exact model ID or context-window size. The assistant used reasoning, shell tools, and subagents for source review and validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8279c99008 |
fix(evals): clarify report details and link tested applications (#18009)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E reports help operators inspect real application journeys. > - A worker environment does not identify the application that a journey tested. > - Reports hid that URL and gave missing diagnostics too much space. > - Standalone reports also sent the logo link to a missing parent page. > - This pull request shows the recorded target and checks in the shared report. > - The benefit is a clear report with useful links and inspectable diagnostics. ## Linked Issues or Issue Description **What happened?** A standalone Product E2E report had no application link in test details or the screenshot gallery. Its details showed several empty diagnostic sections before useful information. The logo could resolve to the page host root and return 404. Campaign notes added outside the report shell could overlap the fixed sidebar. **Expected behavior** Show the recorded application URL beside each test and screenshot. Put recorded checks before optional diagnostics. Keep campaign context inside the shared layout. Link the logo to the evaluations directory. **Steps to reproduce** 1. Render a retained campaign at a standalone page path without a history index. 2. Open a test with screenshot evidence and missing startup timing. 3. Inspect test details and gallery information, then click the report logo. **Paperclip version or commit** Reproduced in the report renderer at `39e811c9c5`. This branch applies the fix to current master. **Deployment mode** Static Product E2E HTML report generated from retained normalized results. ## What Changed - Add optional recorded `targetUrl` metadata and resolve existing Product E2E and provider-connection targets. Only HTTP and HTTPS links without embedded credentials are shown. - Show the target URL in test details and screenshot gallery information. - Put recorded checks and execution facts first. Keep raw usage, timing, and startup details under Diagnostics. Preserve missing metadata as unknown. - Keep campaign titles, descriptions, notes, and evidence links inside the shared report layout during retained-evidence regeneration. - Fix standalone navigation and let the brand lockup wrap at narrow sidebar widths. - Match report profiles by suite and recorded provider/model before falling back to an inferred profile. - Add report regressions and document the retained metadata contract. ## Verification - Focused report tests: 86 passed across 8 files on the isolated branch. Command: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts report-target.test.ts report-navigation.test.ts report-catalog.test.ts first-task-report.test.ts report.test.ts dashboard-regenerate.test.ts`. - Browser verification covered dark desktop widths 1440 and 1000, light intermediate width 780, and dark mobile width 390. It covered the drawer, URL links, gallery keyboard navigation, focus return, and the logo destination. - The published retained report keeps all 3 original passing cases and 7 recorded checks. Its campaign and screenshot bytes remain unchanged. This presentation refresh made zero provider calls. - Full `pnpm build`, `pnpm -r typecheck`, and the eval TypeScript check passed after rebuilding the reused package dependencies. The first typecheck attempt had stale local dependency builds; those errors are resolved. - The local `pnpm test:run` attempt hit repeated embedded PostgreSQL bootstrap failures in server suites. I stopped that attempt and retained its log. It is not counted as a passing check. Current-head CI remains the full test gate. - Review follow-ups add provider/model and cross-suite profile regressions. Malformed campaign presentation metadata is rejected before any report or retained-result writes; ten malformed input cases verify that retained results stay unchanged. ## Risks - This changes presentation across Product E2E reports. Historical reports without a target show “Not recorded” rather than an inferred URL. - Unknown startup reuse data remains in raw metadata but no longer creates an empty table. - Standalone logo links open the shared public evaluations directory. Campaign history links still use a matching history index when present. - No application runtime, database, or measurement verdict changes are included. Private QA configuration and raw campaign evidence stay outside this PR. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3bd0d81c7b |
fix(e2e): follow proven workspace deferrals during onboarding (#18008)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify the user outcome through the browser and scoped APIs. > - An onboarding child can wait while its parent owns their shared workspace. > - The server records that no provider work started and schedules a durable replacement. > - The journey rejected that expected wait, and feedback counted it as unknown legacy execution. > - This pull request follows the exact replacement and records the distinct no-provider evidence. > - The child must still finish through native execution and produce the independently checked download. ## Linked Issues or Issue Description Refs #17940. **What happened?** The fresh-workspace journey failed with `onboarding_first_task_failed` when a child encountered shared-workspace contention. The cancelled admission had an explicit `workspace_wait` receipt with `providerWorkStarted: false`. Its linked replacement was still scheduled. The runtime had not been selected. The default `legacy` value did not represent a provider execution. **Expected behavior** Keep both rows and their run-limit cost. Follow only the unique replacement for the same company, agent, and issue. A pending replacement must not pass. Real failures and contradictory execution evidence must still fail. **Steps to reproduce** 1. Start a native onboarding parent and accept its child-task proposal. 2. Observe the child admission while the parent still holds their workspace. 3. Present the server's no-provider cancellation and its durable scheduled replacement to the journey and feedback collectors. ## What Changed - Add a shared predicate for the explicit no-provider workspace-deferral receipt. Reject runtime, process, output, usage, and cost evidence that contradicts it. - Follow exact, unique retry lineage during onboarding. Wait for active or scheduled replacements without repeating browser actions. Preserve the campaign deadline, run cap, native artifact attribution, and download oracle. - Retain per-run feedback inspection classifications: `native_events`, `provider_not_started`, or `incomplete`. Require exhaustive event reads. Keep pending work and uncertain execution partial. - Add causal regressions and document the evidence contract. Historical failed attempts keep their original grades. ## Verification - Red: the new journey regression failed with the observed `onboarding_first_task_failed` before the repair. The new feedback cases also failed before the repair. - Green: 70 journey tests and 74 feedback-family tests passed. The final affected journey/feedback gate passed 275 tests plus 137 Node checks. Five retry-lane reset cases failed before the review correction and passed after it. - `pnpm test:e2e:runner:typecheck` passed after the retry-counter correction. - `pnpm -r typecheck` and `pnpm build` passed before the final rebase. - The full local `pnpm test:run` and harness unit gate encountered embedded PostgreSQL startup failures. The harness gate passed 4,574 tests and failed its database-startup test. The final focused journey/feedback gate and all 137 Node checks passed. CI will provide independent full-suite results before merge. - Current-head CI passed 55 checks; two checks skipped and one was neutral. Greptile is 5/5, with zero unresolved review threads. - Live Cloud verification will use the reviewed harness and Operations revisions. This PR does not claim that a fresh Cloud campaign has passed. ## Risks A broad cancellation exemption could hide real failures. The exemption requires the server's explicit receipt, unresolved runtime selection, no contradictory execution evidence, and exact durable lineage. Missing successors remain pending until the existing deadline. A successful native child and valid downloaded bytes remain required. No server behavior, approval policy, or budget policy changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection, tool use, and code execution. The host does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e7ea2e2961 |
fix(evals): export grades for expanded qualification batches (#18010)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E campaigns retain each attempt and its independent grades. > - Braintrust receives product and cleanup grades for each retained attempt. > - A six-case qualification produces twelve scores. The export allowed only eight. > - Experiment creation succeeded, but grade insertion stopped before dispatch. > - This fix allows fourteen scores so the six- and seven-case batches retain every result. > - Operators can inspect failed and successful grades without running an agent again. ## Linked Issues or Issue Description Refs: #17044 **What happened?** A retained six-case Product E2E campaign created its Braintrust experiment, then failed with `Experiment ingestion allowance exhausted`. Its twelve deterministic scores exceeded the eight-score cap. No insert request was sent. Agent results and model accounting remained saved. **Expected behavior** The bounded qualification batches must export both grades for every selected attempt. Failed grades must remain visible. **Steps to reproduce** Create an experiment through `openExperimentIngestion`. Insert six or seven events with `product_pass` and `cleanup_pass` scores. The old limit rejects the insert before it reaches the transport. **Paperclip version or commit** Reproduced on master `cff331f6ca0a652b5e41641c68dfa5fa584c22ae`. **Deployment mode** Local Docker controller built from source. The failure is in the shared Product export path. ## What Changed - Raise the finite experiment score cap from eight to fourteen. Keep the three-request, 4 MiB and 60-second limits. - Add regressions for the six- and seven-case batches. Keep the failed product grade in each test batch. - Verify that the receipt collector accepts the old eight-score policy and the new fourteen-score policy. It rejects a fifteen-score policy. - Document the seven-attempt export ceiling. This change does not add agent calls or automatic retries. ## Verification - Before the fix: both new batch regressions failed with `Experiment ingestion allowance exhausted`. - After the fix: 67 tests pass across experiment ingestion, ingestion accounting, experiment projection and scheduled finalization. - Command: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/experiment-ingestion.test.ts tests/runner-e2e/ingestion-usage.test.ts tests/runner-e2e/scheduled-finalization.test.ts tests/runner-e2e/braintrust-experiment.test.ts --configLoader runner --cache=false`. - `git diff --check` passes. All 58 current-head checks are acceptable, including typecheck, build and tests. Both reviews are 5/5, with no unresolved threads. - Live agent attempts will not be repeated to verify this reporting fix. Recovery uses the saved results and a separate bounded ingestion receipt. ## Risks - New exports can submit at most six more deterministic scores per experiment. These are saved grades, not model judge calls. - Larger campaigns still fail closed. Existing receipt files keep their original policies and traffic counts. - This fix does not resolve the plan-lifecycle behavior failures observed in the qualification. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e93732c3d4 |
fix(server): deliver voice replies from commits and saved deadlines (#17578)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Phone calls receive approved answers from the existing chat publication queue. > - The chat maintenance timer checks for voice replies every second, including when no reply is ready. > - A reply can be saved before its publication commits. An early wake alone can miss that final transition. > - Existing session, reply, publication, and action rows already record delivery and recovery work. > - This pull request uses committed changes and saved deadlines to drive phone reply delivery. > - Empty reply queues become quiet while authorization, ordering, and uncertain-send protection remain in place. ## Linked Issues or Issue Description Stacked on #17576. Related delivery infrastructure: #15625. Related voice connection work: #11418. **What existing behavior does this improve?** Background phone reply delivery and recovery of abandoned dispatch intents. **Current behavior** Chat maintenance calls voice reply delivery every second. It also scans old dispatch intents. Publication transport starts a direct push before the final publication commit. The timer masks that timing gap. **Proposed behavior** Committed publication, reply, session, and configuration changes wake the shared coordinator. Ready replies drain in bounded passes. Saved rate-limit and dispatch-expiry deadlines retain recovery. An empty queue or a permanent delivery barrier has no recurring reply scan. **Reason and benefit** Remove recurring SQL from idle phone reply delivery without dropping replies, replaying uncertain sends, or adding a queue table. **Breaking changes** No API or schema change. Explicit manual recovery remains available. Inbound admission cleanup, history retention, and reports remain on their current paths for separate migrations. ## What Changed - Extract ordered reply state and multipart policy within the voice owner. Retain current authorization and atomic dispatch claims. - Register a scoped reply worker with startup reconstruction, fair work batches, failure backoff, drain/resume handling and shutdown ownership. - Extend the existing shared dispatcher with normalized partial-page results to preserve the bounded parent recovery contract. Existing complete-array consumers retain their calls. - Signal before relevant writes in their existing transactions. Both normal publication completion and operator resolution wake the actual conversation after commit. - Bound reply inventory to 100 total headers per refresh. Preserve partial scans, concurrent changes and newer denied-call restoration obligations across reconstruction. - Track abandoned dispatch actions independently of live calls. Quarantine expired intents without resending them. - Use existing active-call authorization checks to reactivate replies that were previously denied. Healthy call inspections do not rescan empty reply queues. - Remove the periodic reply scan and the early direct publication push. Keep provider requests outside database transactions. ## Verification Published head: `1435cab10be3f95c38ab6672d2a890ae49e9d300`. Voice recovery #17576 has landed at master `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. A normal merge reconciles squash ancestry while preserving the complete reviewed 31-file reply patch byte for byte and all unrelated master changes. The PR now targets master; fresh exact-head CI and Greptile are required. The prior 449-case/typecheck/build results below cover the identical owned source, not a new full-suite run. - **449 distinct targeted cases pass**: 111 complete voice-store/reply; 19 complete parent recovery; and 319 complete tool-gateway, secret-proposal, shared dispatcher/coordinator/batching/startup/hostname cases. Production server typecheck and build pass. The policy checker and its Node tests pass. Later edits record evidence only. - Two real normal-publication/operator-resolution tests seed 105 unrelated calls and 105 future dispatches. Five publications selected **1,065/1,060 inventory rows before repair**; both paths now pass the same **125-row total budget** and deliver every answer once. This counts selected SQL results, not physical tuple visits or host savings. - Publication commits now signal their actual conversation. The voice reader returns at most 100 total session/action headers per refresh, preserves phase/cursor state and retries, and resets on worker registration. Current-state checks and dispatch claims still own provider effects. The 105-action case measures the page cap, restarts the same owner during reconstruction, waits at saved future deadlines and quarantines every abandoned action without resending. - A real 403 fixture proves that an older broad scan cannot erase a denied call first observed behind its cursor. Its owner callback still emits the committed restoration wake after scan completion. The existing actual-service authority-restoration and final-publication timing tests remain in the full suite. They are not replaced by direct helper calls. - Guarded removal of conversation scoping, cursor reset or the denial revision fence fails the intended budget, 100-versus-5 restart or missing committed-wake assertion; source is restored. An initial shadowed fixture name and an inadequate due-action reset probe are retained as failed attempts in the [current audit](https://github.com/paperclipai/paperclip/blob/d2c0a7de8c3e9cac8382b1268f3f5d883a2067b3/doc/plans/2026-10-25-voice-reply-delivery-sleep-audit.md#publication-inventory-and-paging-acceptance-october-28). - Parent recovery still passes paging, restart, behind-cursor insertion, feature changes, actual disablement, quiet health observations, current authority, exact-scope failure fences and wildcard-failure isolation through the shared worker. Four existing array-based worker consumers retain their API and pass their full suites. - The two voice fixtures self-migrate separate empty databases. Legacy tool-review/secrets suites use separate migrated clones. Earlier full-local attempts remain limited by embedded PostgreSQL setup and are not passing full-matrix claims. Older counts apply only to their stated revisions. Fresh exact-head Greptile, Ponytail if present, security and all CI remain merge gates. No live-provider, multi-writer, external-wake, deployed-sleep or savings claim is made. ## Risks - Missing a supported producer can strand work. Tests cover real enqueue, retrieval, normal publication, and operator publication paths. Startup reconstructs existing records. - Signals cover the process-local database owner. Independent SQL writers or other replicas need their own wake contract. - Authority restoration relies on the existing live-call recovery check. This dependency must be preserved if that owner changes later. - A provider acceptance receipt does not prove speech. Unknown outcomes remain blocked from automatic resend. - This change does not extend Cloud alarms, allow sleep during live calls, or prove deployed suspension or cost savings. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e1ba77f067 |
fix(server): schedule live voice recovery from session changes and deadlines (#17576)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain durable session state and provider identities. > - The chat maintenance timer scans live voice sessions even when no call exists. > - Session writes, saved expiry and active-call inspection deadlines identify real work. > - This pull request moves live-session recovery to the shared coordinator. > - Idle session recovery becomes quiet without losing exact-call cleanup or current authorization checks. ## Linked Issues or Issue Description Refs #17572. This PR is stacked on receiver discovery. It owns only live voice session recovery. Reply delivery, inbound admission cleanup, inbound history retention and call reports remain separate migration slices. **What happened?** The one-second chat maintenance lane reads voice settings and scans live sessions even when an instance has no calls or only terminal history. **What did you expect?** Reconstruct live calls on startup. Wake on committed session or configuration changes. Inspect active calls and retry cleanup at actual deadlines. Stop querying when no live obligation remains. **Steps to reproduce** Start an instance with no live voice sessions. Let startup settle. The old maintenance method continues to read settings and session inventory every chat maintenance pass. ## What Changed - Add a voice-owned bounded deadline reader and scoped worker on the existing coordinator. Retain concurrent writes across pages and reset partial scans on restart. No tables or migrations are added. - Separate generic connection health observations from configuration changes. Existing receiver and command consumers retain their notifications. Voice eligibility and provider operations remain in the voice owner. - Signal existing session transactions and configuration/feature changes. Keep provider calls outside database transactions. Standalone session writes use one small transaction helper. - Preserve saved expiry, the two-minute unknown-creation grace, active-call authorization and missed-hangup inspection, and exact-provider cleanup. Recovery never creates a provider call. - Retry transient authority lookup failures instead of treating them as confirmed revocation. - Fence expiration against a provider identity bound after discovery. A late known call remains eligible for cleanup. - Preserve work through rollback, drain, restart, concurrent refresh, backoff and shutdown. Page inventory and rotate bounded batches across companies. - Leave the four other voice maintenance duties in the periodic lane. - Document producer coverage and the supported writer contract in `doc/idle-voice-session-recovery.md` and the dated audit. ## Verification Published head: `f0b2e3cc22a18e7d08b10d4fc02aae4a19a52f30`. Receiver #17572 has landed in master at `dce004259394486f6361d0c0dd5a33a5272acc0a`. This normal merge reconciles squash ancestry; the complete 22-file voice recovery patch is byte-for-byte identical to the reviewed and tested `ad79c7e805` patch, while all unrelated master changes are preserved. The PR now targets master. Fresh exact-head CI and Greptile are running; interrupted Ponytail turns are not completed verdicts under the maintainer’s clarified merge policy, and an actual failing verdict still blocks. - **210 distinct targeted cases pass on the current repair**: 19 complete recovery lifecycle cases and 191 complete voice-store, coordinator, batching, startup and hostname cases. Production server typecheck and build pass. Later edits record evidence only. - CI `40633667141` on prior head `1b6323c164` failed the feature-change assertion with `34050 < 25000`; its verification aggregate also failed. The test now disables automatically advancing fake time, drives 20 controlled scheduler turns and requires all 205 calls to close before an unrelated inspection deadline can mask a lost wake. Removing generation replay fails with missing closures. The repair does not benchmark database/host speed or raise the elapsed-time bound. - A real committed exact-call regression reproduces wildcard-backoff starvation. The worker keeps the broad retry but processes independent exact hints and cached due calls. Each check reloads current state and authority. Failed exact scopes remain fenced. Removing wildcard isolation fails the intended assertion; a successful broad retry then leaves the worker SQL-quiet. Full-row and lean queries now share the state/scope/cursor predicate. - Earlier acceptance on `712f3862b5` passed 725 distinct targeted cases. Its measurements remain useful history: five real metadata writes and one health check over 205 future calls selected 1,230 rows before the inventory repair and zero afterward, without changing the saved deadline. Those older test counts are not added to the current 210-case count. - Both external voice fixtures self-migrate; the current full voice-store run uses a separate initially empty database. Earlier fixture/method/observer failures, source mutations and partial edit attempts remain in the [current audit](https://github.com/paperclipai/paperclip/blob/ad79c7e805367fc7e2df3ea53c942ca6c7d27365/doc/plans/2026-10-25-voice-maintenance-sleep-audit.md#wildcard-failure-and-deterministic-generation-acceptance-october-28). Source is restored after guarded mutations. - Older CI `40632057899` failed because its trusted policy script was absent from the checkout, while build separately received a runner shutdown signal. E2E shards succeeded but their aggregate failed on policy. The current checkout includes the landed policy tooling and its checker/tests pass locally. The shutdown cause is unproven; neither old failed run is counted as passing. Earlier full-local attempts stopped on embedded PostgreSQL setup failures and did not pass. - Fresh exact-head Greptile, security and all CI must pass before merge. A completed failing Ponytail verdict blocks; an interrupted turn without a verdict does not. No full current local repository matrix, live provider acceptance, multi-writer support, Cloud wake, host suspension or savings is claimed. ## Risks - Process-local hints cover the existing active writer and shared clients. Independent SQL writers or another replica need explicit wake integration. - Membership, task binding, phone preferences and other authority inputs remain checked on the existing 30-second active-call inspection deadline and on use. This is not an instant cleanup guarantee for every authority mutation. - Unknown creation retains its existing maximum-duration grace. Cleanup failures retain retries. A provider request is never replayed merely because its creation response is unknown. - Genuine configuration changes retain bounded reconstruction. Metadata writes can probe parent eligibility, while health-only observations leave voice recovery quiet. Paging state is process-local and reconstructs from durable rows after restart. - Active calls, the other voice maintenance owners and connector transports remain real obligations. This does not expand Cloud alarm coverage, relax sleep eligibility or prove deployed suspension. - The receiver and Discord parents have landed. Current-master compatibility and exact-head CI and Greptile remain merge gates. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dce0042593 |
fix(server): wake chat receiver discovery from configuration and lease deadlines (#17572)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat receivers own persistent provider connections and fenced database leases. > - The chat maintenance timer repeatedly queries all eligible receivers, including when none exist. > - Configuration changes and known lease deadlines already identify when discovery is needed. > - This pull request moves receiver discovery to the shared delivery coordinator. > - Empty or settled discovery stops querying, while ownership and recovery remain explicit. ## Linked Issues or Issue Description Refs #17557. This is the next stacked slice after Discord command registration. The base is `codex/sleep-19-discord-command-registration`. **What happened?** The recurring chat maintenance lane scans receiver inventory and reconciles Discord and Photon runtimes even when configuration has not changed. **Expected behavior** Reconstruct receiver work at startup. React to committed configuration changes and local listener retirement. Retry foreign ownership at the saved lease expiry. Stop discovery when there is no remaining obligation. **Steps to reproduce** Start an instance with no chat endpoints. Let startup finish. Observe the recurring receiver inventory query from chat maintenance. With a configured receiver, remove its ownership token while listener shutdown is delayed and signal a configuration refresh: the old implementation can replace the listener before shutdown completes. ## What Changed - Preserve healthy receiver progress after a slow failed batch while retaining per-receiver retry backoff. - Register a receiver discovery worker with scoped refreshes, paged startup inventory, fair batches, and per-receiver or scope failure backoff. - Keep owned receiver identities cached without a discovery deadline, so deletion still stops their listeners. - Use existing transaction signals for configuration, credentials, health, access revocation, company deletion, and provider lifecycle changes. Rename the generic command-only helper to describe its broader configuration contract. - Keep lease-key selection with the receiver owner. Shared scheduling contains no provider protocol. - Retain stopping ownership until listener shutdown and lease release complete. Concurrent acquisition joins the existing stop promise. - Remove receiver discovery from the periodic chat lane. Preserve voice and the other unmigrated duties. - Record producer coverage, failure evidence, limits, and remaining work in `doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md`. ## Verification Published head: `ab39569250f4b699a0ab7a829e66dbe27c15020b`. Discord #17557 has landed at master `5bf0142d45`. A normal merge reconciles its squash ancestry by applying only the recorded 27-file receiver patch from parent `547fdf2139`. Every other path matches that master. Every server and package production/test file is byte-identical to the published `6e88ca0201` source that passed the 131-case run, typecheck and build. Unrelated upstream UI/eval/documentation changes are preserved. This PR targets master. Fresh exact-head reviews and CI are required. - **131 distinct targeted cases pass.** Five complete receiver, coordinator, batching, startup, and GitHub health error-projection suites pass 99 cases. Twelve real Discord automatic-registration/receiver and Slack held-transaction cases pass separately, with 1,581 other chat integration cases deselected. The complete Photon channel suite passes 20 cases. - The standalone receiver and Photon fixtures each reproduced missing-table failures on separate initially empty PostgreSQL 17 databases. Both now apply pending migrations before client use; their complete 20-case files pass on those same databases. The large chat producer suite uses a separate migrated clone. - The old review concerns already have source fixes: discovery stops suppress their own completion wake at `1604a9fcba`; slow failed batches supply `nextRunAtOnError` at `ab56393acc`. Guarded mutations prove the existing regressions: reenabling the stop wake yields two inventory reads instead of one, and removing the error deadline fails the 1,100 ms provider-delay next-wake assertion. Both original source files are restored byte for byte. Normal full worker and real producer suites pass with those fixes. - All 131 distinct targeted cases, production server typecheck and build pass on source `6e88ca0201`. Exact source comparison carries that evidence to this normal merge; the unchanged suites were not rerun just for unrelated upstream changes. Receiver and Photon fixtures self-migrate separate initially empty databases; the large producer fixture uses its own migrated clone. Source is unchanged after these checks. Recorded listener-count warnings and controlled provider failure logs remain; no assertion was weakened. An initial whole-file identity check rejected expected parent queue additions; precise patch comparison confirms those declarations are correctly preserved. - Old server shard 12 failed because GitHub reports that its self-hosted runner lost communication. The grouped-test step is cancelled, and the retained log has no failing assertion or completed summary. The underlying host/resource/network cause is unproven. That interrupted job is not a pass; the updated parent and fixture source require a fresh full CI run. - [Current acceptance and failure record](https://github.com/paperclipai/paperclip/blob/ab39569250f4b699a0ab7a829e66dbe27c15020b/doc/plans/2026-10-25-chat-receiver-discovery-sleep-audit.md#compatibility-after-discord-squash-merge-october-27). Prior 218-case and full-workspace results describe earlier revisions. No full local repository-suite, multi-writer deployment, Cloud wake, host suspension, or fleet cost claim is made. ## Risks - Process-local transaction hints cover the active database owner and its shared clients. Arbitrary direct SQL or independent configuration writers need explicit wake integration. Startup/resume reconstruction and known foreign-lease takeover remain supported; this does not add general multi-writer discovery. - Active receiver sockets and lease renewals continue. This change does not prove host suspension or extend Cloud alarm coverage. - Configuration signals can arrive during drain, initialization, or shutdown. The worker preserves dirty scopes and the owner joins listener retirement before replacement. - Voice, Teams, milestone projection, publication maintenance, GitHub recovery, and the remaining stack inventory are separate work. - The Discord parent has landed and the receiver child now targets master. Fresh exact-head reviews and all CI must pass before merge. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, code execution, and test tools. The runtime does not expose a more specific deployment identifier or context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7d8bab773d |
perf: avoid full JIT and activity scans in Mine unread badges (#17973)
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Mine includes tasks the person created, was assigned, or participated in. > - Its unread badge ranks the visible personal top 100 before checking unread state. > - Correlated participation and privacy predicates can trigger expensive PostgreSQL JIT work. > - This change joins native participant IDs before ranking a narrow candidate projection. > - A partial actor index avoids scanning company activity history for each badge. > - Local representative scoped reads improve while dense unrestricted overhead remains explicit. ## Linked Issues or Issue Description **What existing behavior does this improve?** The Mine unread badge in the company sidebar. **Current behavior:** A fresh five-minute window recorded 402 ranking calls and 31.7 seconds of SQL time. Correlated predicates can spend more time compiling than executing. **Proposed behavior:** Build the same company-scoped participant set, use native issue keys, apply fresh authority before the top-100 limit, then retain the original canonical rank and second unread query. **Reason and benefit:** Reduce full JIT compilation and avoid an unindexed company-history activity scan. Realistic activity-only participation is included in the measurement. **Breaking changes:** No API or policy change. This PR adds one concurrent partial index migration. Related Mine rules in #16356 and #17756 are preserved. ## What Changed - Join distinct native participant IDs; guard exact legacy text before UUID conversion. - Skip redundant activity participation already covered by creation/assignment. Sort a narrow eligible candidate projection before canonical activity ranking. - Share existing pending-request and user-activity predicates with ordinary Mine reads. - Add partial `activity_log(company_id, actor_id, entity_id)` for user-authored issue activity. Register its idempotent concurrent migration with the existing top-level/invalid-index-repair runner. - Add real PostgreSQL coverage for visible top 100, unread/archive/source semantics, exact IDs, company scope and fresh grant/private-project access. - Add a pinned service benchmark with separate pre-index/indexed databases, identical data, full positional equality, ordinary-call timings, plans and real migration/cancellation proof. ## Verification - Fourteen focused PostgreSQL 17 tests pass (16.37 s), including assembled badge routes and grant revocation. Full workspace typecheck and build pass on the final production sources. - The earlier full local suite is incomplete: native PostgreSQL startup exhausted system semaphores, independently reproduced with empty initdb. All sixteen failed cases passed with task-owned Docker PostgreSQL 17. This is not counted as a full local-suite pass. Full Linux CI passes on final head `f44e46ce580e8345209d7165f57bd8bf696c517c`. Greptile and Paperclip Review are both 5/5 with zero unresolved comments. - Baseline `70c864633744cd96c0dc7394d631fc7119cd17d2`; source and DDL hashes retained. Two warmups and five alternating samples per arm/cell. Capture and EXPLAIN occur outside timed ordinary service calls. - All twelve 300k-history cells match complete positional rows in both SQL phases and final returned IDs. Half the personal tasks qualify through activity alone; the actor index contains matching and other-human rows. | Local service case | Before median | After median | Before / after | | --- | ---: | ---: | ---: | | dense / unrestricted / JIT on | 18.33 ms | 25.06 ms | 0.73x | | dense / scoped / JIT on | 909.30 ms | 221.10 ms | 4.11x | | sparse / unrestricted / JIT on | 13.64 ms | 5.84 ms | 2.34x | | sparse / scoped / JIT on | 719.07 ms | 10.79 ms | 66.64x | Small paths add at most 0.6 ms at the median. Dense unrestricted paths regress by 6.7–7.7 ms; dense scoped JIT-off is approximately neutral (218.34→223.24 ms). Sparse JIT-off paths improve too. This is not a uniform speedup. The actual baseline fails the compiler-work assertion. The final query avoids full JIT optimization/inlining and exceeds 1.5x where the old plan triggers those passes. Small/sparse controls meet the 25% or 2 ms allowance. Dense no-JIT cases are explicit diagnostics. The optional 60 ms absolute goal still fails at 221.10 ms on dense scoped reads: substantial privacy-predicate execution remains after compilation is removed. The failure is retained rather than counted as green. The actual normal migration runner upgraded 300k activity rows in 668.79 ms including inspection, index build/validation and journal work. Rerun took 21.83 ms; a deliberately cancelled build left an invalid index, and retry repaired it in 187.71 ms. Concurrent reads/writes completed; the final index was 188416 bytes. No migration was run on the hosted database. The million-row mixed-participation sparse control also passes all four visibility/JIT pairs with complete equality: unrestricted JIT-on 13.44→5.76 ms; scoped JIT-on 807.76→11.59 ms. The actor branch visits 36 indexed rows rather than scanning company history. Actual migration+runner took 255.06 ms; rerun 20.32 ms; cancellation repair/retry 241.73 ms. Concurrent reads and writes completed without failures. Reproduce: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-mine-unread-badge.ts --baseline-ref=70c864633744cd96c0dc7394d631fc7119cd17d2 --history-decoys=240000 --verify-migration --assert-performance --evidence-dir=<directory> ``` Add `--fixture=sparse --history-decoys=940000` for the million-row control. The default harness starts disposable embedded PostgreSQL and ignores DATABASE_URL. Recorded evidence used PostgreSQL 17.11 through a scratch constructor provider because the host native runtime has semaphore limits. All normal database allocation/migration/client paths are retained. Source-only EXISTS/lateral alternatives were measured and rejected after company scans or sparse regressions. A time bound would lose historical participation, and existing indexes cannot select a user's activity. This justifies the exceptional partial index migration. The exact sample ranges, plans, failures, source hashes, migration footprint and feedback-loop timing are in `doc/plans/2026-10-27-mine-unread-badge-query.md`. ## Risks - Dense unrestricted reads are slightly slower, and dense scoped reads retain substantial authority work. The PR removes a compilation/lookup bottleneck rather than all badge latency. - The concurrent build scans activity history and can wait for long-lived transactions. Local build/cancellation timings do not guarantee the duration on larger or slower hosted databases. User-authored issue activity gains one narrow index to maintain. - Participation, visibility, ranking and legacy ID semantics must remain exact; the real database regressions exercise those boundaries. - Hosted improvement is unverified. The affected ranking SQL averaged 78.8 ms while HTTP badge requests averaged 752 ms without correlated traces. No merge or deployment is included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and local code execution. The runtime does not expose the exact model ID or context window to this agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a91f78e7e0 |
perf(chat): reduce inbound queue discovery cost (#17971)
## Thinking Path > - Paperclip helps people manage companies of AI agents. > - External chat messages enter a durable delivery queue. > - Queue reads must preserve pending wake ownership and conversation order. > - Retained processed history increases the estimated cost of a correlated pending-action test. > - PostgreSQL 17 enables JIT for these small selectors and reads extra JSON in deadline discovery. > - This pull request uses a scoped pending-action set for open-work discovery. > - The benefit is less database work with the same selected deliveries and deadlines. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound delivery deadline discovery and conversation-head selection. **Subsystem affected** Server chat queue discovery. **Current behavior** A correlated `EXISTS` inside an `OR` has a high cost estimate on retained completed history. PostgreSQL 17 already hashes its pending-action read, but the estimate still enables JIT. Deadline discovery also reads JSON from completed deliveries that have no pending wake. **Proposed behavior** Use `(company_id, endpoint_id, delivery_id)` membership in the pending-action set for the two open-work selectors. Keep ready-only evaluation on its original scoped `EXISTS`. Keep retry, lease, ordering, lifecycle and endpoint rules. **Reason and benefit** The local retained-history fixture reduces deadline selection time. The small control and the final head selection are neutral. Ready-only selection remains an unchanged control. **Breaking changes** None. No API, schema, migration, stored history, worker, timer or notification changes. Related work: #17647 and #15927 change inbound wake-notice scheduling and selection. This PR changes the separate conversation queue ownership predicate. ## What Changed - Use a fully scoped pending-action set for open-work discovery. Exclude null delivery IDs to preserve exact boolean results. Keep duplicate actions from multiplying deliveries. - Preserve the original ready-only query. - Add five real PostgreSQL tests and a reproducible baseline/treatment benchmark. Capture the complete real service SQL, parameters, positional rows and execution plans. - Document the query cause, rejected hypotheses, test results and limits in `doc/plans/2026-10-27-inbound-queue-query-proof.md`. ## Verification Pinned pre-fix baseline: `09ad9adce2cf4742944353a3707a2f908e9bb35e`. Local fixture: Linux ARM64 PostgreSQL 17.11, Node 26.4.0. It has 1500 completed deliveries and actions, two open deliveries, and 32 KiB of synthetic incompressible JSON per historical delivery. A separate control has 20 completed rows. The normal settings use JIT on with a 100000 cost threshold, 128 MiB shared buffers and 4 MiB work_mem. The final quiet retained-history pair measured these local SQL medians. Each arm has two warmups and five alternating samples. The small control is neutral. | Actual selector | Baseline | Treatment | Complete positional output | | --- | ---: | ---: | --- | | Next inbound deadline | 18.78 ms | 1.58 ms | Equal | | Conversation head | 5.71 ms | 6.18 ms | Equal | | Ordinary ready control | 9.23 ms | 8.33 ms | Equal ; unchanged | The deadline plan uses fewer shared buffers. Head physical reads remain unchanged. Deadline discovery avoids JIT on this fixture. The pristine head estimate is below the JIT threshold in both arms, and its timing is neutral. With JIT disabled, deadline discovery still improves; head and ready timings overlap. - Red: the unchanged baseline fails the explicit local fixture work budget for JIT and deadline buffer reads. Green: the treatment passes. There is no timing threshold in ordinary CI. - Five new PostgreSQL tests and six existing assembled chat tests pass. They cover scoped/null/duplicate pending actions, boolean results, payloads, head barriers, reactions, paused/attention/AgentMail exclusions, retry/processing deadlines, provider ordering, lease loss, lifecycle retry and edit/admission races. - Actual old/new ready selectors have equal error outcomes for unrelated malformed retry timestamps. An initial AND-only oracle comparison was inconclusive and is not defect evidence. - `pnpm -r typecheck` passes. - `pnpm build` passes. - Final head `f3cac298d8a8a5377127b0db7057c52019bbe426` passes the complete [CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40625611887). Greptile is 5/5 on that head with no unresolved review threads. The earlier end-to-end cleanup lock-timeout failure passed on this revision. - The complete local suite was not run. The host's known native PostgreSQL startup failure exhausts SysV semaphores. Focused integration tests use a disposable Docker PostgreSQL constructor through the existing test seam. Complete final-head CI is required. Reproduce the real SQL comparison: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-inbound-queue-discovery.ts --baseline-ref=09ad9adce2cf4742944353a3707a2f908e9bb35e --require-work-budget --retained-only ``` Use `--jit-off` for the diagnostic control. The default command uses the repository's bundled PostgreSQL version and records it in output. The reported 17.11 run used the existing test constructor seam with a disposable `postgres:17` container. Raw samples, complete queries, parameters and plans are emitted as JSON. The benchmark uses disposable synthetic data and does not read a live database URL. ## Risks - Query plans depend on data and PostgreSQL settings. These local measurements do not prove deployed speed or request latency. - Head selection still reads historical JSON. Action history remains stored and scanned. This is not a constant-time queue implementation. - The changed false branch has no retry timestamp cast. Ready-only parsing, claims and retries stay on their existing query. - Background SQL totals can overlap. Do not add these savings into an HTTP latency claim. The largest possible saving for an affected selection is bounded by its observed SQL time, about 49–53 ms on average in discovery. ## Model Used OpenAI Codex, GPT-6 family, with tool use and code execution. The runtime does not expose the exact model ID or context-window size. Three parallel agents own separate performance fixes; this PR contains only the inbound queue fix. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f69dfc0e23 |
perf: use indexed latest-run probes for task interruption lists (#17969)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task lists show the latest run's interruption state for each visible task. > - The old lookup applies a coalesced text expression to company run history. > - That lookup cannot use the separate binding indexes for a small task page. > - This pull request selects the newest run through those existing indexes. > - The benefit is less database work with the same interruption result. ## Linked Issues or Issue Description **What existing behavior does this improve?** The latest-run lookup used by task-list interruption enrichment. **Current behavior** The query scans company history for a coalesced native, first-class, or legacy task ID. A local two-task fixture inspects 1,506 runs to select six candidates. A recent hosted sample recorded about 110 ms per latest-run lookup. This is one part of the list handler, not its whole latency. **Proposed behavior** Probe each existing binding index for the newest candidate. Pick the newest candidate across the three binding branches. Read the same evidence fields in the same SQL statement. Preserve native precedence, first-class precedence, legacy text matching, company scope, and ordering ties. **Reason and benefit** The representative local 100-task page improves from 17.17 ms to 3.00 ms (5.72×). A modern 100-task page with 40 runs per task improves from 26.78 ms to 3.82 ms (7.02×). All positional SQL fields, row order, and full service results match. **Breaking changes** None. Stored history, API fields, visibility filtering, execution authority, and external-conversation state are unchanged. Related behavior: #16965, #17509, #17552. The duplicate search found no open PR for this query rewrite. ROADMAP.md has no duplicate planned feature. ## What Changed - Use three disjoint indexed candidate branches with a lateral newest-run selection. - Guard UUID parameters to preserve the old text matching for noncanonical input. - Keep final evidence projection in the same database snapshot. - Add real PostgreSQL work and contract regressions, a pinned-source benchmark, and a verification note. - Require the plan regression to find real run-table nodes. Record the deep-native-history follow-up without adding indexes or DDL. ## Verification - Baseline red: the real two-task lookup inspects 1,506 run rows and fails the work bound. The unchanged semantic oracle passes. - Treatment green: both PostgreSQL query regressions pass. They cover conflicting bindings, malformed JSON, empty values, noncanonical IDs, duplicates, foreign companies, missing runs, and timestamp ties across branches. - Seven paired PostgreSQL 17.11 cases pass complete positional SQL and full service equality. Each arm has two warmups and five alternating measured samples. Fixture payload sizes are random input bytes before Base64 encoding: 24 KiB becomes 32 KiB of stored text per noise field. - Representative 100 × 12 history: 17.17 ms (16.79–17.48) → 3.00 ms (2.64–4.28). Shared buffer hits: 30,033 → 3,400. - Modern 100 × 40 history: 26.78 ms (25.79–27.40) → 3.82 ms (3.47–4.78). - The benchmark command, all cases, plans, fixture limits, and feedback-loop timing are described in [the verification note](doc/plans/2026-10-27-task-interruption-query-performance.md). - Reproduce with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-task-interruption.ts --baseline-ref=09ad9adce2cf4742944353a3707a2f908e9bb35e`. - `pnpm -r typecheck` and `pnpm build` passed at `20d46e3a16d10cf3c5399fc1211b17db6a4afdc6`. The current head has the identical production runtime; it adds one test assertion and documentation. - The focused query, real project-access repair/retry, and presentation suites pass: 19 tests across three files. The earlier plugin-sdk loading failure is resolved by normal build setup. - `pnpm test:run` was attempted, then stopped after repeated native embedded PostgreSQL bootstrap failures. The full local suite is incomplete. All 16 affected real-DB executor cases pass when rerun with disposable PostgreSQL 17 in Docker. - Linux CI passes the full server suites on this head. One unchanged Runner close fixture failed on the first CI attempt; its source tree is identical to the passing upstream #17439 baseline. One targeted job retry passed on the original head. Fresh checks run again for the test/documentation review fix. A separate macOS diagnostic fails the unchanged close variants too; that close issue is not diagnosed or counted as a local pass here. - The original head passed 55 checks with one neutral result and two expected Storybook skips. Greptile gave 5/5. The current head adds one nonempty-plan assertion and documents the deep-history follow-up. Its focused PostgreSQL suite passes 2/2. Final head `c1db106d7ab56b530af82d2bb0556feb0eda290b` is mergeable with all CI checks complete and passing, Greptile 5/5, and no unresolved review findings; the documented per-task history limitation remains an explicit follow-up. ## Risks - More index probes add about 0.43 ms to the one-run tiny control. The maximum 1,000-task page with one modern run per task is 9.99 → 14.11 ms, a 4.12 ms regression. Dense history pages save substantially more in the local fixture. - Each branch can still walk one task's own history. The first-class and legacy probes can discard native-bound rows after index lookup. The plan note specifies a 10,000-run follow-up, migration-free alternatives, and a measured index-build budget if indexes prove necessary. This PR does not make work independent of history depth. - Local SQL results do not prove deployed HTTP gains. The affected query averaged about 110 ms in the hosted sample; its critical-path contribution can be smaller because other enrichment reads run concurrently. - The main issue-list SQL and external-conversation gates remain separate work. This PR has no migration or startup job. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tool use. The runtime does not expose the exact model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
356c7de9b7 |
fix(runner): bind Grok packs to current admission source (#17972)
## Thinking Path > - Paperclip manages agents and their work. > - Native agents use verified Runner and provider-pack artifacts. > - A protected installed pack can be reused when its selected provider profile matches. > - Grok kept profile 1 when its sidecar gained effort, search, and startup diagnostics. > - An older Grok sidecar can therefore pass compatibility and reject the new session fields before provider startup. > - This PR binds the current Grok profile to its admission source so a fresh run selects the current verified pack. ## Linked Issues or Issue Description Refs: #17922, #17942, #17962. Related installation contract: #14024. No duplicate repair was found. **What happened?** The selected Grok profile digest stayed unchanged across native session contract changes. The installed-pack verifier correctly compared that digest, but the unchanged value let a pre-effort sidecar satisfy a current controller. That old parser rejects `grokEffort` and `grokDisableWebSearch` before the new startup diagnostics run. **Expected behavior** A fresh Grok run must use a pack that supports its current session contract. Source-only app changes must still permit warm reuse when the selected Grok profile matches. Reattachment must keep its exact admitted artifact authority. **Steps to reproduce** 1. Retain a protected pack with published Grok profile 1 and ACPX interface 1. 2. Upgrade the controller to the native effort and search contract. 3. Start Grok 4.7 with High effort and disabled web search. 4. The old profile passes installed-pack compatibility, but its sidecar rejects the new fields. **Paperclip version or commit** Reproduced from master at `09ad9adce2cf4742944353a3707a2f908e9bb35e`. ## What Changed - Advance Grok to profile 7. Keep historical versions 1–6 decodable and keep the Grok 1.0.13 runtime pin. - Stamp the Grok admission source identity into a generated launcher comment. Keep the existing physical command-byte digest check unchanged. Bind effort/search/credential handling, sidecar admission, and Rust backend/session/diagnostic source. - Check the source attestation in the existing TypeScript/Rust profile build gate. Keep its required fixture in the Docker context. Refuse to overwrite a versioned identity after source changes. - Add regression coverage for rejecting the old pack, current-profile warm reuse, exact historical recovery authority, and actual sidecar dispatch of Grok 4.7 High with both search policies. - Re-attest Copilot as profile 71 because its bound shared recovery decoder now retains Grok 6. Preserve Copilot 70 and all earlier identities. Keep the compatibility algorithm, other provider identities, dependencies, lockfile, and workflows unchanged. ## Verification - The production installed-pack regression failed before the profile update: the old Grok pack was accepted. It now passes. - 105 server manifest, artifact, archive, and authority tests passed. - 246 Runner profile, physical launcher integrity, Grok effort/search/credential, and actual sidecar process tests passed; six existing platform cases were skipped. - 31 Node packaging and identity tests passed, including an actual generator CLI regression that preserves all authority files when a versioned identity would change. - 55 Rust ACPX unit tests and 17 real sidecar pipe tests passed. After the final profile declaration, all 55 unit checks passed again. - Final `pnpm -r typecheck` and `pnpm build` passed at the final source. - CI on the preceding head caught Copilot’s stale attestation for the shared recovery decoder. The same current-source test failed locally, then passed after profile 71 was generated with version 70 retained. All 106 affected Copilot/profile checks pass. Final head `78e6d76fc52f492371e15574482b3641735a6d89` passed the full CI workflow; the failed prior job log is retained. - Review follow-up: mutating the Rust backend’s actual Grok descriptor field left the prior identity unchanged. This regression failed before binding `acpx_provider_backend.rs` and passes afterward. Profile 7 retains the prior versioned fixture and recovery decoder. All 179 affected Runner checks and 75 focused server pack/manifest checks passed after this change. - `pnpm test:run` hit the recorded embedded PostgreSQL bootstrap failure after five attempts. The full run also exposed a current-release fixture with the historical Grok digest; it now reads the current profile. Its 11 focused manifest tests now pass. The failed local run log is retained; final exact-head CI passed. - The same filtered-context Docker build that failed CI now passes locally after adding its required source fixture. - A separate probe used the built sidecar in a private HOME with no credentials. It admitted the current profile and the new Grok fields, then returned the closed working-directory rejection before provider launch or inference. - A historical parser probe reproduced the pre-effort sidecar rejection. These probes establish the contract boundary; they do not establish native subscription task acceptance. - Initial restricted local ownership tests failed because loopback listeners were denied. The same tests passed with loopback access. One new dispatcher fixture first lacked required instructions; the completed request passed. These setup failures are retained in local evidence. - Live native task acceptance remains pending deployment. Earlier deployed attempts failed before tool output. No successful native shell proof is claimed. - Final exact-head merge gate: all 62 contexts are clean (57 success, four intentional skips, one informational neutral formal review). Greptile and Paperclip reviews are both 5/5, with no unresolved actionable threads. The formal review uses the existing report-only neutral policy. ## Risks - The first fresh Grok run after upgrade may need to stage the current provider pack. Matching current profiles keep their existing reuse path. - Fresh admission rejects older Grok profile identities. Historical decoding and exact reattachment authority remain separate; this change does not reinterpret an old identity or replace an active session's pack. - Future changes to bound source require a new Grok profile attestation. The build gate fails if the checked-in identity is stale. - Model, effort, search, credential, approval, and retry policy are unchanged. ## Model Used - OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks; full-suite database failure is retained above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.8 |
||
|
DottaandPaperclip
|
cff331f6ca |
fix(evals): retain local controller cost origin (#17990)
## Thinking Path > - Paperclip manages AI agents and their work. > - Its eval journals retain model costs and operating reservations under one cap. > - The operating policy records the controller's execution origin. > - The schema currently accepts only GitHub-hosted and AWS Fleet origins. > - A local controller must therefore use an inaccurate label or fail admission. > - This PR adds the local origin and keeps the hosted-only schedule gate. ## Linked Issues or Issue Description Refs #17986. No duplicate open PR or issue was found in the operating-overhead/local search. **What existing behavior does this improve?** The Product E2E operating allowance and its retained journal policy. **Current behavior** At master `40c1ab36cd`, an unscheduled local controller cannot set `runner: "local"`. The generic allowance validator rejects it. Local qualification can use the same accounting controls but cannot record its origin correctly. **Proposed behavior** Accept the local origin in the generic allowance. Keep all component, price, cap, expiry, and receipt rules. The scheduled validator still requires `github-hosted` and rejects a local allowance. **Reason and benefit** Local qualification can retain an accurate execution origin without claiming that its reservations are billed GitHub charges. ## What Changed - Add `local` to the operating allowance's runner type and validator. - Exercise local journal creation, model admission, retained overhead, restart, and reconciliation in the existing boundary test. - Check that a local allowance fails Product scheduled admission. - Pin new hosted Runner controller admission to `github-hosted`. Preserve original labels when reading historical manifests; those reads cannot authorize a new worker. - Exercise the hosted-origin rejection and historical/local reads in both source and copied portable runtimes. - Document the local label and its accounting limits. ## Verification - Before the fix: the new local persistence case and schedule-boundary assertion failed; eight tests passed. - Focused ledger, overhead, operations, and scheduled finalization tests: 71 passed across five files. - Controlled Runner source and portable tests: 86 passed. The new hosted-origin regression failed before the fix. - Full Product support suite at the first commit: 4,524 passed, three skipped, and one retained embedded PostgreSQL bootstrap failure. The same database failure was previously reproduced on a clean baseline. - Separate Node test phase on the host: 137 passed. - Actual Linux controller: database lifecycle tests pass. After installing its missing `zip` utility, the full Product suite has 4,527 passes and one Copilot parent-version coverage failure. That same failure reproduces on clean master `40c1ab36cd` with the same Linux filesystem and dependencies (14 passes, one failure). Copilot files are unchanged. The initial seven ZIP-related failures and the watcher failure remain retained. - The final fresh Linux semantic check passes: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=1 node node_modules/typescript/bin/tsc --singleThreaded -p tests/runner-e2e/tsconfig.json`. The parallel compiler OOM is retained. No semantic checks were omitted. - Both fresh reviews are 5/5, with the addressed thread resolved. Hosted Build first failed during unchanged Pi setup prefetch with an operation timeout. One job rerun recovered; the original failure remains retained. All 58 latest-head checks are now acceptable. - No paid model requests, sandbox creation, schedule changes, or historical journal edits. ## Risks - The caller supplies this origin. It must describe the actual controller. - `local` does not prove zero compute cost, provide a default allowance, or release a hold. - Both maintained hosted admission paths require the GitHub-hosted controller origin. A previously accepted hosted policy with an AWS/local label now fails new admission. Historical reads preserve the original evidence. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
40c1ab36cd |
feat(evals): register explicit Daytona feedback persistence (#17986)
## Thinking Path > - Paperclip manages AI agents and their work. > - Product E2E evals check real server, browser, runner, and provider behavior. > - Feedback persistence already checks saved complaints and suggestions with exact bodies and run attribution. > - Its catalog registers only local cells, so a Daytona campaign cannot select the same workflow. > - This PR registers the existing workflow for the two native Daytona profiles and tests their gateway bindings. > - Remote cells stay explicit-only and unqualified until a bounded live run passes. ## Linked Issues or Issue Description Refs #17980 and #17975. No duplicate open PR or issue was found in the Daytona feedback search. **What existing behavior does this improve?** The `feedback-persistence` Product E2E suite and its explicit execution selectors. **Current behavior** At master `d54f776ca0`, the suite has only local Claude and Codex cells. Selecting the matching Daytona ID fails before admission. The gateway manifest already supports these native remote profiles. **Proposed behavior** Register four cells: local and Daytona for each native profile. Reuse the existing saved-body and completed-tool oracle. Require an immutable image and Daytona credentials. Keep one attempt, one expected work run, five-minute local and inherited 15-minute remote case timeouts, and the existing 60-minute provider creation TTL. **Reason and benefit** A bounded remote qualification can check both a simple task and real feedback persistence. Local evidence remains separate from remote acceptance. **Breaking changes** Explicit `--suite feedback-persistence` now lists four cells instead of two. `--all` still excludes the suite. Daily and weekly manifests still select their existing exact local IDs. ## What Changed - Register the two native Daytona feedback cells and increment the suite definition version. - Address the remote cold-start review: inherit the base task's 15-minute Daytona deadline and retain the five-minute local limit. The regression failed before the fix and now passes. - Check immutable-image requirements, credentials, lifetime, and shared gateway connections in existing contract tests. - Fix the config type in the TTL catalog test added by #17980. Keep its runtime lifetime assertions. - Document remote selection and its unqualified status in the existing eval guides. ## Verification - Before the catalog change, the registration regression failed: one failed and 16 passed. - Focused catalog, feedback, operations, ledger, manifest, and summary tests: 114 passed across six files. - Read-only discovery lists exactly the two Daytona feedback IDs. - The Node test phase passes separately: 137 tests. The retained Vitest database failure prevents the unit launcher from reaching that phase. - Full Product support suite: 4,523 passed, three skipped, and one embedded PostgreSQL bootstrap failure. The same database failure was reproduced on a clean baseline before this PR. - Fresh Linux Product typecheck passed at the review-fix commit: `GOMEMLIMIT=2GiB GOGC=50 GOMAXPROCS=2 node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. The default run and the ineffective JavaScript-heap attempt exited 137 and remain retained. The compiler launches a native child process; its memory controls let the unchanged semantic check complete. Host typecheck still has 15 stale installed DB/Runner export errors; the two TTL test typing errors are fixed. All 58 latest-head checks are acceptable (success, neutral, or skipped). Both fresh reviews are 5/5, with no unresolved review threads. - The updated credential-free Linux controller discovers both planned remote IDs and launches Chromium 151.0.7922.34. Runtime build inputs are unchanged from the prior controller. The Daytona image content ID is still `b55f03dc9c633db300b70206a691ff45a8e779e56a3c546f9358e3282d3597c9`. - No model requests, sandbox allocation, or paid schedule changes. ## Risks - Selecting the whole explicit suite now includes remote cells and requires Daytona configuration. Use exact IDs for a bounded paid campaign. - Registration and contract tests do not prove live remote behavior or token accounting. Remote qualification remains pending. - The shared dependency checkout limits the local host typecheck. The fresh Linux semantic check passed. Earlier compiler OOM failures and the database bootstrap failure remain retained. ## Model Used - OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>nightly/v2026.1028.0-nightly.0 canary/v2026.1028.0-canary.7 |
||
|
DottaandPaperclip
|
d54f776ca0 |
fix(daytona): bound ephemeral eval lifetime at creation (#17980)
## Thinking Path
> - Paperclip manages agent work through Runner and sandbox providers.
> - Product E2E runs paid work in disposable Daytona sandboxes.
> - A controller deadline cannot stop remote compute after the
controller crashes.
> - Inactivity cleanup does not establish an independent wall-clock
lifetime.
> - Daytona supports a lifetime in the creation request.
> - This change exposes that option in the Daytona driver and verifies
the provider expiry.
> - Ephemeral evals can retain a finite lifetime even when their
controller is unavailable.
## Linked Issues or Issue Description
Refs #17947 and #17978. No duplicate creation-lifetime PR was found.
**What happened?**
The ephemeral Product E2E fixture configures inactivity stop, archive,
and delete intervals. Ordinary task leases do not supply a requested
expiry. A crashed controller therefore has no confirmed wall-clock
sandbox lifetime for its cost bound.
**Expected behavior**
Each ephemeral eval sandbox must carry a provider-enforced lifetime from
creation. The driver must reject invalid bounds and reject a sandbox
whose provider expiry is missing, expired, or later than the configured
remaining lifetime.
**Steps to reproduce**
Acquire a Daytona lease from the ephemeral fixture. Inspect the creation
request and returned expiry. Before this patch, the request has no
lifetime and the lease has no expiry unless its caller supplies one.
**Paperclip version or commit**
Current source based on master at
canary/v2026.1028.0-canary.6
|
||
|
DottaandPaperclip
|
5bf0142d45 |
fix(server): schedule Discord command registration from saved deadlines (#17557)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connections register and verify provider commands in the background. > - Discord command recovery ran inside the recurring Gateway runtime scan. > - Its existing durable records already store verification and retry deadlines. > - This pull request gives command registration its own shared-coordinator worker and scoped commit notifications. > - Idle command work stops querying between real deadlines. Existing command authority and recovery remain intact. ## Linked Issues or Issue Description Refs #17554 and #17561. Both foundations have merged. This slice targets master. Runtime discovery and the other chat maintenance duties remain separate work. **What happened?** The chat runtime maintenance loop called Discord command registration on every reconciliation. It repeatedly read durable command state even when the next remote verification or retry was minutes away. **Expected behavior** Use committed producer changes and saved deadlines. Keep the existing five-minute verification policy, Retry-After, uncertain-write quarantine, namespace ownership, and credential fences. Do not add a database table for the timer. **Steps to reproduce** Configure a Discord connection and let command registration settle. In the parent revision, repeated runtime reconciliation still invokes command recovery. This revision reconstructs command deadlines at startup and removes that call from runtime reconciliation. ## What Changed - Expose normalized deadlines from the existing registration store. Park indefinite retries and invalid or foreign-owned records. Signal before store-owned claims, receipts, and retry-result writes. - Discover eligible endpoints in bounded pages. Refresh only affected endpoint, connection, or company scopes after commits. Dispatch at most 25 due commands per pass with fair company rotation. - Wire configuration, credentials, provider lifecycle/failure paths, inbound activation, generic connection mutation/removal/health, access revocation, and company deletion. - Register the worker through the chat service and shared coordinator. Preserve direct foreground setup calls. Refresh an existing local Gateway through its current owner after command work. - Preserve dirty work during drain and concurrent changes, back off unresolved attempts, and join admitted provider work during shutdown. - Document producer coverage, retained deadlines, supported writers, verification failures, and remaining runtime/Cloud obligations in `doc/idle-discord-command-registration.md` and its dated audit. ## Verification Published head: `547fdf2139144e1e6484198278bf4bb0eade8794`. It normally integrates master at `f8563d25077b17681ab6ef5949f7b07415a58ede`, including landed health #17554 and company deletion #17561. Squash ancestry caused merge conflicts; resolution applied this slice's exact 26-file owned patch from its recorded parent to pinned master. Every other path matches master. Migration 0394 and the current lockfile are preserved. - **141 distinct targeted cases pass** on reconciled production source: 78 complete command-store, batching, coordinator and startup-timing cases; 8 real automatic Discord and held-transaction Slack cases; and 55 complete startup/export and company-deletion cases. The 1,581 other chat integration cases were deselected. - Server typecheck and build pass. The company-deletion fixture self-migrates an initially empty isolated database. The two Discord fixtures use separate migrated clones; this is not a fresh-schema claim for those fixtures. Signal-listener warnings remain recorded without assertion failures. - The parent integration preserves health and command notifications at shared producer sites. It changes no provider behavior, schema or coordinator contract. Prior producer, backlog, drain/resume, malformed-hint, transaction, namespace, shutdown and retry regressions remain in the complete command suite. - [Landing compatibility acceptance](https://github.com/paperclipai/paperclip/blob/547fdf2139144e1e6484198278bf4bb0eade8794/doc/idle-discord-command-registration.md#acceptance-after-the-company-deletion-parent-landed-october-27) records current scope and the earlier failures. Older full-matrix or combined-stack runs apply only to their stated revisions. - Fresh exact-head Greptile, Ponytail if present and all CI are required. No full local repository suite, live provider, Cloud alarm, host suspension or savings claim is made. ## Risks - Discord still has real five-minute verification and failure-retry obligations. This does not prove host sleep or external alarm coverage. - Runtime discovery, receipt-to-runtime projection, Gateway lease renewal and persistent sockets remain separate. The remaining runtime poll can still read command receipts; it no longer invokes the command protocol. - Process-local signals cover integrated writers. Independent SQL writers and other replicas require an explicit wake contract. Existing authorization and durable ownership checks remain authoritative. - Refreshes coalesce with a 30-second minimum spacing. Saved work is never discarded because a call returned without progress. - No schema migration, new table, workflow or lockfile change is introduced by this slice. Landed parent repairs are retained. The receiver child must receive this integration before landing. ## Model Used OpenAI Codex, GPT-6 family, with repository tools, code execution, and reasoning. The exact serving version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4c314117d2 |
fix(evals): bind active image identity and isolate routing checks (#17978)
## Thinking Path > - Paperclip manages agent work through Runner. > - Product E2E uses a content hash to identify its Daytona image. > - The hash must include the active provider identity declaration. > - The active Copilot fixture advanced past the manually listed image inputs. > - The runner routing test also captured later YAML steps as shell commands. > - This change binds the active fixture and stops shell extraction at the next step. > - Operators can verify image identity and routing against the current source. ## Linked Issues or Issue Description Refs #17937 and #17947. The related Copilot work includes #15563. This change repairs the shared image contract; it does not add a Copilot qualification claim. **What happened?** Two image tests fail because the active Copilot identity fixture is absent from the image hash inputs. Four routing tests execute later workflow YAML as shell commands after checkout and dependency steps were added to the authorization job. **Expected behavior** The image hash must change when the active identity fixture changes. The routing tests must execute only the selected routing step. **Steps to reproduce** Run the Daytona image and paid runner routing unit files on current master. Six assertions fail before this patch. The same failures occur on the clean |
||
|
|
39279aa34f |
feat(ui): shared error, connectivity and query foundation for disconnects (#15754)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board reads API data and keeps the current task on screen. > - A restart or temporary network failure can interrupt those reads. > - Separate page error handlers can hide cached data or expose raw server codes. > - The app needs one connection state, readable error copy and a consistent read policy. > - This change provides that foundation and moves affected read surfaces onto it. > - The dependent write layer in #17960 preserves edits and submissions across the same failures. ## Linked Issues or Issue Description **What happened?** Temporary API failures could replace loaded content with errors. Different surfaces handled the same outage differently. Some showed raw server codes. A successful query to an unrelated service could also falsely report that Paperclip had recovered. **Expected behavior** Keep cached content visible. Show one connection banner during sustained trouble. Confirm recovery through the Paperclip health probe or its WebSocket. Retry transient reads and show readable errors for permanent failures. **Steps to reproduce** 1. Open a populated task or workspace page. 2. Interrupt API requests with a 503 or a dropped connection. 3. Keep an unrelated external request in flight. 4. Confirm that cached content remains and that the external result cannot clear the outage. 5. Restore the API and check that reads refresh. The write layer first merged into this feature branch in #15769 now has its own dependent PR, #17960. This PR contains the shared foundation and read changes. Merge this PR first. ## What Changed - Classify API errors and provide readable error copy, including non-JSON proxy responses and Retry-After handling. - Use one connection store and health probe with bounded backoff. Confirm recovery from the server health probe or live-updates WebSocket. - Pause reads while disconnected, retry transient errors, and refresh active reads after recovery. - Keep cached data visible through shared query views and affected page, panel and plugin read surfaces. - Add one app-level connection banner and support the pending-write count consumed by the dependent write layer. - Add a query-error rendering guard. Call it directly from the existing CI step. Update development documentation. ## Verification - Historical-chat lookup now follows the same ready/stale roster state as rendering. Three regressions fail before the fix; the six new cases pass after it, including working Retry, fresh data, no-cache error, permanent 403 and company isolation. All six focused query/recovery suites pass 200 tests. Final-commit CI passes and Apex is 5/5 with no unresolved findings. Canary Dry Run initially refused a preview version that was already on npm. It passed after one rerun with fresh registry metadata; no check was disabled. - Integrated current master and retained both transcript interest registration and connectivity reporting on socket open. The seven focused recovery/query suites pass 288 tests, and UI typecheck and production build pass. All 9,451 UI tests across 799 files pass on the merge commit. Final-commit CI passes and Apex is 5/5 with no unresolved findings. - Current head `70d9335d732219352ef0f98414439766bbcb33df`: all hosted CI checks pass. Apex is **5/5** on that head with no unresolved findings. The branch has no merge conflict. GitHub still requires code-owner approval. - The foundation passes 9,430 UI tests in 804 files. After integrating current master page paths, 190 focused page tests pass. - Two unrelated-success regressions fail before the latest fix. With the fix, 62 focused UI tests, all six guard tests and UI TypeScript pass. - The composed foundation and write layer pass **9,505 UI tests in 808 files**, UI TypeScript, a production UI build, and query-error, token and map guards. - A real Chromium journey on the composed source injects a 503 during title save. The loaded task stays visible; one reconnect banner appears; the edit saves after recovery; another task stays unchanged; the saved title survives reload. - Full recursive local checks were not repeated because Runner compilation exhausted local disk during another repair. Hosted checks must pass before merge. No new staging deployment is claimed. - Latest review follow-up: a cached CaseDetail now survives a failed HTTP 429 refresh. Its page regression fails before the fix and passes after it. Eighty focused foundation tests, UI TypeScript, build and guards pass after that change. - Earlier CI evidence is retained: several foundation jobs received a runner shutdown signal together. The write-layer head also failed an unchanged Runner descriptor close test with `provider_transport_failed`. The same Runner source passed that suite in the foundation run. New CI runs cover the updated heads; no check or assertion was weakened. ## Risks - Shared retry defaults change the time before transient read errors appear. Permanent errors still fail promptly. - ApiError.message now contains readable copy. Callers that compare raw codes must use ApiError.code. - Cache success can come from another service or local data. It must not confirm Paperclip recovery. - Durable writes are reviewed separately in #17960. This PR does not change database or mutation contracts. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) produced the original work with tool use. OpenAI Codex, GPT-6 family, assisted with restoration, integration, tests and review fixes. The Codex session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Amy (Opus) <claude@users.noreply.github.com> |
||
|
DottaandPaperclip
|
9b7d5b35eb |
fix(evals): refresh forecasts from complete request receipts (#17975)
## Thinking Path
> - Paperclip coordinates agent work through Runner.
> - Scheduled Product E2E campaigns need measured cost and time
forecasts.
> - The saved forecast samples came from October 21.
> - The October 27 qualifications retain final usage for all 19 model
requests.
> - The Claude measurements are hosted; the Codex measurements are
local.
> - This change refreshes the samples and keeps measurement hosts
separate.
> - Operators can inspect the latest evidence without treating it as
current qualification or an invoice.
## Linked Issues or Issue Description
Refs #17127 and #17946.
**What happened?**
The scheduled forecast still uses the older pilot amounts. It reports no
hosted measurement and does not record the measurement host per sample.
**What did you expect to happen?**
The forecast must retain the latest complete model receipts and their
source hashes. It must keep local and hosted timing cohorts separate. A
changed definition must remain unqualified.
**Steps to reproduce**
Prepare the daily or weekly forecast from the default branch. Inspect
its saved measurement date, totals, and hosted fields.
## What Changed
- Refresh four samples from two intact request ledgers and four retained
Product results.
- Record measurement hosts, final request counts, and original campaign
outcomes.
- Add historical hosted totals. Current hosted totals still require
matching evidence for every selected cell.
- Keep hosts in separate per-cell cohorts. Preserve definition drift and
unknown slow-tail estimates.
- Document the model-cost scope and timing limits.
## Verification
- Forecast and scheduled-preflight tests: 33 passed.
- The extraction replays both hash-checked ledgers, requires final
receipts, and checks the retained pricecards. The four samples cover 19
final requests.
- Daily historical model subtotal: $0.428650100; summed attempt time:
107.286 seconds.
- Weekly historical model subtotal: $0.793676500; summed attempt time:
158.093 seconds. The Codex samples are local.
- Full Product support suite at the initial head: 4,510 passed, seven
failed, three skipped. A clean
canary/v2026.1028.0-canary.5
|
||
|
DottaandPaperclip
|
f8563d2507 |
fix(server): remove chat endpoints before company agents (#17561)
## Thinking Path > - Paperclip manages agent companies and their connected chat channels. > - A company deletion must remove dependent data in one transaction. > - Chat endpoints reference assigned agents with a restrictive foreign key. > - The current deletion order purges agents before those endpoints. > - This pull request removes company endpoint rows and chat history before issue and agent deletion. > - The existing lifecycle checks, company isolation, and rollback remain intact. ## Linked Issues or Issue Description **What happened?** Deleting a company with a chat endpoint fails on the endpoint's assigned-agent foreign key. The failure was found while verifying the company-deletion wake in #17557. **Expected behavior** Company deletion removes endpoint dependencies before their referenced issues and agents. It preserves other companies and global Discord ownership tombstones. A later failure rolls back the cleanup. **Steps to reproduce** Create a company, an agent, and a Discord endpoint assigned to that agent. Complete agent termination, then delete the company. The old order fails with the restrictive endpoint-to-agent foreign key. **Paperclip version or commit** This prerequisite includes merged #17554 at `94a780286f` and supports #17557. Only four cleanup-owned paths differ from that landed parent. Related #16465 covers the deletion feature flag, not dependency order. ## What Changed - Add a chat-owned company-deletion participant using the existing transaction contract. - Remove only the requested company's endpoints before issue deletion and the agent lifecycle participant. - Preserve namespace tombstones and existing restrictive foreign keys. - Add real PostgreSQL regressions for ordering, isolation, and rollback. - Migrate a validated disposable external database before fixture use, use the shared PostgreSQL bootstrap budget for setup, and remove fixture-owned Discord tombstones during test teardown. ## Verification Published head: `9766cc61c42fe0f73e241bbe99f9d9f145f4b11c`. Parent #17554 is merged. This branch reconciles the actual landed health squash `94a780286f` and targets master. Fresh reviews and CI are required for this exact head. - The latest review requested the repository's shared EMBEDDED_POSTGRES_TEST_TIMEOUT_MS (90 seconds) for bootstrap. This replaces the earlier suggested 30-second limit. All six existing cases pass on another initially empty database with that constant; no test assertion or production code changes. The documented contention budget is repository evidence, not a locally reproduced 40-second bootstrap failure. - All six complete cleanup-removal service cases pass on an initially empty dedicated PostgreSQL 17 database with the updated health parent. - The pre-fix real endpoint-deletion case failed with missing-table errors because external fixture setup skipped migrations. Applying pending migrations before client use fixes that failure; no assertion is removed or weakened. - An independent query after the complete suite finds zero fixture ownership tombstones. Production company deletion still preserves those records. - Cases cover endpoint removal before issue and agent references, other-company isolation, namespace ownership preservation, and rollback after a later failure. Both product regressions failed before the original ordering fix. - Server typecheck and build pass with the updated parent and fixture repair at `21be75d2b0`. The landed-parent reconciliation changes only the lockfile; all source and test bytes remain identical. All six cases pass again with the new installed dependency patch. The initial scope-proof command raced a shared `origin/master` update; a new proof pins the actual landed parent and confirms the four-file scope. A later native-checkpoint merge `09ad9adce2` has no overlap with these four files. - [Owning acceptance document](https://github.com/paperclipai/paperclip/blob/9766cc61c42fe0f73e241bbe99f9d9f145f4b11c/doc/company-chat-deletion.md#current-parent-acceptance-october-27) retains the failing baseline, parent scope, and limitations. No full local repository-suite or deployed suspension claim is made. Older inherited gateway/combined-stack evidence is separate from this four-file fix. ## Risks - The endpoint cleanup shares the caller's transaction. A later deletion failure must restore the endpoint; the regression verifies that boundary. - Existing retained voice-history restrictions and external provider teardown policy remain unchanged. This fix does not bypass restrictive foreign keys. - No migrations, lockfile changes, or workflow edits. ## Model Used OpenAI Codex, GPT-6 family, with repository inspection, code edits, and tool-assisted tests. The exact runtime variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
SudheerandClaude Opus 5.5
|
70c8646337 |
fix(ui): drop top margin and border on first contextual sidebar section (#17412)
The first section in the contextual sidebar rendered a leading top margin and divider with nothing above it. Use first: variants so only sections that follow another section get the spacing and border. <!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI has contextual sidebars, such as Skills, Apps, and Agent > - These sidebars share one set of style constants in `contextual-sidebar-styles.ts` > - Each section in the sidebar has a top margin and a top border to separate it from the section above > - The first section has no section above it, but it still shows the margin and the border > - This pull request adds `first:` variants to the section style > - The first section now sits flush at the top, and the other sections keep their divider ## Linked Issues or Issue Description **What existing behavior does this improve?** The section style in the contextual sidebar. It adds a top margin and a top border to every section. **Subsystem affected** UI (`ui/src/components/contextual-sidebar-styles.ts`) **Current behavior** The first section in the sidebar shows a top margin and a divider line. Nothing is above it, so the divider looks wrong and wastes space. **Proposed behavior** The first section has no top margin and no top border. Each later section keeps its margin and divider. **Reason and benefit** The sidebar looks cleaner at the top. The change is small and uses only existing design tokens. **Breaking changes** None. ## What Changed - Added `first:mt-0 first:border-t-0` to `contextualSidebarStyles.section` in `ui/src/components/contextual-sidebar-styles.ts`. Before: <img width="585" height="410" alt="Screenshot 2026-10-24 at 10 14 36 AM" src="https://github.com/user-attachments/assets/12c35f5f-7993-4fdb-89b2-df3394859bf8" /> After: <img width="560" height="377" alt="Screenshot 2026-10-24 at 10 14 24 AM" src="https://github.com/user-attachments/assets/92e1f7ab-f4c0-48ac-832a-a3726770deac" /> ## Verification - Run `pnpm check:token-gates`. It passes. - Open a page that uses a contextual sidebar, such as Skills or Apps. - Check that the first section has no top gap and no divider. - Check that the later sections still show their divider. ## Risks - Low risk. The change affects only the spacing and the top border of the first sectio - Several sidebars share this style, so check each one: Apps, Skills, Agent, and Chat detail. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: Anthropic - Model: Claude Opus 5.5 (`claude-opus-5-5`) made the commit. Claude Sonnet 5.5 (`claude-sonnet-5-5`) helped review the change afterward. - Capabilities: tool use and code execution in Claude Code (CLI). - The one-line style change is human-authored. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only puub.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
c9213d1be4 |
fix: send saved queued messages after Stop (#17959)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People can save follow-up messages while an agent runs. > - Stop preserves those messages and stops the active execution. > - The queue removed its delivery controls when that execution ended. > - Native admission also refused an explicit resend of older queued input. > - This change adds Send for a stopped queue and records fresh board intent. > - The saved message can continue through normal cleanup and admission checks. ## Linked Issues or Issue Description **What happened?** A person queued a message with a screenshot while an agent was starting. Steering was not yet available. They stopped the run. The message remained visible, but no control could send it. Native queue delivery also rejected this transition. **Expected behavior** A stopped queue offers Send. The click preserves message IDs, content, attachments and original authors. It starts at most one eligible continuation. Stop remains effective until the person gives new direction. **Steps to reproduce** 1. Start a native task and queue a second message. 2. Stop the active run before it consumes that message. 3. Open the task after cleanup. Observe the saved message with no send control. Refs #16965 for the queue-control change and #15015 for stopped conversation recovery. No open repair for this exact transition was found. ## What Changed - Show Send for a saved deferred queue with no active run. - Accept the existing board queue action for a stopped native target. Keep live native interruption restricted. - Use the server-recorded click time to authorize older native messages. Reject stale, malformed and wrong-actor intents. - Reuse normal queue dispatch, revision checks, cleanup proofs and execution admission, including a failed native predecessor after a verified Send. - Ignore a fully committed, successful recovered export when admitting a fresh message, while retaining the wait for results still committing. - Honor an applied `/new` conversation boundary while retaining native execution holds until the existing stop and cleanup checks verify them. Workspace holds also survive reset. Check the newest failed or cancelled native execution even when older recovery actions exist. - Keep native automatic delivery separate from the explicit Send action. Update execution and access documentation. ## Verification - Meaningful regressions failed before the fixes: stopped queues lacked Send, native admission refused fresh intent, cleanup and dispatch receipts were missing, deferred rows disappeared, live plan Interrupt rejected its own receipt, and Agent Chat `/new` inherited an old non-workspace execution hold. - Six assembled-wakeup cases fail before the reset cleanup fix. All 11 reset/history cases pass after it: live processes, missing identity, quarantined cleanup, remote leases and live controllers remain held; verified cleanup proceeds and retires the hold. Two more cases fail before the fallback fix and pass after it: Send verifies a stopped or live native predecessor despite a later reset receipt and no recovery record. - **525 server tests pass** across seven queue admission, route, dispatch, envelope, provenance, blocker and conversation files. **343 queue/thread UI tests** also pass. Server/UI TypeScript, token and map guards pass. The final scoped server typecheck also builds native Runner successfully. - The latest regressions prove that completed recovered exports permit fresh input, and clearing conversation history still requires old native stop and cleanup verification; an unsafe workspace still blocks after `/new`; failed native Send reaches actual continuation dispatch with the original queued comment. A result still being committed remains deferred. History plus a newer Stop cannot hide a missing process/controller/lease receipt. - The real Chromium Stop/Send journey passes on current head `75fa5e7222c59f43ff7ac326e81093bc9da28330`: queue a follow-up, Stop, reload, explicitly Send, then verify exactly one useful response and no live run. Earlier browser journeys cover legacy Interrupt, interruption during a write and Stop followed by a new direction. These use an isolated app/database and controlled ACP fixture, not an external AI account or a staging deployment. - Negative cases retain actor, company, issue, assignee, queue and cancellation-receipt bindings; cleanup/process/workspace/controller holds; approvals; pause and budget gates. Typed responses keep their original resolver and immutable result. - Full local recursive typecheck stopped in unchanged Rust Runner compilation with ENOSPC, so the chained full tests/build did not run. All hosted CI gates pass on final commit `75fa5e7222c59f43ff7ac326e81093bc9da28330`, including typecheck, tests, build and E2E. Apex review on that commit is 5/5, with no unresolved findings. Earlier unrelated CI failures involved PostgreSQL fixture-cleanup deadlock and an unchanged decision-expiry deadline assertion; the latter's 20 cases pass locally without source changes. No assertions or limits were weakened. Local resource cleanup was limited to this task's verified completed test processes and rebuildable npm caches. ## Risks - Send can authorize a new turn for a saved message after Stop. A Send click must be newer than the stopped execution; a live plan Interrupt is bound to the cancellation receipt produced by that same click. It does not imply that cleanup finished or that an external action is safe to replay. - The existing endpoint and intent format are reused. No schema or credential change is required. Live native steering and legacy interruption keep their existing paths. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose the exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
6102f8e82c |
fix: pass healthy managed Runner connection tests (#17965)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board tests an agent's selected runtime and AI account. > - Cloud prepares and verifies the managed Runner runtime before execution. > - Its remote ACPX preflight reports that deferred step as a warning. > - The test keeps the warning after the selected account passes its hello probe. > - The board then asks the user to fix a healthy connection. > - This change makes that specific managed step informational after account verification. ## Linked Issues or Issue Description **What happened?** A Cloud agent's Claude connection test passes with Legacy. With Runner, the same account completes the Claude hello probe but shows "Connection needs attention." The only warning says that remote runtime preparation must occur before launch. **Expected behavior** A verified managed connection passes when its only warning concerns runtime preparation that Paperclip performs before execution. Failed authentication and actionable runtime checks retain their results. **Steps to reproduce** 1. Select a working Claude subscription on a Cloud-managed sandbox. 2. Select Runner and run the connection test. 3. Observe a successful Claude hello probe and a warning result. Related: #17961 covers managed OpenRouter model verification. #15880 adds persistent computer runtime interfaces. Neither changes this Cloud connection-test status. ## What Changed - Recognize the deferred ACPX runtime check after a provider hello or live API-key verification succeeds. - Report that check as informational for Cloud-managed sandbox connections. - Preserve every other warning and error, including unexplained warning results. - Add service regressions and document the test/launch boundary. ## Verification - Red: the healthy managed-Claude case returned `warn` after a successful hello. The regression fails before the change and passes after it. - **116 tests pass** across connection verdicts, environment-test HTTP routes and adapter registration. Scoped server TypeScript and the server build pass, including the native Runner build. - Negative cases cover rejected credentials, missing hello receipts, missing runtime packages, actionable or unexplained warnings, self-hosted instances, SSH targets and unexplained provider warning results. API-key success requires live provider re-verification. - Operator evidence confirms that the reported test lease was released with cleanup successful. The agent's saved adapter is Legacy. No saved account, agent or environment setting was changed. - A clean local install hit a frozen-lockfile `patchedDependencies` configuration mismatch. The scoped checks reuse installed workspace dependencies. The final commit has passing hosted CI, including clean-install typecheck, tests and build, and Apex 5/5 with no unresolved findings. - A cold HTTP-suite module load reached the 30-second setup limit while other checks ran. All 54 HTTP cases pass on retry, and the full focused run passes 116 tests without changing assertions or time limits. - No staging deployment is claimed. This change classifies the connection-test result; it does not claim that a native Runner hello turn was performed. ## Risks - This status change is limited to a verified Cloud-managed sandbox ACPX connection and the exact deferred check code. The provider's successful receipt is required. - Runtime package preparation and verification still execute before agent work. A connection test does not replace those launch checks. - There is no schema, permission or credential change. Local, SSH and plugin task probes keep their existing verdicts. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6cc765bd81 |
feat(live): deliver recoverable transcripts to interested views (#17456)
## Thinking Path > - Paperclip should show live agent output without repeating presentation work for every view. > - Execution, acknowledgements and durable checkpoints must remain independent of browser interest. > - The server already owns authorization and redaction at delivery. > - A shared browser connection can declare which visible runs need full transcripts. > - Catch-up must handle subscription races, disconnects, terminal output and slow consumers. > - This PR delivers the backend and UI contract together without a database migration. ## Linked Issues or Issue Description **What existing behavior does this improve?** Visible transcript consumers receive recoverable output through one shared browser connection. **Current behavior** Independent views can create duplicate connections, presentation work and recurring history reads. A subscription race or filtered native history can lose the saved tail. **Proposed behavior** Negotiate run interests, subscribe before catch-up and retain independent canonical recovery cursors. Keep lightweight lifecycle and attention events available without full transcript delivery. PR 4 of #17097 → #17439 → #17455 → #17456. The runtime feature is independent of hot-upgrade recovery; the stack order is for review convenience. ## What Changed - Add additive WebSocket capability negotiation, replacement interests, revision acknowledgement and resynchronization notices. - Reference-count visible consumer interests on one company-scoped browser connection. Preserve older-client/server behavior. - Reuse canonical structural/current-user/registered-secret redaction for plugin run notifications through the same bounded delivery queue. Keep producer and uninterested streams free of recipient lookups. - Refresh application queries after the matching recovery acknowledgement, including one follow-up after an older in-flight read. Ordinary interest acknowledgements add no refresh. - Keep authorization and redaction at delivery. Browser interest cannot change harness execution, PRP ACKs, checkpoints, accounting or canonical logs. - Deduplicate by sequence and log identity. Catch up terminal output and saved tails without recurring reads during healthy negotiated streaming. - Bound delivery to 1,000 items and 1 MiB. Overflow requires resynchronization; persistent congestion closes the connection. - Preserve compact-history expansion from current master across paginated catch-up. Snapshot the original read window before React can defer its update. ## Verification On current source `f6710f0b3c`, Linux [job 122604880503](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880503) executes all five run-read permission cases successfully (2.574 seconds), including grant and revocation on the same connection. Its shard passes all 2,430 cases. Current [server-shard job 122604880416](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880416) has 1,513 passes, one failure and three skips: an unchanged nested-clone expiration/restore test exceeds its 15-second deadline. The preceding `5efc6c48bf` source has identical non-document bytes; [its corresponding job 122601359272](https://github.com/paperclipai/paperclip/actions/runs/40623017917/job/122601359272) passes all 21 nested-clone cases, including that case in 9.505 seconds. The exact timing cause is unproven. The single diagnostic [retry job 122609332597](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122609332597) passes all 21 nested-clone cases (61.316 seconds), including the formerly timed-out case in 7.003 seconds, and all 1,514 shard cases (three skips). The failure remains recorded; no timeout or assertion was changed. After the server squash merge and qualification retarget, all 50 focused server cases and all 91 focused UI cases pass again, with both typechecks and token gates. Runner vendor preparation rebuilds current master inputs before those tests. This source needs fresh CI and reviews; prior CI and browser receipts do not qualify it by inference. Current candidate `f6710f0b3c` now targets master `6987c2f20b`, after merged #17097, #17439 and #17455. Its diff contains only transcript delivery and its documentation. It adds no database migration. Retargeting changes only the evidence document; all production and test code is byte-identical to `5efc6c48bf`. On the eighth integration, all 50 focused server delivery cases and all 91 focused UI cases pass, with both typechecks and token gates. The following 35-case receipt belongs to the earlier delivery correction. Four low-queue authorization/decoration stall variants and the late-rejection case fail before correction. They now verify the five-second read deadline, old and negotiated clients, revisions that cannot reset the deadline, unaffected sibling connections, rejected late results, and timer cleanup. The first combined toast run has four loopback EPERM failures and 31 passes; its permitted repeat passes all 35. Earlier 28-case evidence excluded the toast file. The permission correction requires the current HTTP company-scope transcript permission, plus task access, for both native events and legacy logs. Prior Linux source `49a8783e9b`, job 122549913643, executes all five run-read cases, including grant/revocation on the same connection, successfully. That source placed this file in server shard 11; later shard assignments can differ and must be checked against their actual inventories. The older `405bedd189` receipt retains a separate scheduled-retry teardown race and inherited fixture collision; neither is waived. The new head must pass again in Linux CI. The seven extracted stream-recovery tests preserve their assertions. All 39 related legacy/native cases, UI typecheck and token gates pass after restoring the initially missed fixture helper. Earlier full workspace build/typecheck and Chrome journeys at `445a76563c` cover two views, company switches, reconnect, terminal catch-up, reload and zero recurring healthy transcript reads. Those browser receipts are historical source evidence and do not qualify this new head by inference. The [qualification record](https://github.com/paperclipai/paperclip/blob/f6710f0b3c84c6fdb2021df67206261b446dfc97/doc/plans/2026-10-22-hot-upgrade-qualification.md) retains exact earlier UI/history, source-boundary and cleanup failures with their fixes and limits. The workload-cost test preserves bounds and authorization callback counts for zero interests, one viewer, multiple browsers and a slow viewer. Its socket congestion is simulated. It measures neither SQL nor network costs. Current permission checks and the final hosted active-agent/viewer matrix remain unmeasured; no production savings are claimed. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and ordinary merge eligibility remain required. The earlier recipient corrections pass all 47 registry/plugin/delivery/WebSocket/toast cases and all 174 UI provider/recovery/interest cases, both typechecks and token gates on source `b202e59a25`. A real bus/host-service test registers a synthetic run secret through the production registry and reproduces its raw notification before the fix; native/legacy/final notifications now redact it, disposal prevents late worker notification. A later P1 review reproduces lost final notifications on settings-read and secret-resolution failure. Both cases fail before correction. Any delivery failure or subscription timeout now sends one content-free stream error and removes the plugin subscription. It never changes the agent execution or run status. Later events start no new redaction reads. The final integrated six-file server lane passes all 50 cases, with server typecheck and token gates. Existing browser delivery behavior is unchanged. Storage and secret-provider seams are scripted. Two real React Query/provider regressions first miss a run during resync; both now refresh after the valid replacement ACK, including pending-read and stale-scope cases. The prerequisite qualification batch passes all 118 cases and E2E typecheck after master integration. Fresh CI, security scan and reviews remain pending on this head. The contribution-rule review correction removes 104 explanatory comments introduced by this slice and keeps the ordering, deadline and plugin-delivery explanations in the delivery document. Existing comments and functional test/lint directives remain. All edited source files have identical parsed non-comment tokens before and after the change. All 50 server delivery cases and all 91 cases in six affected UI suites pass, along with token gates. The new head requires fresh CI, security and review. ## Risks A slow or disconnected viewer must resynchronize from authorized canonical history. It may see a display gap until catch-up completes. Company changes and revoked access must remove delivery interests. This slice adds no database migration and cannot change execution or recovery authority. Full transcript delivery evaluates current company-scope permission per delivery. Active plugin recipients also perform canonical redaction reads; viewer-load database costs on this final source remain unmeasured. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1028.0-canary.4 |