6231 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
DottaandPaperclip	0a99165e2b	fix(workspaces): fence physical aliases and preserve capture scope ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 13:39:17 -06:00
DottaandPaperclip	1b93ac9e07	test(workspaces): restore migrated trees from shared object storage ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 13:39:16 -06:00
DottaandPaperclip	8a0cb107ba	fix(workspaces): migrate overlapping workspace trees offline ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 13:39:16 -06:00
DottaandPaperclip	2035118526	fix(workspaces): preserve archived data during remote migration (#18826) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote workspace storage lets the web server run without a persistent volume. > - Existing deployments also contain archived and historical task workspaces. > - Retention can remove their folders while preserving encrypted restore points or Git reconstruction records. > - The importer rejected those records and only accepted the task's current workspace. > - This pull request imports verified saved data without reopening or rebinding tasks. > - The benefit is preservation of existing work during an explicit storage migration. ## Linked Issues or Issue Description Refs #18403 and #17472. **What happened?** An explicit remote-workspace import rejected archived records, historical task bindings, and removed folders with valid retained data. **Expected behavior** Import the recorded files or exact Git commit. Preserve archived state and the task's current binding. Refuse missing, expired, corrupt, or changed sources. **Steps to reproduce** Archive a task directory with a restore point, remove its folder, and run the remote-workspace import. Repeat with a Git restore point containing an unpushed commit and staged and unstaged edits. The previous importer reports an inactive workspace or missing source. ## What Changed - Preserve lifecycle state and current task binding while importing archived and historical records. Recheck the binding before publication and on resume. - Expose the task-directory owner's validated restore-point reader. Reuse it for ordinary restoration and migration. - Restore archived directories and Git data into private scratch space. Support saved Git recipes and exact legacy branch-deletion receipts. - Recover pruned unpushed Git commits from the encrypted bundle and its explicit prerequisites. - Keep independent snapshot download, restoration, source checks, writer guards, encryption, and atomic publication. - Document the supported import sources and the remaining tenant-enrollment gaps. ## Verification - Before production edits, two new regressions failed on archived status and historical task binding. - Final focused verification: 58 tests pass across remote-workspace migration and isolated task-directory restoration. - Real PostgreSQL, Git repositories, encrypted storage, and restored file bytes cover cold restoration after removing the old folder and source object, pruned unpushed history, index and worktree edits, corrupt/missing/expired sources, binding changes, and source-folder races. - A Git fixture initially lacked runtime ownership metadata. Correcting the fixture made the valid retained-source case pass; no production validation was weakened. - Full repository typecheck and build passed. Full `pnpm test:run` stopped in chat integration shard 2 after two failures: a retry-notice timeout and a subsequent fake-timer error. All four cases in that retry-notice group passed in a separate focused rerun without source changes; the complete suite has not passed locally. All exact-head CI jobs passed. The separate Paperclip Review app check is still queued; all existing review threads are resolved. Server typecheck passed after the review fixes. - Both review findings were reproduced: the real cleanup owner's verified-tip receipt was rejected, and a lost upload reply for a legacy Git source failed with `operation_content_changed`. The fixes accept the owner's scoped receipt metadata and rebuild reconstructed Git indexes from their staged trees. Both regressions now pass, including retained staged and unstaged edits. - Additional coverage restores from a fresh HTTP clone after deleting the original repository, imports a nested local checkout with encrypted settings, and rejects an unapproved nested source without publishing remote authority. - This PR does not claim a completed tenant enrollment or a new zero-downtime deployment proof. ## Risks Imports can fail when old receipts or repository objects are unavailable. They retain their migration journal and leave the original state intact. Operators must resolve pending writers and recovery before importing. Shared physical roots, nested worktrees, and unresolved sandbox recovery require separate migration work. This change does not remove those blockers or alter automatic retention. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. This session does not expose a more specific serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal task ID or instance-derived details - [x] I have run the focused tests locally and they pass; full typecheck/build passed and the full-suite failure is documented - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 13:36:22 -06:00
4 people	fef86e32e9	feat(execution-admission): add project concurrency policy and admission storage (#18746) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Projects group tasks. Agents run tasks as heartbeat runs. > - Today, workspace mode and shared-workspace flags decide whether tasks run at the same time. A project cannot set its own limit on concurrent task executions. > - A correct project limit must count every project that a task uses, including @-mentioned projects. It must also keep the count when a project is archived or deleted, and when a run is admitted more than one time. > - This change cannot go into one pull request with claims and settlement. Thus this pull request adds only the policy semantics and the durable accounting storage that later claims will use. > - This pull request adds a new `execution-admission` module, three tables and the deletion guards. No route writes a cap and no claim path enforces it. > - The benefit is a tested storage contract (constraints, idempotency, rollback, indexed counts, lifecycle preservation) before any behavior changes. ## Linked Issues or Issue Description Refs #2831, Refs #14333 (related agent-level and adapter-level limits; not duplicates). **Subsystem affected** Server execution admission (new module `server/src/modules/execution-admission/`) and the `packages/db` schema. **Problem or motivation** A project cannot limit how many of its task executions are active. The current shared-workspace flags mix resource safety with capacity. A future project limit needs storage that records each admission attempt and every project that the attempt charges. Run IDs cannot be the identity, because a run can be released before dispatch and admitted again. **Proposed solution** Add a separate policy table that keeps "never configured" apart from "explicit no limit". Add admission epochs with one charge row per distinct project. Settlement deletes the charge set in one transaction. Later pull requests integrate claims, settlement, waits and the public control. **Alternatives considered** Columns on `projects` would leak an unenforced cap through every project reader. A project column or a `uuid[]` on `heartbeat_runs` cannot enforce per-project uniqueness or company scope. `budget_reservations` has no project set. A counter would be a second source of truth. ## What Changed - Added `project_task_execution_policies`. No row means never configured. A null maximum means an explicit "no limit". A check accepts only positive maximums. A composite foreign key keeps the project and company together. The row is deleted with its project. - Added `execution_admissions`. Each row is one admission attempt of a run: `(run_id, attempt)` is unique, and a partial unique index allows one unsettled attempt per run. It keeps the context revision, the nullable primary project (provenance, no foreign key) and the settlement identity. A check sets `settlement_id` and `settled_at` together. - Added `execution_admission_project_charges`. It has one row per distinct admitted project, unlimited projects included. Composite foreign keys keep the admission and the project in one company. The project foreign key has no cascade, so a charge cannot disappear with its project. `(company_id, project_id)` is indexed for active counts. - Added migration `0426_absent_guardian.sql` (renumbered from `0423`, then `0425`, as master added `0423`–`0425`). The generated SQL was reordered so the unique indexes exist before the composite foreign keys that need them. Snapshots `0419` and `0420` were pruned to keep five snapshots. - Added the `execution-admission` module (domain → application → adapters): - Domain: policy patch parsing and application (omitted keeps, null clears, partial updates keep other fields, malformed values are rejected), canonical project-set preparation (primary ∪ references, deduplicated, ascending, bounded with an explicit `too_many_projects` result), and the settlement replay decision. - Application: internal `readTaskExecutionPolicy` and `updateTaskExecutionPolicy` commands. No route, tool, CLI or import calls them. - Adapter: the project-set lock (`FOR NO KEY UPDATE`, ascending, one statement), the policy store, and the storage transitions `recordAdmission`, `settleAdmission` and `countActiveProjectCharges`. These transitions are not exported from the module entry point. Claims (later) must add the capacity check and the lifecycle evidence. - Project deletion returns `409 project_execution_admission_pending` while the project has a charge. Agent deletion returns `409 agent_execution_admission_pending` while its runs have an unsettled admission, and removes settled history. Company deletion calls a new module participant before it deletes runs and projects. - Documented the tables, the lifecycle rules and the proposed global lock order in `doc/DATABASE.md`. Added module ownership to `server/src/modules/README.md`. ## Verification - `cd server && npx vitest run src/modules/execution-admission` → 2 files, 26 tests pass (the Greptile finding on an empty primary ID added an assertion to an existing test). The adapter tests use embedded PostgreSQL and cover: - policy semantics, the maximum check and the company foreign key; - charges for every distinct project, a primaryless set and an empty set (no rows); - replay of one admission ID, conflict on a changed record, refusal of a second live attempt and the partial unique index; - settlement of the whole set once, replay, a different settlement, then attempt 2 for the same run; - company isolation of projects, runs and admission IDs, the duplicate-charge key, and rollback of an aborted transaction; - active counts per project, and an `EXPLAIN` that uses `execution_admission_project_charges_project_idx`; - archive keeps charges, `projectService.remove` returns 409, a direct project delete fails on the foreign key, and delete succeeds after settlement; - `deleteAgentDependencies` returns 409 while unsettled, and `deleteCompany` removes only that company's admissions. - `cd packages/db && npx vitest run src/execution-admission-migration.test.ts` → pass. It removes the new tables and receipt, seeds a legacy project and run, applies pending migrations, and compares the existing rows. - `pnpm --filter @paperclipai/db check:migrations` → pass. Five snapshots remain (`0421`, `0422`, `0424`, `0425`, `0426`; `0423` is SQL-only on master). - `node scripts/check-module-boundaries.mjs` → pass. - `packages/db` `tsc --noEmit` → pass. A scoped `tsc` of the module entry points and domain tests → pass. - Existing tests for the changed deletion services → 6 files, 161 tests pass (`resource-lifecycle-events`, `prompt-injection-prevention`, `agent-lifecycle/lifecycle`, `case-deletion`, `project-workspace-root-provenance`, `companies-service`). - Not run locally: the full `server` typecheck. `tsc` and `tsgo` were stopped by the out-of-memory killer on the 7 GB machine (exit 137, also single-threaded). The full `pnpm -r typecheck`, `pnpm test:run` and `pnpm build` were not run locally; CI ran them on head `520b69d553`: all gates pass, including Typecheck, Build and the 12 server test shards. After merging master (`1c9ca5829f`) into head `c088729361`, the migration check, migration upgrade test, module tests, module-boundary check and `packages/db` typecheck were rerun locally and pass; CI is rerunning the full suite. ## Risks - Migration risk is low. It creates three empty tables and indexes. It does not change existing tables or rows. - Behavior changes only when admission rows exist. Nothing writes them yet, so project, agent and company deletion behave as before. - Later claim work must take the project-set lock in the documented order. A claim that takes project locks after issue locks could deadlock with project deletion. - Settlement deletes charge rows, so a settled admission keeps its primary project and context revision but not its full project set. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, embedded PostgreSQL tests). Default context window and reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: nickyleach <331803+nickyleach@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 11:35:28 -08:00
DottaandPaperclip	d8a9b3eb0c	fix(ui): show task identifiers in activity toasts (#18802) ... Project the company-scoped identifier for authorized task events and use it in toast titles, fallback bodies, and links. Retain authorized delivery when the optional identifier lookup fails. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 13:28:15 -06:00
DottaandPaperclip	fb6554ac62	fix(native): settle source wake with terminal projection (#18804) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each agent wake request can bind to one heartbeat run. > - Native recovery can finish and project a run after its live executor has stopped. > - The legacy completion callback settles the wake, but native terminal projection did not. > - A succeeded run could therefore leave its exact wake claimed with no finish time. > - This pull request settles that claim in each native terminal projection transaction and reuses durable projection recovery for old committed rows. > - The benefit is consistent run and wake receipts without a provider replay or a new sweeper. ## Linked Issues or Issue Description Refs #13856 and #10109. These open PRs describe orphan claimed wakes and cleanup after terminal persistence. This change fixes the native projection owner and uses its existing durable worker. It does not modify those contributor branches. **What happened?** A native run can be accepted and committed, then reach a terminal state through recovery. Its source wake stays claimed because the live completion callback does not return to settle it. **Expected behavior** Terminal projection must settle only the same company's exact source wake bound to that agent and run. Completed, failed and cancelled outcomes must retain their meaning. Other live, queued, cancelled, unbound or newly rebound wake records must remain unchanged. **Steps to reproduce** Commit an accepted native result in the real database fixture. Leave its source wake claimed. Call native finalization with run-status projection enabled, without the live completion callback. Before this change, the run becomes succeeded and finished while the exact wake remains claimed. ## What Changed - Settle the exact source claim inside native finalization, committed recovery, and exhausted-finalization projection transactions. Fence it by wake ID, company, agent, run and current claimed state. - Lock the matching claim and register settlement notifications before the write. A failed transaction rolls back both run projection and wake settlement. - Let the existing durable projection worker discover a terminal run with an unfinished exact claim. A repaired committed row leaves the pending set. No new timer, sweep or table is added. - Add real database regressions for native recovery, all three terminal outcomes, old committed receipts, idempotence, scoped denial controls, notification ordering and rollback. Cover provider/workspace finalization exhaustion and denial when the caller does not own terminal projection. - Preserve the existing provider, accounting, workspace, permissions and result contracts. No API, schema, user command, access-map entry or documentation contract changes are needed. ## Verification - The new native coordinator/finalizer regression failed before the production change: the run was succeeded and committed, but the bound wake was still claimed. - The initial focused database run did not reach the assertion because the SDK was unbuilt. A second attempt hit sandbox loopback restrictions. A third exposed skipped package library hydration. These setup failures were corrected before the meaningful red was captured. - Real PostgreSQL projection and response suites passed 89 tests before the independent review correction. The reviewer requested notification intent registration before mutation. The correction adds a notification-order and rollback case. All 90 tests now pass across both suites. - Server typecheck passed after building the isolated checkout's Runner and SDK prerequisites. Fresh corrected-source server typecheck and production server build also pass. Full repository checks run in current-head CI. The earlier serialized-9 CI failure was the independently repaired inbox snapshot fixture in #18798 (expected nine reader calls, observed eight). A normal merge of the containing master supplies that fix; the native suites plus the repaired inbox suite now pass 96 tests together at merge head `456c92ef8d60c3fcbeaf66ff8010a7ebd3d1109d`. Server typecheck passes. That head was superseded by the reviewed retry-exhaustion correction below. The subsequent review found a third terminal write in retry exhaustion. Real database provider and workspace exhaustion regressions failed with claimed/null-finished wakes before the fix. The same fenced helper now runs inside that transaction. At final source head `9718d4c2ec6066accf549dc6fea795891f6b90ec`, all 100 focused tests, server typecheck and production server build pass, including projection-disabled denial and rollback of run/coordinator/wake on notification failure. - Final current-head CI passed: 54 success, 2 skipped, and the neutral Paperclip Review at 5/5. Greptile also reports 5/5. Required ci / verify and ci / e2e passed. All review threads are resolved. - No live task write, provider work, accounting repair, deployed replay or manual rollout was performed. ## Risks - A stale wake binding must never be settled. The update requires the exact wake ID, company, agent, run and claimed state. Tests cover unbound and newer-run bindings, foreign company and agent, queued and cancelled records, and retained live ownership. - Old claimed debt now participates in the existing projection predicate. It uses the source wake primary key and stops matching after settlement. Existing quiet-worker tests remain in the focused suite. - Wake settlement must share the projection transaction. Notification failure and outer rollback tests preserve both original rows. - Interruption, timeout, retained historical terminal-failure coordinators, and delayed response presentation keep their current owners. This change does not infer or replay provider work. ## Model Used OpenAI Codex, `gpt-6.1-sol`, Extra High reasoning, with source inspection, code execution and regression testing. Context-window capacity is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked existing public issues or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation where needed; existing contracts are restored - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 13:17:16 -06:00
DottaandPaperclip	e0c6992426	fix(heartbeat): prevent saved input starvation during queue reloads (#18819) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Accepted messages and task Retry requests use durable wake receipts. > - The recovery worker loads receipts in pages of 100 and checks batches of 25. > - Company activity can reload those pages before the pending batch drains. > - Reloads reset ready receipts' dispatch times and can repeatedly favor receipts already checked. > - This pull request preserves the priority of unchecked ready receipts so saved input can reach normal admission. ## Linked Issues or Issue Description **What happened?** Saved input can remain deferred after the preceding run finishes. Repeated company eligibility changes reset the dispatch times of pending receipts during paginated reconstruction. A PostgreSQL regression checks only 75 of 178 receipts on the original code despite twelve eligibility changes. **Expected behavior** Each ready receipt must reach its admission check. Existing wait deadlines, retry backoffs, authorization, and execution gates must still apply. **Steps to reproduce** 1. Seed more than one page of deferred receipts. 2. Keep those receipts pending after their checks. 3. Send company eligibility notifications between scheduler passes. 4. Check that every receipt is reconsidered, including receipts later in the backlog. **Paperclip version or commit** Reproduced on `367f29478d`. **Deployment mode** Built from source with PostgreSQL. The defect occurs before provider execution. Related: #10195 reports a similar starvation symptom in a different promotion loop. #18621 added saved-input Retry after a successful run. This change fixes fairness in the deferred-receipt worker. ## What Changed - Add an optional entry merge callback to scoped deadline reconstruction. - Use it in the deferred-receipt worker to retain unread ready receipts' priority while refreshing their state. - Place receipts whose wait has expired into the ready queue during reconstruction. - Keep real wait, claim, and dispatch deadlines authoritative. - Add a paginated starvation regression, real process dispatch coverage, and GitHub lock-collision recovery coverage. - Document the recovery fairness rule. ## Verification - The PostgreSQL starvation regression checks only 75 of 178 receipts before the fix. It checks all 178 afterward, both with ordinary receipts and with expired execution waits. - Seven focused scheduler cases pass, including moved interrupt grace, moved native error deadline, fresh claim/wait deadlines, and unchanged-gate grace. - GitHub endpoint lock-collision recovery, retained board authority, HTTP Retry, and five current admission-gate cases pass. - Real process acceptance passes on the final code: task Retry crosses the repeatedly invalidated backlog, reconstructs its persisted queued run after a worker restart, saves `receipt-proof` in run output, and has one successor for the Retry receipt. The test holds executor capacity while advancing queue time, then starts fresh workers at PostgreSQL time before releasing a real process lease. The defect is in admission scheduling before provider execution; this acceptance uses PostgreSQL and a local process adapter. - The full workspace build and typecheck (`node scripts/typecheck-low-memory.mjs`) pass locally. The affected worker suites pass 52 of 53 cases together; the existing native multi-company batch case times out in that combined run and passes in isolation without code changes. All new regression and acceptance cases pass. - Local `pnpm test:run` stops in its first chat shard on three 15-second timeouts (Teams long native HTTP picture delivery, direct final-response materialization, and contended inbound notices). All three pass in isolation without code changes. Later full-suite groups did not run locally after that shard failure. - [The full GitHub Actions matrix passes](https://github.com/paperclipai/paperclip/actions/runs/41697489374) on `ac91df2cc6db6c77529d923177b375b8c92efd45`, including all general-test, serialized-server, browser, build, and typecheck jobs. Server shard 8 initially failed an upstream process-adoption test and passed its one rerun without code changes. - [Greptile is 5/5](https://github.com/paperclipai/paperclip/pull/18819#issuecomment-6612091535) on the same current head, with no actionable findings or unresolved review threads. The separate Paperclip Review check is neutral; its previous request was bound to an older commit. Focused command: ```sh pnpm exec vitest run server/src/__tests__/deferred-execution-receipt-work.test.ts server/src/__tests__/scoped-deadline-work.test.ts ``` ## Risks - Ready receipt order changes during repeated reconstruction. Admission still reads current durable state. - A ready receipt must not inherit an obsolete priority after it acquires a future deadline. The merge applies only when both pending and loaded entries are immediate. Deadline safety tests cover that boundary. - The merge callback is optional. Other scoped workers retain their current reconstruction behavior. ## Model Used OpenAI `gpt-6.1-sol` in Codex, max reasoning effort, with tool use and local code execution. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused regressions and acceptance; full-run timing limitations documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 13:17:00 -06:00
Nicky LeachandClaude Opus 5.5	8ab89f41e4	fix(build): typecheck test projects in 4 GiB (#18738) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Contributors and agents typecheck this repository, often on machines with limited memory > - paperclipai/paperclip#18737 made `pnpm -r typecheck` fit in 4 GiB because the server now checks in layers > - The test projects outside `pnpm -r` still import server source through relative paths, so each one checks most of the server as one program > - These projects peaked between 6.9 GiB and 14.9 GiB, so they fail on a 4 GiB machine > - This pull request makes these projects read the server layers as declarations, and adds a CI step that keeps them that way > - The benefit is that every TypeScript project in the repository can now be checked in 4 GiB, and a later change cannot silently undo this ## Linked Issues or Issue Description Refs paperclipai/paperclip#18737, which is merged. This pull request builds on the server layers that it added. No issue exists for this gap. **What happened?** `tsc -p tests/runner-e2e/tsconfig.json` peaked at 14.9 GiB at default settings. `tests/speko-native` peaked at 12.2 GiB, `server/src/__tests__/helpers/maintenance` at 9.6 GiB, and `tests/lifecycle-baseline` at 6.9 GiB. On one thread, `tests/runner-e2e` still peaked at 7.7 GiB. **Expected behavior** Each project passes with 4 GiB of memory. **Steps to reproduce** Run `tsc -p tests/runner-e2e/tsconfig.json` in a Linux container with `--memory=4g --memory-swap=4g`. The kernel stops the process. ## What Changed - Added `tsconfig.layers.json` to the four projects. Each file extends the project's `tsconfig.json` and references the generated server declarations. - Added `scripts/typecheck-test-projects.mjs`. It builds the server declarations and then checks the named projects, or all four. It takes `--low-memory`. - Changed `test:e2e:runner:typecheck` and `test:lifecycle-baseline:typecheck` to call that script. `docker-runner-check.yml` calls the first script by name, so that workflow needs no edit. - Added `--declarations-only` to `server/scripts/typecheck-layers.mjs`. The script now also generates `tsconfig.declarations.json`, so hand-written configs do not name individual layers. - Exported `runPnpm` from `scripts/typecheck-low-memory.mjs`, so the new script can use it. - Added a step to the `Typecheck + Release Registry` job. It runs straight after the server typecheck and reuses its declarations through `--reuse-declarations`. - The script now fails if a project loads server source, even when the types pass. Test support files, declaration files and JSON files do not count. This check does not enforce a memory limit. - Added a unit test that fails if a new project under `tests/` is neither in `PROJECTS` nor named as a single program. `tests/runner-acceptance` is the one named project. - Added `server/src/types/express.d.ts` to `tests/lifecycle-baseline/tsconfig.json`. That plain config failed alone with 11 TS2339 errors, and it now passes. - Exported `isTestFile` from `server/scripts/typecheck-layers.mjs`, so the source check uses the same rule as the layers. - Turned off `declaration` in the layered configs. These projects emit nothing, and declaration checks gave a false TS2883 error when `tests/runner-e2e` read the server as declarations. - Added tests, one of which runs the real compiler. Updated `doc/DEVELOPING.md` and `tests/lifecycle-baseline/README.md`. The references are not in each project's `tsconfig.json` because Vitest reads that file. A test showed that Vitest fails when a referenced config does not exist yet. The `tsconfig.json` files do not change, so editors and Vitest see the same configs as before. No new `package.json` scripts were added, as `.greptile/rules.md` requires. ## Verification I tested commit `9763880fab`, which is based on `1c9ca5829f`. I copied the commit into a named Docker volume and ran a fresh install. Each row ran in its own Linux arm64 container with `--memory=4g --memory-swap=4g` and 8 CPUs. The first row started with no build output. All rows used `--low-memory`. CI does not set a memory limit. I used the limit only for this measurement. | Project | Errors | Peak (cgroup) | Max process RSS | OOM kills | Time | | --- | --- | --- | --- | --- | --- | | `tests/runner-e2e` | 0 | 3.34 GiB | 3.06 GiB | 0 | 57 s | | `tests/speko-native` | 0 | 3.60 GiB | 3.27 GiB | 0 | 51 s | | `tests/lifecycle-baseline` | 0 | 3.43 GiB | 3.15 GiB | 0 | 50 s | | `server/src/__tests__/helpers/maintenance` | 0 | 3.39 GiB | 3.13 GiB | 0 | 52 s | `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm test:e2e:runner:typecheck` and `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm test:lifecycle-baseline:typecheck` also passed in the same limit, with peaks of 3.55 GiB and 3.52 GiB. I also ran the CI sequence in one container with the same limit: `pnpm --filter @paperclipai/server typecheck`, then `node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations`. Both passed, with a peak of 3.46 GiB. The new step took 16 s. The server layer build sets the peak in each row. Against the declarations, the projects peaked between 0.9 GiB and 2.2 GiB at default settings on macOS. Server source files that each project loads, without test support, declaration or JSON files: | Project | Plain `tsconfig.json` | `tsconfig.layers.json` | | --- | --- | --- | | `tests/runner-e2e` | 1,345 | 0 | | `tests/speko-native` | 1,647 | 0 | | `tests/lifecycle-baseline` | 1,343 | 0 | | `server/src/__tests__/helpers/maintenance` | 1,716 | 0 | I counted these on `3f0f31b2f9`. The source check in the script confirmed 0 for all four projects on this commit. **Limits of this proof:** - Without `--low-memory`, the 4 GiB run does not pass. The kernel stopped the layer build for `tests/runner-e2e` on `3f0f31b2f9`, and the script exited with status 1. I did not repeat this run on the final commit. The script does not turn on low-memory mode by itself. - The Linux runs used arm64, not x64. - The new CI step has not run in CI. `pr-trusted.yml` runs from `@master`, so the step first runs after this pull request merges. Check its first run. **Other checks on this commit:** - `server/scripts/typecheck-layers.test.mjs`: 38 tests pass. One test uses the real compiler to show that the source check tells a layered config from a config with no references. - With the references removed from `tests/speko-native/tsconfig.layers.json`, the script exits with status 1 and reports 1,644 server source files. - With the declarations removed, `--reuse-declarations` exits with status 1 and reports TS6053. - A type error in a test project makes the script exit with status 1. - `tests/lifecycle-baseline/tsconfig.json` passes with 0 errors. - `tests/lifecycle-baseline` Vitest suite: 533 tests pass. - Workflow tests that read `pr-trusted.yml` and CI routing: 114 tests pass. **Earlier checks:** - `server/src/__tests__/server-package-build-script.test.ts` passed 10 tests on `3f0f31b2f9`. This test failed in `ci / General tests (server (9/12))` on an older head. It tested a build script split that the final #18737 removed. - A changed signature in `server/src/services/run-liveness.ts` failed `tests/lifecycle-baseline/authority.test.ts` through the rebuilt declarations, before the rebases. When I restored the signature, the check passed. **Not run locally:** - `pnpm test:run` and `pnpm build`. CI covers these. **Failures that this pull request does not cause:** `pnpm test:e2e:runner:unit` has 13 failures on this commit, in `cursor-cloud-replay-fault`, `first-task`, `operating-receipts`, `pi-controls-evidence` and `workflow-security`. The same 13 tests fail on `master` at `6ef37398c8`. A comparison of the test names shows no differences. PR CI does not run this suite. ## Risks - Low risk to runtime behavior. No shipped code changes. - **Slower check:** `pnpm test:e2e:runner:typecheck` now builds the server layers on each run. Before the rebase, it took 30 s on macOS at default settings, about the same as the old single program for that project. On this commit it took 57 s in low-memory mode in the 4 GiB container. The new CI step reuses the declarations, so it took 16 s. - **Narrower check:** Server source is now checked only by the server's own layers, not again under each project's compiler options. The projects also skip declaration-portability checks. These projects emit nothing, so those checks had no output to protect. - **Unchanged path:** `tests/runner-e2e/native-completion-checks.mjs` still runs `tsc -p tests/runner-e2e/tsconfig.json` as one program. It needs about 15 GiB at default settings. I did not change it because that file and its inputs are part of a fingerprinted source contract, and no workflow calls it. - **Layer growth:** The new checks do not measure memory. If the layers or the import cycle grow, the checks still pass and the peak rises. This is intentional. Crossing 4 GiB does not fail a build. - **Project list:** The CI step checks only the projects in `PROJECTS`. The unit test covers new projects under `tests/`, but not a new project in another directory. - **Reused declarations:** `--reuse-declarations` trusts the declarations on disk. CI uses it straight after the server typecheck on the same commit. If that typecheck fails, the job stops first. Stale declarations can pass locally, so `doc/DEVELOPING.md` warns against this. - **Job dependency:** A type error in a test project now fails `Typecheck + Release Registry`, and so `ci / verify`. - **Not covered:** The new step runs no tests. `node scripts/typecheck-low-memory.mjs` and the Docker image build also have no new guard. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Provider: Anthropic. Model: Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-03 11:09:12 -08:00
DottaandPaperclip	4a38ff9c92	fix(ci): make Cloud readiness reruns and fixtures reliable (#18823) ... ## Thinking Path > - Paperclip manages agent work and preserves task state. > - Cloud readiness verifies an exact source revision before release. > - Its workspace tests must wait for execution and database cleanup to finish. > - Two attempts hit the same ten-second polling limit and read a running row. > - The second attempt also lost AWS runners and exceeded a fixture cleanup limit. > - This pull request waits for owned execution, bounds client shutdown, and gives verification reruns an independent runner pool. > - Assertions, source identity and required verification gates stay in place. ## Linked Issues or Issue Description **What happened?** [Cloud readiness run 41684102136](https://github.com/paperclipai/paperclip/actions/runs/41684102136) failed twice on master `367f29478d14003cc5c0f161b7ce8f3164993634`. Attempt 1 failed `heartbeat-workspace-busy.test.ts:473`: expected `succeeded`, received `running`. Attempt 2 failed the same helper in the inherited-workspace resume case at line 744. The helper polls for ten seconds, then returns the active row as if the wait finished. Attempt 2 also exceeded the 30-second `workspace-preparation.test.ts` teardown limit. Its client close had no deadline, and database plus directory cleanup shared the default hook budget despite the fixture's existing 90-second startup budget. Other attempt-2 failures show runner shutdown or lost communication. Several shutdown signals arrived at 18:05:17 UTC. Read-only AWS inspection confirms that all eight infrastructure-failed instances carry `runs-on-workflow-job-interrupted=true`. Five have explicit `Server.SpotInstanceTermination` reasons, including the server-1 runner that lost communication. Three were terminated through the fleet's shutdown path. CloudTrail records the Rust runner's interruption tag at 18:03:37 UTC and its termination by the RunsOn fleet role at 18:05:31 UTC, despite EC2 showing a user-initiated termination. Repeating the entire workflow reused the same Spot fleet. **Expected behavior** Wait for owned heartbeat execution before reading final state or scheduled retries. Bound database-client shutdown and allow the fixture's cleanup cost class to finish. Rerun verification on GitHub-hosted runners after an interrupted fleet attempt. Preserve all checks and report prior failures. **Steps to reproduce** 1. Delay the test adapter beyond the helper's ten-second polling window. 2. Invoke the ordinary local shared-workspace run or resume the deferred sandbox run. 3. Observe the old helper return `running` before the execution completes. 4. Evaluate the old runner expressions with `github.run_attempt = 2`; they still choose the interrupted AWS pool. **Paperclip version or commit** Base: `367f29478d14003cc5c0f161b7ce8f3164993634`. **Deployment mode** GitHub Actions Cloud readiness, with disposable PostgreSQL fixtures. Related: #11293 fixed the earlier gap between cancellation and retry-row insertion. This change waits for the complete execution instead. Duplicate searches covered the workspace test, retention fixture and runner shutdown reports. ## What Changed - Replace both workspace-test polling helpers with the existing heartbeat completion barrier. Keep final status, retry and coordination assertions. - Bound workspace-preparation client shutdown to one second. Apply the existing embedded-PostgreSQL fixture budget to its teardown. - Use AWS only for the first exact-source attempt. Route later verification, chaos and cloud-migrator planning/build attempts to GitHub-hosted runners. The publisher keeps its existing routing. The final branch also incorporates upstream #18821's CODEOWNER routing fix. - Add runner-expression coverage for numeric, string, missing and later attempt values. Document retry routing. ## Verification - The original workflow's failed assertions and interrupted jobs remain linked above. A local two-suite run also passed all 157 assertions and failed only the original 30-second teardown hook. - All 341 workflow contracts pass. After the master synchronization, all 347 routing and CI-planner controls pass. Actionlint passes for the three changed workflows. - A matched delayed-adapter comparison uses disposable PostgreSQL 17 databases, the same 11-second adapter delay, and the same 90-second experimental outer budget. The old helper fails with `running`; the completion barrier passes. Production test deadlines stay unchanged. - Earlier experiments hit a 30-second experimental outer limit and the macOS host's shared-memory limit during native database startup. The completed comparison uses task-owned databases on an existing test server. Those earlier attempts are not counted as passes. - All 31 workspace-busy cases pass with normal test deadlines. The 126-case workspace fixture also passes with its 90-second teardown budget. - Final-head cleanup measurements: main client close 7 ms, owned database cleanup 20,938 ms, and workspace-home removal 826 ms. A measured fixture held 124,494 PostgreSQL files (208 MiB) after repeated table resets. The measured database cleanup alone consumes most of the old 30-second hook budget. The separate baseline reproduced a timeout at that hook; the fixture now uses its established 90-second cost class. - Repository typecheck and build pass on synchronized head `2aceb3abd144a569efddabed3b7e0550ea23ceb2`. - All 157 focused cases pass on the synchronized branch, including complete PostgreSQL 18 fixture shutdown. [Full hosted CI](https://github.com/paperclipai/paperclip/actions/runs/41697804471) passes on the final head, including the required verification and E2E aggregates, all server/chat/workspace groups, Runner lanes and all eight browser shards. The separate GitHub quality-review workflow also passes. All 60 reported checks are terminal: 54 pass, three are informational neutral results, and three are expected skips. There are no failures or pending checks. The complete local `pnpm test:run` command was not run; targeted fixtures already encountered this host's shared-memory limit. Hosted CI supplies the full Vitest and browser matrix. - Greptile and Paperclip Review both give synchronized head `2aceb3abd144a569efddabed3b7e0550ea23ceb2` 5/5. There are no open review threads. - Module-boundary and whitespace checks pass. ## Risks - Hosted reruns can have different cache hit rates and queue times. They execute the same source and verification commands. - The first attempt retains the existing AWS Spot fleet policy. AWS evidence establishes several interruption-triggered failures; later verification attempts use an independent pool. This does not change the fleet's external capacity policy. - The completion barrier waits for owned background execution. A hung execution still fails the test's outer deadline. - The larger teardown budget remains finite. PostgreSQL's bounded stop protocol and cleanup errors still fail the hook. - No production execution, database schema or provider contract changes. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code edits and test execution. The runtime does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing> canary/v2026.1103.0-canary.12	2026-11-03 13:08:28 -06:00
DottaandPaperclip	83b5f7d486	fix(ci): keep CODEOWNER PRs on AWS as master advances (#18821) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its PR checks use reserved AWS capacity for authorized maintainers. > - The routing gate waited for GitHub capacity and compared an old test merge with a newer base branch. > - Normal merges into master then sent trusted PR jobs to the GitHub queue. > - This change restores AWS admission and validates the immutable merge selected for the run. > - Current CODEOWNER and rerun identity checks still control PR execution. ## Linked Issues or Issue Description Related: #18376 moved the gate to GitHub-hosted capacity. #16042 introduced reserved AWS admission. **What happened?** In PR #18811, GitHub created test merge `608d862ff1650db108a3832a2c67f8795924f0e7` at 17:17 UTC. Master advanced at 17:25. The gate started at 17:30 and rejected the older merge parent because it did not contain the newer master tip. The authorized PR then selected GitHub-hosted runners. The gate itself also needed a GitHub-hosted slot. **Expected behavior** Authorized maintainers use reserved AWS admission and PR workers. Normal base advancement does not change runner placement. Current owner checks, PR head checks, run-attempt checks, and base-history validation remain required. **Steps to reproduce** 1. Start a PR run whose author and sender are current CI CODEOWNERS. 2. Advance master while its routing job waits. 3. Run the gate against the original event merge and the current base tip. 4. Before this change, the gate selects `ubuntu-latest`. After this change, it selects the public PR AWS fleet. **Paperclip version or commit** Reproduced on master `367f29478d14003cc5c0f161b7ce8f3164993634`. **Deployment mode** GitHub Actions with RunsOn Fleet on AWS. ## What Changed - Restore the reserved routing fleet for the synchronized maintainer admission roster. - Check the immutable event merge and its PR head. Accept base snapshots between the event base and the current base tip. - Allow GitHub to regenerate its live merge ref while the original run waits. - Retain stacked-base validation and all current CODEOWNER, repository, author, sender, and rerun-attempt checks. - Add advancement, regenerated-merge, unrelated-history, and admission-placement regressions, including a stacked parent that descends from the event base but diverges from the live base. - Align the developing and CI profiling guides with reserved AWS admission and current CODEOWNER checks before PR execution. ## Verification - `node --test .github/scripts/tests/ci-routing.test.mjs scripts/__tests__/e2e-shard.test.mjs`: 103 passed. - `node --test .github/scripts/tests/ci-profile.test.mjs .github/scripts/tests/ci-plan.test.mjs`: 31 passed. - `actionlint .github/workflows/ci-routing.yml`: passed. - `git diff --check`: passed. - Mutation check: removing the snapshot-to-live-base comparison in a temporary workflow copy makes the new side-branch regression fail, proving it protects that ancestry guard. - [Greptile: 5/5](https://github.com/paperclipai/paperclip/pull/18821#issuecomment-6612170970) with a successful check on `abd6a8ce347228219d8e5127f34f57151788830e`; both review threads are resolved. - A regression reproduction on the original workflow selected GitHub instead of AWS. The patched workflow passes that scenario. - A read-only execution of the patched routing step against the actual event for run [41682867664](https://github.com/paperclipai/paperclip/actions/runs/41682867664), using live GitHub metadata, selected `runs-on/fleet=paperclip-public-pr-x64/env=public-ci`. This did not allocate a runner or rerun CI. - Full repository typecheck, Vitest, and build were not run in this worktree, which has no installed dependencies. The change is limited to workflow routing, its tests, and documentation. Broader CI checks remain queued. ## Risks - A stale admission roster can allocate the five-minute metadata gate. The gate still checks current CODEOWNERS before it authorizes any PR code on AWS. - The routing runner group must authorize `ci-routing.yml@refs/heads/master`. The fleet already exists; live placement still needs confirmation after merge. - Rewritten base history, changed PR heads, and unavailable identity or ancestry evidence continue to deny AWS PR execution. - Existing runs retain their previously resolved workflow. New runs use this change after it reaches master. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, code execution, and live GitHub API tools. The runtime does not expose the exact serving model variant or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> native-ingestion-benchmark-2026-11-03-b0	2026-11-03 12:47:16 -06:00
DottaandPaperclip	ccc63ea458	fix(server): prove completed history before idle sleep (#18646) ... Reviewed and verified at 592bce69c6. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 12:37:25 -06:00
DottaandPaperclip	367f29478d	fix(recovery): preserve the bound remote writer before continuation (#18803) ... ## Thinking Path > - Paperclip manages agent work and preserves task files. > - A remote workspace records the exact run that owns its unsaved files. > - A later setup failure can become the latest task run without owning those files. > - Retry only inspected that latest run, so the original writer had no file-recovery owner. > - A released reusable source also failed the legacy retention rule. > - This change registers the bound writer, saves its files, and then admits the saved user request once. > - Existing stop, ownership, approval and execution-evidence checks remain effective. ## Linked Issues or Issue Description **What happened?** A native run prepared a remote workspace and failed before finalization. Its sandbox stopped with a dedicated stop receipt. The workspace stayed dirty and retained that run as its writer. A later setup failure became the latest task run. Retry inspected the newer run and left the original file obligation unregistered. New attempts failed at the writer fence. Registration alone was insufficient. After file recovery, continuation still required the Retry source to be one of the older recovery sources. A newer pre-provider failure did not satisfy that check. Workspace selection could also choose a fresh directory instead of the recovered files. **Expected behavior** Find the writer through the task's current workspace binding. Register only its exact prepared source. Preserve the released lease and its stop receipts. Save the files before continuation. Independently prove that a newer setup failure never dispatched work. Select the verified saved workspace and admit the Retry once. **Steps to reproduce** 1. Fail a native run with a prepared remote workspace and a released reusable sandbox. 2. Keep matching dedicated stop receipts and the workspace's dirty writer authority. 3. Record a newer pre-provider setup failure for the same task. 4. Request Retry. Before the fix, no file-maintenance record owns the actual writer. **Paperclip version or commit** Reproduced before production edits on master `f7ad1dc7ca`. The branch was rebased on master `adb49a44346849e751548b2fb640693dbfad4b6f`. **Deployment mode** Authenticated hosted execution with Paperclip Runner and remote workspace storage. Companion: #18798 (now merged upstream) fixes a pre-existing inbox test synchronization race. Its test-only commit `110aa88489cad926a451c972ad731ede469347c1` is cherry-picked here with attribution so this PR can run the complete CI matrix. Related: #17800 registers missing preservation for the selected source run. #18678 holds unfinished remote exports. #18718 repairs a prior warm-controller projection failure. #17916 addresses a separate legacy warm-copyback policy case. ## What Changed - Discover the failed writer through the company-scoped task/workspace binding during Retry and authenticated message admission. - Reuse the exact prepared-source validator for released remote workspaces. Registration changes neither the lease policy nor its stop receipts. Physical recovery retains its full allocation checks. - Check a newer failed preparation separately before allowing an older, repaired source to release continuation. Provider events, leases, results and execution ownership remain disqualifying evidence. - Select the bound saved workspace through the continuation's verified repair receipt. - Add PostgreSQL safety cases and an HTTP Retry workflow with real file capture and saved-copy reads. Document the execution contract. - Reuse #18798's completion barrier for the existing inbox cancellation test; preserve its eight-slot limit, rejection and coalescing assertions. ## Verification - RED before production edits: the exact stopped writer has no `workspaceRestoreRecovery` after Retry. Six unsafe-source controls pass on the baseline. - GREEN: the expanded Retry/message/preview selection passes 17 cases. All 627 tests in the five affected recovery suites pass on recovery commit `b572304f772d236b8a8bdf095e13e651d808f41c`, including Stop, ownership, saved messages and duplicate Retry. - The HTTP workflow registers maintenance, keeps execution held before save, reads back the saved file bytes, and admits exactly one continuation with `resume_preserved_workspace`. A registration-only repair first failed after save because it did not reconcile the newer Retry source. The incident-shaped pair passes with an independent active stranded-task action present. - Greptile identified the path where the newer setup failure already owns a reconciliation hold. Its new regression failed before the correction (saved files, zero queued continuations); the contradictory-dispatch control passed. All four HTTP variants pass after the correction, covering both hold shapes and both dispatch-evidence controls. - Repository typecheck and build passed. Server typecheck and build also passed after the recovery changes. Module-boundary and whitespace checks pass. - The full local Vitest command failed in the GitHub fast-response acknowledgement test (`delivery_unknown`: one response write instead of zero). That isolated case passes on this branch and unchanged master; the original full-run cause remains unestablished. A sibling run passed 199 test cases but timed out in workspace-suite database cleanup. Unchanged master also passed all 126 workspace cases and timed out in the same cleanup hook; all five acknowledgement outcomes passed in that baseline comparison. These local failures remain recorded rather than being reported as suite passes. - The first hosted matrix failed the pre-existing inbox cancellation assertion (expected nine computes, saw eight). The same assertion failed in [master CI](https://github.com/paperclipai/paperclip/actions/runs/41664339150/job/126015526840) and reproduced locally before applying #18798's existing repair. All six inbox HTTP/database cases pass after that repair. Final head `7f2fafcbace74cb1ad8ea68c8375a69b42ed30d9` adds only that attributed test change to the verified recovery code. - [Final hosted CI](https://github.com/paperclipai/paperclip/actions/runs/41677310377) passes on head `7f2fafcbace74cb1ad8ea68c8375a69b42ed30d9`, including `ci / verify`, `ci / e2e`, typecheck, build, every server/chat/workspace group, Runner checks and all eight browser shards. There are 54 successful check results, two informational neutral results and three skipped results, with no pending or failing results. - Greptile and the Paperclip reviewer both report 5/5 for that final head; no review threads remain open. The Paperclip review check is informational (`neutral`) and records its local-testing limitations. - The file workflow uses real PostgreSQL, subprocess file capture, encrypted saved objects and production recovery code. Provider resume/stop calls are simulated at the environment driver boundary. This does not prove a live Daytona or Codex recovery. No deployment or original-task retry is claimed. ## Risks - File-maintenance discovery must not authorize another executor. Existing writer, coordinator and physical allocation checks still own that decision. - A false bootstrap summary cannot prove that a newer run never executed. The new path also requires its immutable native-adapter claim, unresolved preparation state, absent coordinator/process/session/lease/result evidence, and no dispatch events. - Additional database reads run only during explicit task admission or Retry strategy selection. They are company/task scoped and add no polling loop. The saved-writer query returns one matching snapshot and omits prompt bodies. - No schema, credential, provider profile or deployment setting changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, code execution and test tools. The runtime does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 11:25:24 -06:00
DottaandPaperclip	1c9ca5829f	perf(ci): overlap Runner workloads and cache debug workspace builds (#18794) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Required CI checks protect its server and Runner contracts. > - Recent full runs waited on general server files and the final Runner lane. > - That Runner lane ran package tests before native server tests. > - PR jobs also rebuilt unchanged debug workspace binaries. > - This pull request overlaps those workloads and reuses exact trusted build inputs. > - The benefit is a shorter critical path with the same test coverage and AWS authorization. ## Linked Issues or Issue Description Refs #18544, #18773, #18774, #18798, #18799. **What existing behavior does this improve?** The wall time of required PR tests and the precision of affected-test selection. **Current behavior** General server and native files use two workers. The second Runner Vitest lane runs its package shard and then the native server group. Each Runner job compiles debug workspace units again. A Vite raw import also retains its consumer for unrelated source edits. **Proposed behavior** Use three isolated workers for general server files and four for native files. Reuse the two Runner slots for a complete package suite and a complete native server suite. Restore all debug binaries only on an exact input key. Record raw imports as explicit file inputs. ## What Changed - Separate the Runner package and native server workloads without adding CI jobs. Preserve the old command arguments for master-pinned workflows, local commands, and other shard counts. - Increase file workers while retaining fork isolation, serial cases, and the shared-database guard. - Add an exact debug workspace cache. Derive binary targets from Cargo. Include source, fixtures, generated inputs, manifests, compiler identity, flags, Cargo configuration, and checkout path in its key. Discard incomplete restores. Only verified master pushes can save it. - Keep older PR checkouts working when they lack the new cache helper. - Track Vite `?raw` imports as repository file inputs. Keep unknown loaders, untracked files, and opaque effects conservative. - Add cache and coverage guards and document the behavior. AWS runner authorization is unchanged. - Keep paired test benchmarking available when install hooks correctly disable materialized dependency reuse. The old benchmark asserted a cache hit in that safe fallback. - Wait for active dispatch work to finish before asserting idle reads. Analyze the two 16,400-row database fixtures before their queries so they have current planner statistics. Suppress only PostgreSQL notices inside the cleanup transaction to avoid logging every cascaded table; warnings/errors remain visible. Row counts, real queries, assertions, database reset, and timeouts remain unchanged. ## Verification - 196 affected CI tests passed, including routing, coverage, selection failure fallbacks, shared-database serialization, and existing/new cache guards. Final worker/cache adjustments also passed their focused checks. - `pnpm build` and `pnpm -r typecheck` passed locally. Full test coverage is being verified by the Linux CI profile; `pnpm test:run` was not repeated as a full serial local run. The paired CI benchmark passed. Both paired benchmarks passed, and full CI passed again after integrating newer master commits. The external Paperclip Review app is the only pending status. - Actual debug artifacts remained fresh after simulating checkout timestamps: Cargo completed in 0.82 seconds; the content key was unchanged. This is local reuse evidence, not a measured GitHub cache transfer. - Same real UI inventory for a retention-test-only change: 861 files; retained selection fell from 413 to 401. All 12 omitted files are raw-input consumers; edits to their inputs remain covered by regression tests. - The first local worker comparison is invalid. Development dependency links disappeared during its second run. The two-worker baseline also had two macOS path assertions in company-skills. Neither result is counted as a successful speedup. Dependencies were restored. A local repeat was interrupted before producing results. The completed Linux comparison below supplies the valid paired evidence. - [Paired Linux benchmark](https://github.com/paperclipai/paperclip/actions/runs/41671244016): baseline and candidate ran the same 113 general-server files and 1,777 cases on one host. Wall time fell from 381.29s to 346.89s (9.0%). Both exited successfully and file/case inventories matched exactly. The fixture command took 31.77s versus 28.65s. This is one sequential pair, excluding setup, queueing and the full CI critical path. - [Repeat paired Linux benchmark](https://github.com/paperclipai/paperclip/actions/runs/41675496862): the same 113 files and 1,974 cases took 485.06s in the baseline versus 368.54s in the candidate (24.0%). File and case inventories again matched exactly. The baseline includes newer master changes than the first pair, so these are two independent comparisons, not repeated timings of an identical tree. - Initial full AWS profile: package suite passed 4,891 tests (289 files, one skipped); native suite passed 172 tests (9 files, 12 pre-existing platform/feature skips). Native execution took 239.06 seconds; the older sample ran 165 native tests in 391.29 seconds. The package lane now takes 11m01 including setup and debug compilation. This is an indicative cross-run comparison, not a paired full-CI speedup claim. - The initial run failed two UI fixtures, one serialized inbox settlement assertion, and one dispatch idle assertion. The two UI failures reproduced locally. The fixes preserve authorization, admission, and zero-idle-read assertions. All four affected files passed locally: 48/48 tests. Token gates passed. All four also passed in the next AWS profile. That profile passed every other workload but timed out the 16,400-row keyed attention fixture at its existing 180s limit; this is accounted for by the explicit fixture-statistics change. Local focused comparisons pass both large cases before and after; their roughly 33s test execution is unchanged locally, so this is not claimed as an additional speed gain. A full local file run passed both large cases (7.57s / 18.05s) but hit the unchanged 30s teardown deadline on its first test while emitting thousands of cascade notices. Cleanup now suppresses those notices within its transaction; the complete file then passed 34/34 in 107.72s. The latest-head AWS run passed the previously failing shard: the keyed bulk case took 40.25s (previously exceeded 180s), the keyless case took 8.02s, and the complete attention file took 89.61s. The 16,400-row fixtures and original timeouts remain intact. - [Full CI profile before master integration](https://github.com/paperclipai/paperclip/actions/runs/41675497699) passed on commit `5858956d4ac2c99c911ad2447b63e1f8fc84e31e`, including all tests, build, typecheck and final verification. Greptile reviewed that exact commit at 5/5; the only actionable review thread is resolved. The separate Paperclip Review app is still queued. - [Final merged-head CI](https://github.com/paperclipai/paperclip/actions/runs/41678956312) passed on `499aa206a6e03bf8f3249692ea731367a000591d`: 48 successful jobs and one intentionally skipped file-selection job. Backup checkpoint qualification and legacy security status passed. Greptile reviewed this exact commit at 5/5. The PR is conflict-free; only the separate Paperclip Review app remains queued. - Integrated current master after #18798 and #18799 merged the inbox and UI fixture fixes separately. Preserved master's inbox settlement fix and verified its entire file (6/6 tests). Those three files are no longer part of this PR's diff. Removed the temporary `profile-ci` label after both successful benchmark pairs; the normal planner still schedules every job because the CI configuration changes are global inputs. - Second AWS profile: native tests took 229.97s with the same 172 passes / 12 skips; package tests took 427.44s with 4,891 passes / 10 skips. The complete package job took 10m44, including setup and 79s debug compilation. - The PR uses the trusted workflow from master. Candidate worker/layout code runs now; workflow-level debug restores begin after merge and the first successful master cache writer. ## Risks - Extra workers can expose load-sensitive tests. The final full CI profile passed, but one run is not a long-term flake-rate measurement. - The cache is an optimization only. Misses rebuild, and test commands still execute. A new Cargo workspace member requires a cache-contract review. - Compiler alternatives remain experiments. Required TypeScript checks are unchanged. Typechecker experiment: three sequential checks per tool on identical built projects, with TypeScript 7.0.2, Bun 1.4.4, and tsrs 0.20.1-ts7.1.0-beta. Median seconds (TypeScript / Bun / tsrs): shared 0.429 / 0.083 / 0.157; UI 2.554 / 0.390 / 0.740; server 9.949 / 1.194 / 1.782. TypeScript and Bun passed all three projects. All tools caught the same four deliberate assignment, property, nullability, and generic-constraint errors. tsrs reported one additional server TS2345 diagnostic in the run-dispatch adapter, so it did not pass the project comparison. These local compiler times exclude generation and build steps; they are not full CI savings. ## Model Used OpenAI GPT-6 Codex. Tool use and code execution assisted this change. The exact deployment identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 11:17:53 -06:00
DottaandPaperclip	8c2502503d	fix(github): preserve repository access during PR webhooks (#18805) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Agents need current repository access before they can start work. > - Managed GitHub access checks run while webhook delivery can update the same grant. > - Pull request events update webhook health but do not change repository permissions. > - The refresh guard treated those health updates as access changes and could exhaust its retry. > - This change keeps repository preparation available during normal pull request activity. ## Linked Issues or Issue Description **What happened?** A pull request webhook can make a managed GitHub repository refresh fail with `github_access_changed`. Repository discovery then omits the failed connection. Workspace admission can report that repository access is no longer available, even after the checkout succeeds. **Expected behavior** Pull request activity must preserve valid repository access. Installation changes and newer access snapshots must still stop a stale refresh. **Steps to reproduce** 1. Start a managed GitHub access refresh. 2. Process a pull request edit while the provider request is in flight. 3. Repeat on the bounded retry. The old guard rejects both checks because `lastWebhookAt` changes. ## What Changed - Use `accessRevision` and `lastAccessRefreshAt` as the existing access fences. Stop using webhook delivery time as an access fence. - Allow the existing commit merge to retain the latest webhook timestamp and health after a pull request event. - Exercise real webhook processing and PostgreSQL during refresh. Cover a pull request event, an access conflict followed by that event, and installation suspension. - Document the distinction between access changes and delivery health. ## Verification - Red: both pull request race cases failed on the old code with `github_access_changed`. - Green: all seven focused refresh cases pass. These include the existing concurrent access-change checks and the new suspension case. - Command: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts -t 'binds a managed GitHub identity'`. - Full affected group: 526 tests pass across the tool-access, repository, and webhook suites. - `pnpm -r typecheck` and `pnpm build`: pass. - Current-head CI: 54 checks pass. The duplicate Paperclip review is neutral. Two Storybook checks are skipped. Greptile and the completed Paperclip review both rate this commit 5/5 with no findings. There are no unresolved review threads. - The duplicate local `pnpm test:run` did not complete. Its first four chat shards passed. It was stopped after CI completed on this commit; the full CI test matrix passed. The complete affected local group passed separately. - Retained earlier failure: the first affected-group attempt passed 525 tests and had one socket hang-up in the profile attention feed test. That test passed alone without changes. The unchanged base tool-access suite also passed all 498 tests, and the complete fixed affected group then passed. The cause of the socket closure remains unproven. - Environment: source checkout on macOS arm64, Node 26.4.0, pnpm 9.15.4, disposable PostgreSQL. Provider responses and Cloud transport use controlled fixtures. The webhook handler, refresh logic, transactions, and saved grant reads are real. - Live acceptance remains unverified. The available CLI token cannot list GitHub App installations. The affected staging deployment still serves the old code. This PR does not deploy or recover the original task. ## Risks - Removing an access fence could admit stale permission data. The access revision and refresh timestamp checks remain. The real suspension webhook regression confirms that stale provider data cannot restore removed access. - This change adds no retry, timer, schema change, or credential forwarding. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime model ID and context window are not exposed in this session. Used reasoning, code editing, and terminal tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 11:15:49 -06:00
Nicky LeachandClaude Opus 5.5	2e37c73c05	Tell the agent about the launcher's memory limit (#18801) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The native Runner supervises the harness and tool processes for each run > - A launcher can put a memory limit on those processes > - When a process goes above the limit, the kernel kills it, and the agent sees only exit code 137 > - Any SIGKILL gives that code, so the agent cannot know the cause > - The agent can then run the same command again and fail again, and the people who read the run see no reason > - Only the launcher knows if a limit applies, so Runner must not assume one > - The kernel gives the limit and the kill count in cgroup files > - This pull request adds an optional `--task-memory-cgroup` flag that names the task's cgroup > - The benefit is that the agent knows the limit before it starts, the agent can confirm a kill, and the run shows each kill ## Linked Issues or Issue Description Related: #18730. That pull request lets the kernel kill large tool processes under Runner. This pull request tells the agent when that occurs because of a memory limit. **What happened?** A tool process on a VM with a 6 GiB task limit went above the limit. The kernel killed it, and Runner and the agent continued. The agent received exit code 137 and no other data. It ran the same type check in different forms, and the kernel killed it five times. **Expected behavior** The agent knows about the limit. It can confirm that the limit killed its process. It can then reduce the work or report a blocker. The run shows the kills. **Steps to reproduce** Start Runner under a memory limit. Start a tool process that goes above the limit. The agent and the run show only exit code 137. **Deployment mode** Native Runner on Linux, when the launcher gives the flag. ## What Changed - Add the optional `--task-memory-cgroup <path>` flag to `paperclip-runnerd`. Runner has no default value. - Runner reads the limit from `memory.max` on the parent of that cgroup. Runner reads the kill count from `oom_kill` in `memory.events` on that cgroup. - When a limit exists, add a note to the Codex and ACPX instructions at launch. The note gives the limit, the meaning of exit code 137, the paths of the files to read, and what to do after a kill. - Record a `provider.notice.recorded` warning when `oom_kill` increases. Runner watches `memory.events` for the kernel's change signal, so an idle Runner does not read it. Runner takes the starting count before it executes commands. It does not report kills from before it started. - Without the flag, or when the parent has no limit, the instructions do not change. Without the flag, Runner does not read the files and records no notice. - Add unit and integration tests. ## Verification Tests ran in `rust:1-bookworm` on Linux arm64: - `cargo fmt --all` - `cargo clippy -p paperclip-runner-core --bin paperclip-runnerd --lib` has no warnings in the new module. - `cargo test -p paperclip-runner-core --lib -- task_memory restores_settled_session_for_explicit_run_attachment_only`: The task memory tests pass. The second filter covers the note in the ACPX session configuration. - `cargo test -p paperclip-runner-core --test task_memory`: this test uses a separate process. It checks that no note and no monitor exist before the flag value is set. It then checks the note in the Codex `thread/start` and `thread/resume` requests. The tests use files in a temporary directory that have the format of the cgroup files. In a separate test on a real kernel, a kill above a limit on a parent cgroup increased `oom_kill` in the cgroup of the killed process. It did not increase `oom_kill` in the other child. On a real kernel, an unprivileged process that watched `memory.events` through a read-only mount received the change signal at the kill. I did not run the complete suite locally. CI runs it. I did not run this with a real launcher or on a real VM. The notice payload has the same fields as other Runner notices, but no test sends it through the control plane. ## Risks - The note adds text to each provider's instructions when the launcher gives the flag. If the text makes the instructions go above the provider limit, Runner does not add the note. - The task must be able to read the cgroup files. If it cannot, the note is absent or the agent cannot read the paths, and the run continues as it does now. - A kill while the run is not ready gives a notice later. A kill after Runner records 64 notices gives no notice. The agent can still read the file. - This pull request does not send a message into an active turn, because most ACPX profiles do not support this. The agent must read the file to confirm a kill. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass: targeted tests only, as listed above - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes: not applicable, because no document lists the `paperclip-runnerd` flags - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-03 09:02:19 -08:00
Nicky LeachandClaude	6ef37398c8	fix(build): fit typecheck and image build in 4 GiB (#18737) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors, agents, and image builds run `tsc` to check and compile the repository. > - The repository uses TypeScript 7, which is a native binary. Node heap flags do not limit it. > - The server is one large program, so one `tsc` process needs about 10 GiB. > - The operating system stops `tsc` under a 4 GiB memory limit. > - This pull request checks the server in layers, removes the server from the CLI program, and removes declarations and checks from the image emit. > - The benefit is that the full typecheck and the image `tsc` step now complete in 4 GiB. ## Linked Issues or Issue Description I found no duplicate pull request or issue. A search for `tsc`, memory, typecheck, and OOM found only runtime memory work, which is a different subject. **What happened?** `tsc` used too much memory, and the kernel stopped it with exit code 137 in resource-constrained environments. Measured peaks before this change: | Command | Peak | | --- | --- | | Server `tsc --noEmit` | About 10 GiB | | Server `tsc` emit, as in the Docker build | About 10.7 GiB | | CLI `tsc --noEmit` | About 7 GiB | | `pnpm -r typecheck`, whole process tree | About 10.2 GiB | **Expected behavior** The typecheck and the image `tsc` step complete under a 4 GiB memory limit. **Steps to reproduce** 1. Start a Linux container with `--memory=4g --memory-swap=4g --cpus=4`. 2. Install the dependencies on the base commit. 3. Run `pnpm -r typecheck`. The kernel stops the process with exit code 137. 4. Run `pnpm --filter @paperclipai/server build`. The `tsc` step needs more than 4 GiB. **Paperclip version or commit** Base commit `09162fe893`, with TypeScript 7.0.2. **Deployment mode** Not applicable. This issue applies to development, CI, and image builds. It does not change the server at runtime. ## What Changed - Add `server/scripts/typecheck-layers.mjs`. It reads the import graph on each run and runs one `tsc` process per layer. The layers are the files below the largest import cycle, the cycle, the files above the cycle, and the tests. Each layer reads the earlier layers as declarations. - Add `scripts/run-tsc.mjs`. It adds `--singleThreaded` when `PAPERCLIP_TYPECHECK_LOW_MEMORY=1` is set. The server layers and the UI typecheck use it. - Add `scripts/typecheck-low-memory.mjs`. It checks one package at a time in this mode. It has no `package.json` entry, as `.greptile/rules.md` requires. - Change the server `typecheck` script to build only the Runner TypeScript output. It does not build the Runner release binary now. `pnpm build` and `typecheck:build-gaps` still run the full Runner build and its generated-file checks. - Change the `tsc` step of the server `build` script to `scripts/run-tsc.mjs`. The `Dockerfile` sets `PAPERCLIP_TSC_EMIT_ONLY=1` for that build, so the image emits JavaScript without checks or declarations. Without this variable, `build` still does a full check for npm packages and CI. - Use a non-literal specifier for the `@paperclipai/server` import in `cli/src/commands/run.ts`. The import result already had no useful type, and the literal specifier added the full server to the CLI program. - Add a layered server typecheck step to the `pr-trusted` typecheck job, because `typecheck:build-gaps` does not run this script. - Update `AGENTS.md` and `doc/DEVELOPING.md`. ## Verification Results on Linux use a clean copy of this branch, a new `pnpm install`, Node 24, Rust 1.97.1, 4 CPUs, and a 4 GiB memory limit with no swap. Results before the merge of `master`: | Command | Result | Peak non-reclaimable memory | | --- | --- | --- | | `node scripts/typecheck-low-memory.mjs` | Exit 0, 0 OOM kills | 3,492 MiB | | `PAPERCLIP_TSC_EMIT_ONLY=1 pnpm --filter @paperclipai/server build` | Exit 0 | 3,656 MiB, from the Rust release build | | Server single-program `tsc --noEmit --singleThreaded` | Exit 137, 1 OOM kill | Reached the limit | Results on macOS on the merged head, with peak memory for the full process tree: | Command | Before | After | | --- | --- | --- | | `pnpm -r typecheck` | 10.2 GiB, exit 0 | 5.47 GiB, exit 0 | | `node scripts/typecheck-low-memory.mjs` | Not available | 3.81 GiB, exit 0 | | CLI typecheck | 7.06 GiB | 1.68 GiB, before the merge | Other checks: - Compiler tests: new tests run the real compiler on a small fixture. They prove that the layers emit declarations and pass, that a renamed export fails the layer that imports it, and that a changed type fails the test layer. - Declaration imports: a new compiler test proves that types from `.d.mts` and JSON imports stay correct in later layers. On the real server, a wrong use of `AgentFileManifest` in the cycle layer failed with `TS2353`. - Coverage: a test on the real server source asserts that each source file is in one layer and that no import goes to a later layer. The test layer contains the remaining files. `src/__tests__` stays out of the root list, as in `server/tsconfig.json`. - Error detection: I added one type error to one file in each of the four layers. The typecheck reported all four errors and exited with 1. - Cross-layer errors: I renamed an export in the lower layer. The cycle and upper layers reported the error and the typecheck exited with 1. I reverted these temporary edits. - Emit: full `tsc` and `--noCheck --declaration false --declarationMap false` gave byte-identical JavaScript and source maps for 2,249 files on the base commit. - Runtime: `scripts/server-boot-smoke.sh node server/dist-runtime/index.mjs` passed with a bundle from the emit-only build. - CLI: the esbuild bundle keeps the dynamic import, and `node --check` passed. - Tests on the merged head: `typecheck-layers`, `prepare-runner-vendor`, `cli-invocation-safety`, the CLI `runtime` tests, and all 503 `.github/scripts/tests` tests passed. Before the merge, `release-verify-workflow` and `runner-release-crate-cache` passed. - Full `pnpm --filter @paperclipai/server build` passed before the merge. Failures during this work: - The first low-memory run failed in the UI with `TS5023: Unknown compiler option '-b'`, because the flag came before `-b`. I corrected the argument order and added a test that runs build mode with the flag. - The first rename check did not rename the export, because BSD `sed` does not support `\b`. Its exit 0 was not evidence. I ran the check again with `perl`, and the result above is from that run. - The first version of the real-compiler fixture had a `TS2774` error in its own source. The test found this error, and I corrected the fixture. - CI failed `server-package-build-script.test.ts` on `1abc5e5a43`, because that test reads the steps of the server `build` script, and I had moved them to `build:prepare` and `build:assets`. My first search for readers of this script did not include `server/src`. I removed those scripts and `build:image`, restored one `build` script, and kept the test unchanged. - `GOMEMLIMIT` did not help enough, and I removed it from the plan. On these steps, `3GiB` saved at most about 300 MiB. With default checkers, the cycle layer was stopped with and without it, and the lower layer went from 7 seconds to 151 seconds. Not run or not proven: - I did not run the full `pnpm test:run`, a root `pnpm build`, or a full `docker build`. I expect CI to run the test and build gates. - I did not measure the Docker build steps other than the server build. - I did not run the 4 GiB checks on x86-64. - I did not measure the merged head in the 4 GiB container. The macOS result on the merged head is 3.81 GiB, and the merge added one source file to the lower layer. ## Risks - The margin is small. In the 4 GiB container, the cycle layer peaks about 600 MiB below the limit, and the Rust release build in the image build peaks about 440 MiB below it. Growth of the largest import cycle uses this margin first. - `node scripts/typecheck-low-memory.mjs` applies only to `pnpm -r` packages. These commands still check the server as one program and need more than 4 GiB: `pnpm build`, `test:e2e:runner:typecheck`, `test:lifecycle-baseline:typecheck`, and the `tests/speko-native` project. - A local flow that used the Runner binary from a previous typecheck must now run `pnpm --filter @paperclipai/paperclip-runner build`. - The image does not include server declarations now. I found no consumer in this repository. A downstream image that reads `server/dist/*.d.ts` must change. - An image build does not find type errors now. A typecheck in CI must find them before the image build. - The import scan uses a regular expression. A false edge can only make a layer larger. A missed edge causes a `TS6307` failure, so it cannot skip a file. - The new CI step adds about one minute to the typecheck job. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with reasoning, tool use, and code execution. The runtime does not show the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-11-03 08:53:35 -08:00
DottaandPaperclip	5f1b59ba2c	fix(db): preserve binary data in checkpoint backups (#18650) ... Reviewed and verified at 69792ccbcf. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 10:47:19 -06:00
DottaandPaperclip	e401ed006b	test(inbox): wait for snapshot completion before fresh reads (#18798) ... ## Thinking Path > - Paperclip helps people manage AI agents and inspect their work. > - Mine combines task and run data behind the issue-read admission limit. > - Its cancellation test aborts a request while the server still owns its snapshot computation. > - The test waited for seven separate list requests and assumed the snapshot had also finished. > - CI can return those list requests first, so the next snapshot read correctly shares the existing computation. > - This pull request completes a snapshot read before it checks that a fresh read starts new work. > - The regression keeps its admission-limit checks without depending on database read order. ## Linked Issues or Issue Description **What happened?** Full [CI run 41671193176](https://github.com/paperclipai/paperclip/actions/runs/41671193176/job/126041533414) failed `retains admission until aborted snapshot work settles (coalesced reader: false)`. It expected nine computation starts but saw eight. The aborted snapshot was still running after the seven list requests returned. A follow-up snapshot request could share that work. **Expected behavior** The test must retain the occupied slot while the aborted work runs. It must wait for snapshot completion before it expects a new computation. **Steps to reproduce** Run the full serialized CI shard containing `server/src/__tests__/inbox-mine-routes.test.ts`. The assertion depends on read order and can pass in isolation. The unchanged focused cancellation cases passed locally after the CI failure. The test does not await the aborted server computation, which explains the race. **Paperclip version or commit** CI failure: `1859d17b3ada7865c0ad4c5835eda23c32e33a48`, based on `f7ad1dc7cacedf7153a45e854ddfdd544aa87abd`. This branch now includes master at `adb49a4434`. **Deployment mode** Source checkout; real HTTP requests and disposable PostgreSQL. Refs #18790, which introduced the snapshot test. The user-reported [run 41665098924](https://github.com/paperclipai/paperclip/actions/runs/41665098924) also had three UI fixture failures and several runner shutdowns. All three UI failures reproduced locally. This PR first repaired them; concurrent #18799 merged the same repair while CI ran. The branch preserves that upstream repair, so the remaining diff is this inbox test. No duplicate inbox completion repair was found. ## What Changed - Complete a snapshot request as a barrier for any remaining aborted snapshot work. - Then assert that a fresh snapshot request is a cache miss and starts exactly one new computation. - Retain the eight-slot limit, rejected ninth request, and coalesced-reader assertions. ## Verification - The first full CI matrix exposed the eight-versus-nine assertion failure linked above. Both UI shards passed after the fixture correction. - All six real-database inbox route cases pass with the completion barrier. Server typecheck also passes after that repair. - Before the master merge, both affected UI suites and the account-scoped access reader passed 30 tests. The reader still rejects another account's response. Token gates pass. - On final merged head `f93d1e309a1c76bb6ded57f76efe08b876afa164`, the combined 36 focused tests, `pnpm -r typecheck`, and `pnpm build` all pass. - The local `pnpm test:run` completed four chat shards with 1,279 passing tests. It was stopped during shard five when master advanced. It is not reported as a full-suite pass. The final full CI matrix verifies the current head. - Full [CI run 41674890115](https://github.com/paperclipai/paperclip/actions/runs/41674890115) passed on final head `f93d1e309a1c76bb6ded57f76efe08b876afa164`: 48 jobs passed and one optional selection job was skipped. All PR checks are green with none pending. - Greptile and Paperclip Review are both 5/5 on that same head. Greptile added zero comments, and no review threads are open. ## Risks Low risk. The diff changes test synchronization only. It uses actual request completion and adds no timing delay. Production behavior, permissions, schemas, and API contracts do not change. No operator documentation change is needed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 10:46:13 -06:00
DottaandPaperclip	1423d745e8	feat(server): expose owned sleep blocker diagnostics (#18632) ... Reviewed and verified at 26e5a22b3c. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-03 10:45:10 -06:00
DottaandPaperclip	adb49a4434	test(ui): match access fixtures to signed-in accounts (#18799) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The board checks that access data belongs to the signed-in account. > - The startup changes in #18790 added this account check. > - Two existing test fixtures did not supply the account identity. > - Cloud readiness now rejects those fixtures before their intended assertions. > - This pull request adds the matching user ID to each fixture. > - The tests can then check invitations and pending shell settings with the same access boundary as the app. ## Linked Issues or Issue Description Refs #18790, #18770. Cloud readiness [run 41665172584](https://github.com/paperclipai/paperclip/actions/runs/41665172584) at `f7ad1dc7cacedf7153a45e854ddfdd544aa87abd` failed both UI shards. The private invitation case showed an access error. Both pending-shell cases made two session reads. All three failures reproduce unchanged in the focused local run. The access query compares the response user ID with the current session. These fixtures omitted that field. The mismatch correctly starts account-change handling and prevents the tests from reaching their intended state. ## What Changed - Give the invitation fixture the same user ID as its signed-in invitee. - Give the pending-shell fixture the same user ID as its signed-in user. - Keep all assertions, account-change rejection, and production behavior unchanged. ## Verification - Before the repair: the focused App and ShellConfigurationGate run reproduced all three CI failures; 24 other cases passed. - After the repair: all 76 cases pass across App, ShellConfigurationGate, CloudAccessGate, board-access-query, and CompanyContext. These include account mismatch and revoked-access coverage. - `pnpm check:token-gates` and `git diff --check` pass. - Full `pnpm build` and `pnpm -r typecheck` pass on `fa4ec98859d573e4197bb022cedc76f36fad1cf8`. Current-head Greptile reviewed this commit at 5/5 with no findings. CI is still running. Normal squash auto-merge is enabled for this exact head and will wait for the required checks. - The full local workspace test suite is not repeated for this two-line fixture repair. The earlier full-suite chat flakiness and cleanup timeout remain recorded in #18770. This PR does not claim that those failures are repaired. - No API, visual, or operator documentation changes are needed for these fixture values. ## Risks Low risk. The change only adds existing response fields to two test fixtures. It does not relax account validation or change test expectations. Cloud readiness must still pass on the resulting master revision before rollout. ## Model Used OpenAI GPT-6 Codex, with reasoning, tool use, and code execution. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1103.0-canary.11	2026-11-03 16:20:11 +00:00
DottaandPaperclip	f79264d253	perf(recovery): project current wait facts during action reconciliation (#18793) ... ## Thinking Path > - Paperclip manages work and execution for AI agents. > - Recovery actions keep unresolved work visible and preserve its owner. > - Action reconciliation checks whether a durable wait or another live path has appeared. > - Its current-wait check collected a complete retry fingerprint through seven queries. > - This caller used only the durable-wait flag and reason. > - This pull request reads those facts in one fresh statement and shares the existing reason policy. > - The local service uses less CPU. Fresh mutation guards and full retry fingerprints stay in place. ## Linked Issues or Issue Description **What existing behavior does this improve?** Active recovery action reconciliation. **Subsystem affected** server/ — orchestration and recovery. **Current behavior** Each admitted source action collects blockers, children, interactions, approvals, work products, runs and wakes. It sorts and hashes these records. This caller uses only the durable-wait flag and reason. **Proposed behavior** Read three current existence facts in one company-bound statement. Use the same reason policy as the complete fingerprint collector. Keep child checks, outside-action checks and mutation services. **Reason and benefit** The actual local PostgreSQL service uses less CPU across 8, 32 and 100-action synthetic cohorts. This removes unused work on mostly empty results. The live workload mix and fleet benefit remain unknown. **Breaking changes** No API or schema changes. Query facts now share one statement snapshot. Scalar issue fields still come from the source cohort read. **Additional context** No duplicate projection change was found in the public PR search. The roadmap has no overlapping core feature. This is a narrow performance change. ## What Changed - Add a current durable-wait reader and share reason selection with the full fingerprint collector. - Use the reader only in active action reconciliation. Keep the full collector for retry and child policy. - Add PostgreSQL regressions for reason priority, scope, board waits, native holds, healthy children and concurrent decisions. - Add a manual reproducer and local paired results. Each invocation uses a unique evidence directory and exclusive output files. ## Verification - Focused PostgreSQL regressions: 11 passed. - Separate actual-service proof: 7 passed. Four CPU tests were excluded. Results, records, semantic relation edges and notifications match the pinned baseline. Late query facts and a concurrent action decision are covered. - Retained primary local paired CPU medians: 8 actions 7.091 → 2.886 ms; 32 actions 21.922 → 8.374 ms; 100 actions 68.384 → 32.718 ms. All nine CPU and wall pairs improve at these sizes. The data file retains all 72 primary samples and ranges. - Warm application counts: 81 → 33 queries at 8 actions; 321 → 129 at 32; 1001 → 401 at 100. Rows rise from N to 2N because each EXISTS statement returns one row. - See `doc/benchmarks/2026-11-03-active-recovery-projection.md` and its JSON data for commands, provenance and limits. - `pnpm -r typecheck` and `pnpm build` passed on head `f2f8c4a755ac67679d78c6005f420a21a101c1ad`. - Shipped manual helper smoke with `--proof-only`: 7 passed, 4 timing tests skipped. All count-proof sample arrays are empty. Unique output names and cleanup of the three temporary source files were verified. - Exact-head CI completed with 52 successful checks, three intentional skips and a successful Snyk status. Greptile completed successfully at 5/5 with no inline findings. The `review` workflow passed commitperclip dependency and PR quality gates. No separate Paperclip Review check or verdict was published. The local `pnpm test:run` attempt passed all six chat shards (1,834 tests), then reported 62 Muse failures and one gateway failure in the general-server batch. The initial cases each reached the unchanged 15-second timeout. The Muse case was followed by fast failures. After the full batch breadth and complete CI coverage were established, the owned known-failed run was interrupted once with SIGINT. Remaining general-server, serialized and workspace scopes were not completed by that local invocation. Its interrupted tool exit code is not a pass. Each initial timeout case then passed in one fresh isolated control. A single fresh full Muse file passed all 256 tests. Source review found no changed projection path in either initial case. The original timeout causes remain unresolved; later passes and green CI do not erase them. The first reserved CPU run lost its raw files after a harness path error caused an unintended repeat to overwrite them. Its summary and log remain historical evidence. The separately retained repeat is the primary inspectable evidence above. It had no renewed quiet reservation. The runs are not pooled. The corrected untimed proof and public helper prevent output replacement. No third CPU run was made. ## Risks - One statement observes its query facts together. This does not reproduce every race history of seven concurrent statements. - Existing action update predicates, issue mutation locks, board ownership, native reconciliation holds and company guards remain in place. - The evidence is a local source service benchmark on macOS and real PostgreSQL. It is not deployed CPU or request latency acceptance. The census was startup-adjacent. Conditional scale suggests only a modest fleet return. - The original local batch timeout causes remain unexplained. No hosted, Sentry or OTLP improvement is claimed. No merge or deployment is requested. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, tool use and code execution. The host did not expose the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused and isolated controls; the broad local failure/interruption is described above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 10:13:03 -06:00
DottaandPaperclip	f7ad1dc7ca	fix(workspaces): make attached project repositories writable (#18770) ... ## Thinking Path > - Paperclip lets people assign work to AI agents and inspect the result. > - A task can use repositories from more than one project. > - Remote workspace preparation treated every additional project as a read-only source. > - This rejected valid Git repositories before the provider started, even when no source was read-only. > - The task page then hid the preparation reason behind a generic failed-attempt notice. > - This pull request prepares authorized repositories as writable task copies and shows redacted workspace validation errors. > - People can work across both projects and see the reason if folder preparation fails. ## Linked Issues or Issue Description **What happened?** A remote task with a selected Git-backed project and another mentioned Git-backed project failed before provider execution. It reported `remote_workspace_readonly_sources_unsupported`. Neither project had a read-only setting. The task notice showed only that the attempt failed. **Expected behavior** Each attached repository should be writable inside the task workspace. This should preserve source access checks and saved work. A workspace validation failure should explain its cause on the task page. **Steps to reproduce** 1. Configure remote workspace storage and a native sandbox environment. 2. Select a Git-backed project for a task. 3. Mention another authorized Git-backed project in the task. 4. Start the task. Preparation rejects the additional source before it starts the provider. **Paperclip version or commit** Reproduced against `dc9f387edc`, before this change. **Deployment mode** Managed remote workspace storage. Local regression checks use disposable PostgreSQL and independent filesystem processes. Related merged work: Refs #18403 and #18504. Those changes established remote storage and metadata-only references. They still reject configured cross-project repositories. No duplicate writable-repository repair was found. Refs #18774 for the test-only retention-fixture repair reused to unblock CI. ## What Changed - Add every configured Git repository of an authorized referenced project to `.paperclip-repositories/` as an editable task copy. - Retain saved paths, refs, pins and files. Recheck retained project access and source trust. Keep explicit read-only, local-source import and remote feature gates. - Keep committed snapshot repository coverage separate from later requests. Honor configured default refs, resolve remote branch names, and publish secondary checkouts only after checkout succeeds. Share configured-ref resolution between root and attached-repository scripts. - Include redacted workspace validation messages in the task interruption projection and existing failure notice. Preserve Retry permissions and render messages as plain text. Explain retained-folder access and trust denials while preserving the dedicated low-trust project-access path. - Add source, authorization, redaction, UI and save/restore regressions. Update the implementation contract and workspace maps. - Reuse the reviewed test-only repair from #18774. Batch the shared-workspace history fixture, verify its complete privacy-source map, and drain owned test bodies before cleanup. Production retention behavior is unchanged. ## Verification - Red before repair: four source-resolution cases failed with the reported read-only error. The task-error projection test also failed because it omitted the cause. - Green: 49 source and transport tests, 16 task projection and redaction tests, and 10 UI tests. The assembled preparation check passes. All 48 remote workspace lifecycle tests pass. The full preparation suite passed all 126 assertions but its cleanup hook exceeded 30 seconds. The unchanged base reproduces the same cleanup timeout with all 126 assertions passing. The focused preparation check finishes cleanly. - The real PostgreSQL/Git lifecycle checks cover both first-turn and later project attachments. They verify the configured release branch, saved edits, replacement allocations, unchanged source folders, committed archive contents, and released writers. - Review regressions were red before repair: the default ref became null, later attachment failed during restore, non-default remote branch checkout failed, and invalid secondary refs left partial checkouts. They now pass. - The local browser shows the workspace validation explanation and Retry action. The fixture uses a saved failed run; this proves the error display, not a successful live sandbox run. - `pnpm -r typecheck` and `pnpm build` pass. Map, module-boundary and token checks pass. - A local two-turn acceptance probe repeated at `17ed67eaa7` used real Codex CLI 0.165.0, `gpt-6-astra`, and `xhigh` effort. Codex wrote files in both task repositories. The services saved them, removed the first allocation and its Git sources, restored another allocation, and Codex read the saved files and wrote a follow-up in both folders. This uses the real source resolver and persistence services with a local command-runner stand-in for sandbox transport. It does not qualify Daytona, managed subscription forwarding, or the original task. The first probe waited for stdin; closing the test harness pipe fixed that setup error before provider work. - `pnpm test:run` passed the first chat shard, then failed one chat response concurrency assertion in the second shard. That case passes in isolation on this head. The same isolated chat case passes on the unchanged base. Both corresponding unchanged-base chat shards also pass (351 and 324 tests). The cause of the earlier wider-suite chat failure is not established. The preparation cleanup timeout and chat failure remain part of this record. - CI at `17ed67eaa7`: all other verification jobs passed. Browser shard 6 initially reported a request callback from the preceding test after teardown; its single unchanged-code retry passed. Server shard 10 timed out the 2,100-row retention confirmation fixture at 15 seconds on both attempts. That exact case passes locally, but the repeated CI failure required a repair. Its setup repeatedly rebuilt a shared-workspace privacy map. The logs and failures remain part of this record. - Reused the test-only repair from #18774 without changing its source file. At `ab774e9418`, all 54 retention tests and all 25 ordinary production privacy tests pass locally (79 total). Full CI [41650626903](https://github.com/paperclipai/paperclip/actions/runs/41650626903) passed on that commit, and its exact-head Greptile review was 5/5. The fixture preserves its 2,100 rows, query-bound assertions, privacy-source map, and completed-history test deadlines. - The additional review identified retained-source access/trust denials that still bypassed the workspace explanation. Real-resolver regressions reproduced both plain-403 failures before the correction. Both now yield a safe workspace validation explanation, while the dedicated low-trust denial and saved workspace remain unchanged. The optional duplicate ref-resolution block is also consolidated. - Final review corrections: 65 source, transport and error-projection tests passed, followed by all 48 lifecycle cases against an owned disposable PostgreSQL database (113 total). The initial combined command skipped the explicitly gated lifecycle suite; the subsequent dedicated invocation enabled and passed all 48. Full workspace typecheck and build pass. The first standalone lifecycle launcher lacked the repository TypeScript loader; rerunning with that loader resolved its import setup error. At previous head `7be02ea09c4ac76df560e8b517050cdf398f33f0`, full CI [41654422329, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41654422329/attempts/2) passed, Greptile is 5/5 on that exact head, and all review threads are resolved. The first attempt timed out the unchanged voice-session report/history case at 15 seconds; 3,027 cases in its shard passed. That exact case passed locally, then its single unchanged-code shard retry passed. The wider-suite timeout cause remains unproven. The completed Paperclip advisory check is neutral; its earlier findings are corrected. No staging deployment or original-task recovery is claimed. - Master advanced while checks ran, producing conflicts in two adjacent test additions. Merged `27d35314be` into this branch as `d5c8d552939c86056abf8fe10833881b2ade41a4`, preserving both workspace-error tests and the audited Stop/receipt tests. Kept workspace maps to minimal corrections under the updated map rule. All 82 source, transport and interruption cases, all 48 real-database save/restore lifecycle cases, full typecheck, build, and map checks pass locally on the combined code. Current-head [CI 41660719214](https://github.com/paperclipai/paperclip/actions/runs/41660719214) and Greptile are pending; the earlier verified head has not been merged. ## Risks - Additional remote project repositories are now editable task copies. Shared source folders and upstream push permissions do not change. - Snapshot coverage uses existing workspace metadata. Older workspaces capture their previous inventory at reservation; successful save publication advances coverage and snapshot identity in one transaction. No SQL migration is needed. - A retained copy can contain files from a project that is no longer mentioned. Preparation rechecks that project's current access and trust policy before reuse. - Explicit read-only descriptors still fail. Source-less projects remain metadata-only. Local folders still require import. No migration or provider-pack update is needed. - Error display reads only workspace validation errors. The existing run-secret registry and username censor run before display truncation. A redaction failure omits the explanation. - The retained fixture repair pauses its row trigger only within the isolated seed transaction, rebuilds the exact source map, and restores the trigger before commit. A rollback regression and 25 ordinary privacy cases pass. Its paged-case test budget honors the existing bounded inner waits; production deadlines are unchanged. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The exact serving model ID and context-window size are not exposed by this session. The test-only fixture changes are reused from #18774, whose model record identifies OpenAI GPT-6.1 Sol. The optional live acceptance probe uses `gpt-6-astra` with `xhigh` effort through Codex CLI 0.165.0. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 15:33:44 +00:00
DottaandPaperclip	b8f8b904fa	feat(routines): preserve automation ancestry through webhooks (#18772) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines let external services start agent tasks through a normal webhook. > - A repair routine must recognize failures from work that it created. > - The receiver added in #18714 put Sentry payload parsing and repair policy in core routine intake. > - This pull request replaces that receiver with verified creation provenance on normal routine calls. > - Sentry can filter the resulting tags before it sends a normal routine webhook. > - The benefit is reusable provenance and loop prevention without a provider-specific route. ## Linked Issues or Issue Description **Problem or motivation** A Sentry repair alert can start another repair when its own work fails. The dedicated receiver in #18714 solved this through a Sentry-specific payload contract. Ordinary routine webhooks also need a trusted way to retain the automation that created their tasks. **Proposed solution** Use the existing public `/fire` route and its current authentication. An optional source run credential proves a live Paperclip caller. Store the caller separately from webhook JSON. Follow recorded task creation links and export bounded ancestry tags on run-failure events. Let Sentry alert filters exclude repair-origin work. **Alternatives considered** Keep the dedicated Sentry receiver, accept sender-supplied origin IDs, or filter only the immediate routine. The first couples core intake to one provider. The second accepts unverified claims. The third misses child tasks, standalone follow-ups, and calls through another routine. **Roadmap alignment** This improves the shipped Scheduled Routines and Activity log capabilities. It does not add a new scheduler or incident system. Related work: #18714 is the implementation replaced here; #9727 adds source-task navigation and UI attribution; #10206 proposes exception-led routine dispatch. This PR uses existing task creation records and does not implement either of those broader proposals. ## What Changed - Remove the Sentry-only route, parser, authentication exception, and payload-specific tests and documentation from #18714. - Add optional `X-Paperclip-Source-Run-Token` support to normal routine webhooks. Validate the standard run JWT, company, instance, agent, live run, and cancellation state. Authenticated routine API calls also retain their validated caller. - Add nullable `routine_runs.created_by_run_id`. Keep the reference after source-run deletion so missing history becomes unknown. Separate callers during coalescing, including legacy open tasks. - Follow company-scoped routine receipts, creator runs, task parents, and legacy creation activity. Mark missing, foreign, cyclic, private, or excessive ancestry for safe alert filtering. - Add event-local `automation_origin_status`, `automation_routine_ids`, and `task_shareable` tags through the existing Sentry SDK hook. Cap ancestry at 32 reads and share a 750 ms deadline across metadata and ancestry lookup. Keep the SDK one-second wait and eight-lookup concurrency bound. Redact the source credential from HTTP logs and Sentry request headers. - Preserve unexpected database failures as server errors without rejection bookkeeping in an aborted transaction. Document 409, 415, and 422 responses for the normal webhook. - Keep `routineScopedRunId` as generic opt-in `X-Paperclip-Idempotency-Scope: routine` support. Default trigger receipts and timestamped HMAC replay protection retain their current scope. - Add [the Sentry repair tutorial](https://github.com/paperclipai/paperclip/blob/codex/routine-creation-provenance/docs/guides/board-operator/sentry-run-failure-repairs.md), API documentation, OpenAPI headers, and capability-map updates. ## Verification At commit `0f4a0051528edd1367a921a76a8050c3fd087872`: - `pnpm -r typecheck` and `pnpm build` passed for the full workspace. Migration numbering and safety checks passed. - The focused service, route, Sentry, redaction, and OpenAPI suites passed 228 tests in five files. The database Sentry resolver and OpenAPI invocation passed 33 tests in two files. These are separate invocations with some overlap. - The full routine HTTP suite passed all 15 tests. The authenticated API case failed before the caller-wiring fix and passed after it. The fixture loads the real route-owned services. No test deadline was raised. - The real `@sentry/node@10.71.0` suite passed all 12 tests. It uses an in-memory transport and sends no events to Sentry. It covers terminal failures, saved-message interruptions, event isolation, and source-token redaction. - Regression tests also cover forged, expired, foreign, and cancelled caller credentials; payload spoofing; inherited routine ancestry; legacy coalescing; deleted history; private work; cycles; query and time limits; database error causes; and routine-wide receipts. The new error and budget cases failed before their fixes. - `node scripts/check-maps.mjs` and `git diff --check` passed. The map check covers 35 areas and 192 capabilities. - The full local Vitest command was stopped for a required master merge after four chat shards passed 1,278 tests. No full local pass is claimed. An earlier combined HTTP run hit an existing malformed-ID timeout whose original cause is not established; the final full HTTP suite passes. An extra local run of the unchanged heartbeat scheduling file passed 12 of 13 tests and failed a readiness-status assertion. Its logs also reported a missing fixture user membership. The same 13-test file now passes in current-head CI. - All GitHub Actions checks pass on this commit. Three jobs initially lost their self-hosted runners. One server shard initially failed while deleting a company in an unchanged heartbeat test because of a `company_skills` foreign key. One retry passed all affected jobs without code changes. That server shard passed all 499 tests, and all eight browser shards passed. - Fresh Greptile review is 5/5 on this commit. All six review threads are resolved. The separate Paperclip Review check remains queued. Its bot reports that work is paused. A maintainer must resume the assigned review work to complete that check. No full green check state is claimed while this external gate is pending. - Live Sentry configuration and deployment are outside this PR's acceptance evidence. The tutorial describes setup, a controlled failure, and a repair-origin exclusion check for operators to run after deployment. ## Risks - Normal app webhooks retain the sender's full JSON in company-readable routine history and the execution task. Alert filters and task visibility are not content scrubbers. Operators must scrub sensitive event data in Sentry before forwarding it and keep connection tests in setup mode. - Existing users of `/sentry-run-failures` must move to `/fire` and add the documented Sentry filters before enabling repair intake. The old route is removed. Stored tasks and receipts remain. - The additive migration uses `ADD COLUMN IF NOT EXISTS`. It does not rewrite historical tasks or infer missing provenance. - Unknown or unshareable ancestry is excluded by the tutorial's alert rule. Missing history, an enrichment timeout, or more than five routine IDs can therefore suppress a repair webhook while retaining the Sentry error. - An external webhook without a verified source credential starts a new receiving-routine origin. Its JSON cannot establish upstream Paperclip ancestry. Sentry must not store a Paperclip run credential. - Routine-wide receipt reuse needs a stable sender key. A static Sentry custom header is not a per-event key. The tutorial keeps delivery deduplication separate from loop prevention. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-6 model family, with reasoning, repository access, code editing, and test execution. The session does not expose the exact model variant or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 09:33:17 -06:00
DottaandPaperclip	b40e6edaea	perf(inbox): combine startup reads and reduce refetch pressure (#18790) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Mine shows the work and failures that need the viewing user's attention. > - A reload waits for account access, task rows, parent captions and failure state. > - Those reads form several browser request stages before useful rows appear. > - Live invalidations can also restart the same task-list request more than once. > - This change supplies startup data together and removes redundant refresh work. > - It preserves current authority, failure presentation, archive state and row stability. ## Linked Issues or Issue Description Refs #18645. Related: #18642. This change implements the read-receipt behavior independently on current master. It does not use that PR's branch or patch. **What happened?** Mine starts its primary request after the session, company and board-access reads. It then requests parent captions and rich failure context. Four observed loads spent another 0.45–0.70 seconds after the primary rows arrived. One slower load waited 1.71 seconds for parent context. The same run observed two HTTP 429 responses. These timings identify request waits; they do not attribute the delay to SQL or Node CPU. Read receipts also invalidate general task collections and dashboards. Overlapping parent and child invalidations can start two requests and abort the first. **Expected behavior** The initial page should use one account-safe startup projection and one complete Mine response. A read receipt should refresh the affected viewer's inbox state. Each invalidation batch should refresh a covered query once. Required captions and current failure state must be ready before first reveal. Existing rows should remain usable during a transient refresh error. **Steps to reproduce** 1. Open a populated Mine inbox with parent tasks outside the visible index and a current failed run. 2. Reload with browser HTTP cache retained and application query state empty. 3. Inspect the session, company, board access, task-index and context request sequence. 4. Open or mark a task read and inspect the unrelated collection refetches. **Paperclip version or commit** Baseline: `44a9a043b80c18314d8dd89556f444dae696d3d6`. **Deployment mode** Authenticated board UI. Local proof uses the real APIs and database with a synthetic session resolver. No hosted candidate deployment is claimed. ## What Changed - Add optional, same-user `boardBootstrap` data to the existing session response. Seed account-keyed company and board-access reads from it. Keep ordinary endpoint reads for fallback, retry and live reconciliation. A returning account cannot reuse revoked cached access. Cache collection and later real refetches remain effective. - Add the board-only Mine snapshot endpoint. Reuse the canonical issue selector, private-task filtering, run-read decisions, secret redaction and interruption/recovery presentation. Assemble the fixed 100-task page, parent captions and bounded 200-run failure window on the server. Start independent reads together. Optional source errors keep successful task rows usable. - Keep array-shaped source caches for archive and undo. Key Mine source, context and reveal state by account. Validate the response's user stamp before committing it. Remove array-only cross-tab Mine broadcasts, which cannot carry its paired failure context. The server retains authenticated in-flight coalescing. - Scope read-receipt invalidation to the affected viewer's inbox, badges and chat unread state. Keep company and task audit views fresh. Preserve genuine task mutations, run updates and subscription-gap reconciliation. - Remove covered scalar-prefix invalidations within a batch. Preserve predicates, exact matches and different query/refetch constraints. A later event can still refresh again. - Retire Mine in-flight reuse after local company activity or run lifecycle changes. A post-archive read starts fresh. Old work keeps its admission slot until it settles, even if every HTTP client aborts. The eight-compute limit stays in place. - Add API, component and browser regressions. Extend the production timing harness with matching API/UI artifact receipts, retained-cache reloads, request status counts, layout checks and optional controlled network latency. Update API and verification documentation. ## Verification - Full `pnpm -r typecheck` passes. - Full `pnpm build` passes. - Token gates, map checks and diff whitespace checks pass. - 72 Inbox component tests pass. Startup-specific checks pass, including account switching, revoked cached access, malformed/absent bootstrap data and cache collection. The canonical authenticated bootstrap route also passes against real PostgreSQL. - Six real PostgreSQL Mine route cases pass, including full equivalence with legacy authorized projections, privacy, fixed bounds, membership revocation, coalescing, archive freshness and aborted-client admission. Four focused legacy coordinator controls pass. Six assembly tests pass. - Refetch and invalidation regressions pass. Meaningful pre-fix tests observed two request starts and one abort for overlapping invalidations; the fix produces one start and no abort. The archive race test observed the post-archive GET joining the old body before the fix. - All 11 existing startup browser checks pass, including Tasks/Mine code loading, desktop/mobile bounds, non-streamlined Coordinator, task navigation, failed preloading and Back position. The nine focused startup, fallback, partial-source error, rate-limit recovery, archive/undo and geometry cases also pass. The 429 cases use the inherited retry window. Archive and keyboard undo are corroborated through the real API and reload. Authentication and failure fault injection in these cases are explicitly simulated. - Matched production before/after timings are complete. Full aggregate Vitest coverage is left to CI. At 15:15 UTC, the final-head runner gate and `review` workflow pass. The 45 build/test/policy jobs are queued; the Build job has no `ubuntu-latest` runner assigned and has executed no steps, so remote aggregate results remain unverified. Paperclip Review and Superagent Security Scan are in progress. Superseded runs for this PR were cancelled. No current-head check has reported a failure; the CI checklist remains open. The remaining step is runner allocation and completion of those external checks; persistent queueing needs the repository CI owner's attention. - Greptile completed on final head `ecde295565309275c4f2fae903caf6892f1eaafd` at 5/5. The map-scope finding is fixed and resolved; there are no open review threads. ### Matched browser timings Five alternating baseline/candidate pairs per network scenario, measured serially in headful Chrome 156.0.8078.80 on Apple silicon with Node 24.22.2 and a 1440×900 viewport. Both production UIs use their matching real API revision against one fixed disposable PostgreSQL fixture: 34 tasks, 31 visible task rows, one paused agent and one terminal failure. No active agent/provider work. HTTP cache is retained; each initial sample reloads the document and clears application query state. Warm navigation and Back retain query state. CPU and throughput are native. The 100ms scenario uses CDP minimum request-to-response-header latency without request interception. It is a controlled request-dependency experiment, not a staging reproduction. Values are median milliseconds with min–max in parentheses. Paired saving is baseline minus candidate for each alternating pair; its median differs from the difference of arm medians. Positive saving favors the candidate. | Network / journey | Baseline | Candidate | Median paired saving | | --- | ---: | ---: | ---: | | Native / initial rows | 187.6 (128.9–208.5) | 191.7 (130.3–210.9) | −23.3 | | Native / Recent → Mine | 100.17 (99.90–101.87) | 100.07 (99.88–101.37) | −0.16 | | Native / task → Back | 70.45 (56.63–72.02) | 66.64 (56.52–67.71) | +4.94 | | 100ms / initial rows | 930.3 (914.1–1015.7) | 799.0 (697.1–826.0) | +211.8 | | 100ms / Recent → Mine | 100.09 (97.73–100.16) | 100.18 (96.65–100.98) | −0.02 | | 100ms / task → Back | 74.97 (62.77–75.82) | 75.47 (61.51–76.19) | +0.35 | The controlled-latency initial pairs all improve, saving 108.3–217.0ms. Native initial pairs have mixed signs (−67.5 to +78.1ms), so they establish no reliable native page-speed improvement. Warm and Back timings are effectively unchanged. Median primary-request start is 110.4→135.2ms natively and 497.8→480.9ms with 100ms latency; these runs do not establish a large startup-only timing gain. API requests through the two-second post-reveal window fall from median 32 (32–37) to 27 (25–28) natively, and 37 (33–38) to 30 (25–32) with 100ms latency. All 20 samples preserve the same ordered 31 task links, first-row DOM identity and geometry. All recorded API responses are HTTP 200; zero unfinished requests, 429s or JavaScript errors. Lower 429 incidence on staging remains unproven. Measured source: baseline `44a9a043b80c18314d8dd89556f444dae696d3d6`, candidate `4c12b2ef0ecf0754676e99bd6b9399e84408646c`. Final head `ecde295565309275c4f2fae903caf6892f1eaafd` changes only two feature-map files and existing browser test expectations since that measured candidate; production executable inputs are identical to the measured candidate. Asset hashes, API health revision receipts and every raw sample were retained. These results do not claim hosted-page improvement or deployment of this PR. ## Risks - Startup enrichment adds work to the session read. Failure of this optional projection falls back to the ordinary company and access endpoints. It does not invalidate a valid session or replace per-request server authorization. - Account changes, stale cache entries and same-company navigation can expose old presentation data. Account keys, principal stamps, reveal scopes and canonical bootstrap replacement have dedicated negative cases. Permission failures hide cached task rows; transient network, 429 and 5xx failures preserve them. - Current failure state can differ from the historical failed run. The snapshot uses the existing canonical rich projection and verified missing-ID coverage. Partial context failures retain an explicit retry path. - Mutation retirement follows events delivered to the current server process. This adds no cross-replica invalidation protocol or distributed read-after-write guarantee. - New UI assets require the matching API. Older UI/session consumers retain their existing response and endpoint behavior. There is no database migration, provider call or deployment change. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code execution, browser automation and delegated source review. The exact runtime model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 09:28:57 -06:00
Dotta	5dee4993f1	fix(server): distinguish aborted HTTP responses in access logs (#18781) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use HTTP logs to investigate task updates and database contention. > - Node starts each response with status 200 before it sends any headers. > - The custom HTTP formatter printed that status when the client disconnected before a response. > - This made an aborted request look like a successful task update. > - This pull request labels incomplete responses as aborted and preserves error severity. > - Operators can distinguish HTTP delivery from database commit evidence. ## Linked Issues or Issue Description - Refs: #18753. - Related: #14369 removes writes from legacy comment attribution reads. This change addresses a separate logging defect. The missing-write incident in #18753 remains unconfirmed. We reproduced a misleading `PATCH /issues/:id 200` log when the client received no HTTP response. We did not reproduce an HTTP 200 delivered before the mutation committed. ## What Changed - Label premature connection closure as `request aborted before response` or `request aborted during response` in both success and error log paths. - Log aborts as warnings, including on routes whose successful polling logs are normally silent. Keep explicit errors and 5xx responses at error severity. - Keep private webhook paths and query values out of aborted-request logs. - Add real HTTP disconnect coverage and real PostgreSQL task-update tests with synthetic data. - Document that HTTP completion logs do not establish database commit or rollback. ## Verification - Baseline: the new HTTP disconnect test failed because the log said `PATCH /api/issues/synthetic-task 200` at info level, while the client received no response. - `node node_modules/vitest/vitest.mjs run server/src/__tests__/http-log-aborted-response.test.ts server/src/__tests__/http-log-redaction.test.ts server/src/__tests__/issue-update-lock-contention-postgres.test.ts` passed: 81 tests in 3 files. - `pnpm -r typecheck` and `pnpm build` passed. The build used a second clean clone at the exact PR commit. - GitHub CI completed at `e3e96d836119df5fa1c7f22615484265de5aa700`: 52 checks passed and two Storybook checks were skipped. Greptile returned 5/5 with no findings. - Local `pnpm test:run` passed all six chat shards (1,834 tests). Its general server stage then reported four failures. We stopped the run after preserving these results and comparing the failing cases. The later workspace and serialized stages were not completed locally. The full local suite is not green. - The four observed failures were the two OAuth concurrency cases in `tool-access-service.test.ts`, the bounded workspace conflict case in `workspace-runtime.test.ts`, and the `enableNativeRunner`-disabled stop-control case in `muse-control.test.ts`. - Comparisons used the same sanitized environment on unchanged master `2b14ec03a48f17606db53fd7fc0edc53bc930409` and this PR. Both OAuth cases pass in isolation on both revisions. The full tool-access file passes all 498 tests on the PR; master has a different 15-second timeout, with 497 tests passing. The workspace case passes on the PR but fails port-reservation setup on master. The Muse case passes on both. These results do not explain or clear the original broad-suite OAuth and Muse failures. - Tested on macOS arm64, Node.js 26.4.0, pinned pnpm 9.15.4, and disposable embedded PostgreSQL. The API fixture uses a synthetic process-adapter agent. No model provider runs and no production database or real credentials participate. - The PostgreSQL fixture verifies an actual lock waiter with `pg_blocking_pids`. A blocked issue update does not return 200 before release. After release, status, monitor policy, inline comment, and audit receipt are persisted before the completed response. - Holding the company row with `FOR NO KEY UPDATE` does not block these fixture writes. They return 200 with persisted data while the compatible parent-row lock remains held. - An injected database trigger failure returns 500 and leaves the issue, comment, and mutation audit unchanged. Two influence-attempt receipts remain under the existing documented cap policy. - The combined lock/disconnect/failure test verifies no delivered response, an aborted log, and rollback after an injected database failure. The injected failure is not evidence of the reporter's production root cause. - Source inspection at release `v2026.1023.0` (`1b4c803816e82b0265c8422e1a804327a7ed8c54`) finds the same unconditional success formatter and awaited issue-update transaction. That release was not runtime-tested. ## Risks - Log consumers that parse the literal method/path/status message must handle the aborted-request message. - Client disconnects during streaming or SSE responses also use the aborted warning. Expected disconnects may therefore produce more warnings. - A disconnected request may still commit after the client leaves. This change reports transport completion and does not cancel or retry mutations. - This PR does not close #18753. The original missing-write cause and a received-200-without-commit scenario remain unconfirmed. - Local full-suite failures remain unresolved despite green CI. Keep this PR in draft until that verification gap is accepted or resolved. > This is a targeted bug fix. ROADMAP.md does not list overlapping feature work. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, tool use, code execution, and parallel subagent investigation. - The exact runtime model ID and context window size were not exposed to the agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-03 09:26:03 -06:00
DottaandPaperclip	27d35314be	feat: connect experimental Cursor Cloud to native Runner (#18069) ... ## Thinking Path > - Paperclip helps people manage AI agents at work. > - Native Runner owns execution and task authority. > - Operators need an explicit native Cursor Cloud choice. > - Tools/files need company, task, session and run scope. > - This connects setup and lifecycle to Runner Core. > - Legacy agents retain their selected runtime/history. ## Linked Issues or Issue Description Depends on #18068. Managed-only API placement and API-owned Stop containment are in #18310. Refs #5664, #11803, #17299, and #17471. No duplicate native integration was found. ## What Changed - Add explicit native setup, Test, saved settings and dispatch with no-prompt account/catalog/callback checks. - Scope MCP to company, agent, task, session and run through authenticated HTTPS. Register verified artifacts/work products; preserve unpriced accounting, completion and budgets. - Connect Question/Plan continuation, cold handoff and admitted fresh sessions; recheck waiting admission under the task lock. - Present audited Stop intent while preserving physical containment proof; bound typed Stop metadata. Preserve title edits through preference/navigation readiness. - Select one live/history representation per run to prevent duplicate settled cards. Integrate upstream task/read-state/retry/schema contracts and repair two test fixtures. ## Verification Core #18068 merged normally as `3910352c923a`. Product head `5a60f541c982` has passing repository CI/build/browser checks, including required `ci / verify` and `ci / e2e`, plus a fresh exact-head Greptile 5/5 review with zero unresolved threads. Two non-required external app checks remain pending: Paperclip Review is queued waiting for its assigned agent; Superagent Security Scan is reviewing the diff. These are not claimed completed. The normal merge endpoint enforces the repository rules; no policy or status override is used. The dependent Top PR is addressing its fresh deferred-Stop review finding. Product `5a60f541c982` passed 270 UI and 20 selected Cloud/native/disk-full tests, UI/server typechecks, tokens and maps. Top’s clean normal merge passed tokens/maps/diff checks; its matrix is pending. Earlier receipts retain Core’s 47 Rust, 4 compiled-native retirement and 19 settlement controls, normal Runner build, and runtime 9af build/recursive typecheck. Reproduced loading/shutdown CI fixture failures passed 7/7 browser and 121/121 unit tests, including a broken-unsubscribe negative control. No timeout/oracle was relaxed. The actual-parent restack adopts 75 mainline paths, including schema/permission and generic runtime changes, beyond the prior two fixture repairs and client projection fix. The full difference from hosted `9af1a5de42a2` is therefore larger than four files; backend source also differs. The merged config-read checks match Core391, and the Cursor placement/runtime/relay/selection modules match frozen9af; selected affected regressions pass. Paid proof keeps its actual 9af source/image identity. The separately tested live/history card fix is not claimed deployed on that host. On 9af, original saved settings survived reopening and fresh no-prompt Test authenticated catalog/model/binding; no new Save is claimed. Fresh Linux admission passed. With `claude-sonnet-4-5`, Artifact downloaded exact 27 bytes and matched 121 canonical identities in complete S3-served history. First Stop’s strict quiet FAILED on a 10.373-second gap despite independent cancellation/no late artifact. One targeted unchanged-sampler retry retained all 24 samples: 35.076 seconds quiet / maximum gap 4.584 seconds, 67 ordered canonical identities, and separate CANCELLED/artifact-absent status after the original 180-second deadline. The pre-reload duplicate settled card is disclosed. Exact-ID cleanup for the three new provider agents is awaiting authorization, so full grades and the fresh Artifact’s post-cleanup download remain blocked. A historical deleted-provider artifact was newly downloaded on 9af; original F226 Question/Plan/reconnect and cleanup proof keep their identities. Retained failures: stale fixture 132/133; E8 48/135 with 87 reset-hook failures; Mac 17 SQL failures/skips; earlier title/startup/custody/Stop/deletion failures with unknown causes/bounded recoveries; `dea023` Stop’s warning/50 duplicate identities; pre-repair ACK and original Core CI failures. WSP’s 251-assertion, 7/8-suite attempt failed physical PostgreSQL teardown; the unchanged wrapper rerun passed 128. Retry alone establishes no cause or full local-suite pass. All paid attempts remain recorded; vendor dollar cost is UNKNOWN. ## Risks Experimental: Cursor independently acknowledges neither the effective model nor instruction digest before the first prompt. Explicit values are submitted and checked through behavior; unpriced provider dollar cost stays UNKNOWN. Callbacks need authenticated reachable HTTPS. Scope, revocation, budgets, completion and cleanup remain authoritative. Operator Stop intent is separate from physical containment. Saved legacy agents retain their selected runtime. Prior shared title-save/deletion failures remain disclosed. ## Model Used OpenAI Codex, GPT-6 family, assisted with reasoning, repository tools, code execution, browser verification, and delegated code review. The serving model’s exact API ID and context window are not exposed. No value is inferred from the model catalog. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and the limits of available model metadata - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch describes the change and contains no internal ticket ID - [x] I have run the scoped local tests reported above; pending gates are stated separately - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All required CI gates pass on the current head - [x] Fresh Greptile is 5/5 with no unresolved findings on the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 09:01:39 -06:00
DottaandPaperclip	3d5878980b	fix(ui): show agent avatars in the chat picker (#18709) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Chat lets a person choose an agent for a conversation. > - The shared agent picker shows a role icon by default. > - An icon does not show the agent's saved visual identity. > - This pull request uses the shared avatar component in that picker. > - The picker now matches the avatar in the agent profile and chat sidebar. ## Linked Issues or Issue Description **What happened?** The chat header switcher and main sidebar picker show agent role icons. The conversation sidebar already supplies avatars through its existing render option. **Expected behavior** All chat pickers show the agent's saved avatar. Uploaded images and default appearances use the same shared component. **Steps to reproduce** 1. Open an agent conversation. 2. Use the agent switcher in the chat header. 3. Compare each choice with the agent's profile avatar. Related work: [chat header profile navigation](https://github.com/paperclipai/paperclip/pull/17169) changes profile links. [Task list avatars](https://github.com/paperclipai/paperclip/pull/18607) changes a separate surface. ## What Changed - Use `AgentAvatar` at size 32 as the default in `AgentChatPicker`. - Keep the existing custom render option and selection behavior. - Test custom and default avatars inside the picker. Test the custom render option. - Keep the feature map unchanged for this small UI fix. ## Verification - Passed: focused picker, avatar, sidebar, and chat page tests (36 tests across 6 files). - Regression proof: the new picker avatar test fails with the original icon rendering and passes with this fix. - Passed: `pnpm --filter @paperclipai/ui typecheck`. - Passed: `pnpm --filter @paperclipai/ui build`. - Passed: `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `git diff --check`. - Full local typecheck and build could not finish because this environment has no Rust `cargo` toolchain. - The full test run was stopped after concurrent validation exceeded the workspace memory limit. The first UI build and local server were killed. The UI build passed when run on its own. Focused tests passed before the full run. - Browser verification passed against a fresh local source-checkout instance with real API responses and embedded PostgreSQL. No agent runs were started. Chromium checked header and sidebar pickers, an uploaded custom image, role search, Enter selection, reload, and a 390-pixel mobile viewport. No page errors occurred. - Desktop, mobile, and sidebar screenshots were captured. They are not committed to the repository. - Previous head `9bf268c308e51113b0a10fb1314444cc99882b04`: all CI checks passed; Greptile was 5/5 with no outstanding findings. - Head `4e85eedd155ccf1624233009905d5ebfe8c3849b`: all 36 focused tests, UI typecheck/build, token gates, map checks and whitespace checks passed again after merging master. Fresh review gave 4/5 because this small fix added feature-map detail. - Current head `e13af2d5650e95ff552b1bfe392820c490eccf23`: removed all four added feature-map lines. The map matches the base version. Map checks and whitespace checks pass. The code and tests did not change. All 30 check runs are complete: 25 passed, 4 were skipped, and 1 was neutral. Build, typecheck, tests, all eight E2E shards, and both CI summary gates passed. The legacy commit status passed. Greptile gave 5/5 with no open findings. All review threads are resolved. The separate Paperclip review ended early with a neutral check; its prior map finding is fixed. ## Risks - Low risk. The change uses the existing avatar component and its image-error fallback. Picker rows now use the same avatar size as the conversation sidebar. ## Model Used - OpenAI Codex, GPT-6, with reasoning, repository editing, shell tools, and browser verification. The runtime does not expose a more specific model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #123` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 08:54:08 -06:00
DottaandPaperclip	e16de641ba	fix(heartbeat): retain Stop and receipt debt after setup failure (#18776) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server owns execution, Stop delivery, and usage settlement. > - A restored execution can fail during preparation. > - A preparation failure does not prove that earlier provider work never started. > - Replacing its result metadata can remove an existing Stop and falsely release its accounting hold. > - This pull request preserves those obligations at the terminal write. > - The benefit is honest cleanup and receipt debt without new execution authority. ## Linked Issues or Issue Description **What happened?** A recovered native run can fail a configuration check before dispatch in the new controller. The setup failure writes bootstrap metadata with `providerWorkStarted: false`. It also replaces the previous result metadata. This can remove a pending native Stop. Accounting then accepts the new bootstrap flag and releases the original hold despite an incomplete receipt. **Expected behavior** Keep the current run-Stop and receipt debt when setup fails after earlier native execution. Preserve an acknowledgement that commits during the failure write. Fresh setup that never dispatched must remain eligible for pre-provider settlement. Keep account-identity checks effective. **Steps to reproduce** 1. Save admitted native execution, an active provider checkpoint, an original held reservation, and an audited pending run-only Stop. 2. Recover the run and fail preparation with an account configuration error before the new controller dispatches. 3. Run ordinary cost reconciliation with an incomplete receipt. 4. Observe that the base removes the Stop and releases the hold as if the run never started. Related: #18682 repairs exact-session Stop cleanup after restart. This change protects setup failure before that path. #18702 handles AI connection-request waits. #18035 proposes broader accounting policy. #18778 separately handles retained cleanup across account changes. Account, budget, and receipt authority are unchanged here. ## What Changed - Mark setup-failure status writes through an internal lifecycle option. At the atomic write, a prior coordinator attempt bound to the same run, company and task prevents false pre-provider evidence, even with a stale loaded profile. Lock the coordinator before the run so an in-flight claim cannot be missed. Selected execution input alone does not imply provider work. - Keep genuine pre-dispatch setup eligible for settlement after fresh native input is persisted, whether the loaded profile is current or stale. - Preserve only exactly bound native run-Stop fields from the current database row during failure writes, including the intentional handoff marker. Keep the latest acknowledgement when the update waits behind another writer. Normal Stop delivery still advances its own metadata. Replace unrelated stale result fields. - Add real PostgreSQL regressions for receipt debt, original holds, stale snapshots, late acknowledgement, fresh setup, dispatched work, and foreign or task-scoped metadata. - Update the execution contract. ## Verification - On untouched master, the recovered-setup regression fails six assertions: Stop removal, false bootstrap evidence, false accounting settlement, lost debt, and released hold. The fresh-setup control passes. - A second regression fails four assertions with the first repair present when the loaded profile lacks the current saved execution. The atomic current-row check fixes that case. - The focused safety selection passes. It includes real row-lock contention. A controlled concurrent writer seeds audited acknowledgement state while the terminal update waits. The product code retains that state and creates no acknowledgement or provider turn. This race test does not prove physical provider termination. - Fresh native and legacy setup can settle without a receipt. Dispatched native and legacy work keeps its debt. Foreign run, company, issue and task-scope metadata stays excluded. Unrelated old result fields are replaced. - After the final review fixes, all three completion, Stop and budget-cancellation suites pass: 121 tests. The exact `pnpm run typecheck:build-gaps` entry point passes after the final source and service-test edits. Module-boundary checks pass. An ordinary adapter pre-provider receipt control also passes. - Native review also found that fresh selected input could be mistaken for prior execution. Real-admission loaded/stale cases fail against the earlier head; a prior-attempt-only control passes. The correction uses the bound coordinator attempt and includes a foreign-task negative. Reservation `providerStartedAt` is stamped at reservation creation and is not used as provider-work evidence. - A row-lock regression also exposes a first execution claim committing during the terminal write: the prior statement snapshot misses it and falsely releases the hold. Locking the exact coordinator before the run fixes that race and preserves the existing claim lock order. Concurrent Stop acknowledgement tests remain green. - Review found two preservation gaps. New real-database reds show handoff classification loss and overwritten budget Stop delivery progress. Failure-only preservation and the bounded handoff marker address them. - Initial helper-import and reservation-order fixture errors are retained separately from causal failures. No assertion or deadline was relaxed to pass the regression. - Current-head Greptile is 5/5. The optional native review stopped before completing and returned NEUTRAL; this is not substantive approval. Its prior finding was reproduced and fixed. Coordinator and independent source/evidence reviews found no additional issue; neither is a hosted recovery test. - These are local database tests with saved provider state. No hosted recovery, physical provider termination, historical-state repair, or deployment is claimed. Full repository tests and build passed in current-head CI: 55 successful checks, three expected skips, and the explicitly incomplete optional native review returned NEUTRAL. Required verify and e2e aggregates passed. ## Risks - The current database row must retain canonical Stop state while replacing other result metadata. The preservation is limited to the same native run, company, issue and run-only Stop scope. - Setup-failure history checks keep uncertain work pending. They do not create a receipt, acknowledgement, reservation or charge. - The setup-only write option leaves ordinary adapter pre-provider receipts unchanged. No account-identity, budget, dispatch, or provider authorization gate is bypassed. - Already overwritten historical state needs separate evidence and repair. This change does not reconstruct it. - No schema, public API, provider profile, or deployment setting changes are required. ## Model Used OpenAI Codex, `gpt-6.1-sol`, Extra High reasoning, with source inspection, code execution, and regression testing. Context-window capacity is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1103.0-canary.10	2026-11-03 08:49:28 -06:00
DottaandPaperclip	3910352c92	feat(runner): add experimental Cursor Cloud provider (#18068) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runner gives each provider durable execution, tool, and completion contracts. > - Cursor Cloud currently has a direct adapter outside that native contract. > - Cloud work needs exact ownership across HTTP, SSE, Stop, and recovery. > - This PR adds a qualified Cloud API driver and verified proxy to Runner. > - Product setup and hosted placement remain separate dependent changes. ## Linked Issues or Issue Description Refs #5664, #11803, #17299, and #17471. Product integration is #18069. Managed-only API placement is #18310. The related direct-adapter model/transcript work is not replaced. No duplicate native Cursor Cloud integration was found. ## What Changed - Add native `cursor_cloud` / `cursor_cloud_api` with explicit model/repository validation, bounded HTTP/SSE, durable per-turn checkpoints and a qualified Node proxy launched through the Rust facade. - Commit terminal custody before publication/release; coalesce consumers, preserve ambiguous create/Stop/recovery fences, reject late tools, and distinguish definitely-unsent oversized requests from ambiguous responses. - Support scoped cold attachment, current authority, reconnect and verified artifact staging. Keep unsupported warm/external and Windows paths closed. - Preserve early source cursors and verify immutable legacy prefixes before suffix replay, including supported compaction/masking. Optional proof publication failure falls back to complete cold verification. - Preserve canonical records at the durable native producer with JSON-safe privacy and the existing 8MiB bound; drain admitted successor flushes. Reconcile upstream semantic/pack/schema/lifecycle changes and retain S3 and legacy CLI checks. ## Verification Core head `37c442ec8dd4`: all 66 check contexts are terminal successful/neutral/skipped, including required `ci / verify` and `ci / e2e`; fresh exact-head Greptile is 5/5 with zero unresolved threads. Dependent Product `81e828d20a77` and Top `dd2959689627` include the later client handoff correction and are running their own fresh gates. Core: 47/47 Rust, 4/4 compiled-native retirement, 19/19 settlement and normal Runner build passed. Runtime Top `9af1a5de42a2`: build and recursive typecheck passed. Earlier source-scoped evidence includes 398 focused tests, five relay tests and token/type gates. The 251-assertion, 7/8-suite run failed physical PostgreSQL teardown; the unchanged stable WSP suite passed 128 through the documented wrapper. The downstream fixture corrections passed 7/7 browser and 121/121 startup tests without timeout/oracle relaxation; a broken-unsubscribe control fails. Hosted runtime `9af1a5de42a2` has a passing normal build and recursive typecheck. Latest dependent heads additionally contain a pure client fix for stale live/history overlap, with 48 focused tests, UI typecheck and token gates passing. Its new client is not deployed in the frozen hosted image; server/native/API/provider source is unchanged by these four downstream files. Fresh hosted Linux admission on `9af1a5de42a2` retired its owned runner/proxy/socket with zero prompts. The saved configuration survived reopening, and Test authenticated the bound key/catalog/selected model without a prompt. Artifact with `claude-sonnet-4-5` completed, downloaded the exact 27 bytes, matched 121 canonical identities against complete logs and independently reached FINISHED. The active Stop retry was clicked within its immutable boundary, produced one canonical interrupted turn and acknowledged run-only cancellation, retained a valid 35.076s quiet window (maximum gap 4.584s), and independently remained CANCELLED/artifact-absent beyond the original 180-second command deadline. All provider-observer credential callbacks, DB clients and loaders closed; accounting is settled and explicitly unpriced. The first fresh Stop retains its failed 10.373s sampling gap and every sample. On the upgraded9af server, the historical F226 artifact was reopened and downloaded with its exact original hash after its provider agent had been deleted/GET404 earlier today. Historical F226 full Artifact/Stop cleanup, Question, corrected Plan and reconnect retain their original identities. Deleting the three newly created disposable provider agents and the new-case post-delete grade remain blocked on explicit user approval after automatic review rejected expanding the two original approved IDs. No new-case cleanup or complete fresh matrix pass is claimed; this is separate from the directly observed task/Stop behavior and retained exact cleanup proof. Retained failures: stale fixture 132/133; E8 48/135 with 87 reset-hook failures; Mac 17 SQL failures/skips; earlier startup-title, title-save, custody/Stop and deletion failures with unknown causes and bounded recoveries. `dea023` Artifact passed 17 checks but Stop failed its false warning and 50 duplicate log identities. Pre-repair ACK failure and original Core CI failure remain recorded; retry alone establishes no cause or full local-suite pass. Provider dollar cost stays UNKNOWN. ## Risks Experimental: Cursor independently acknowledges neither the effective model nor instruction digest before the first prompt. The caller’s explicit values are submitted and behavior checked live. Unpriced provider dollar cost stays UNKNOWN. Only qualified per-turn controllers are admitted. Uncertain work and incomplete cleanup stay fenced. Existing damaged/duplicate archives remain failed evidence. Prefix proof adds bounded object-store I/O; the 8MiB producer bound does not guarantee future providers’ output fits. ## Model Used OpenAI Codex, GPT-6 family, assisted with reasoning, repository tools, code execution, browser verification, and delegated code review. The serving model’s exact API ID and context window are not exposed. No value is inferred from the model catalog. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and the limits of available model metadata - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch describes the change and contains no internal ticket ID - [x] I have run the scoped local tests reported above; pending gates are stated separately - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All required CI gates pass on the current head - [x] Fresh Greptile is 5/5 with no unresolved findings on the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 08:38:57 -06:00
DottaandPaperclip	2b14ec03a4	fix(ai-connections): remove account execution locks (#18778) ... ## Thinking Path > - Paperclip manages AI agents and the accounts that supply their model credentials. > - Several runs can use the same AI connection at the same time. > - Runtime preparation treated unfinished credential cleanup as exclusive account ownership. > - Manual account tests also held a company credential mutex for the entire provider test. > - A failed cleanup could therefore cancel fresh runs before the provider started and keep them in automatic retries. > - This pull request removes that execution gate and those read and probe locks. > - Each invocation keeps its own credential home. Credential updates still use the existing freshness and authorization checks. ## Linked Issues or Issue Description **What happened?** Fresh runs stopped with `ai_connection_busy` and the message “The selected AI account is still completing credential cleanup. Retry shortly.” Pending cleanup from an earlier invocation blocked every new invocation of that account. Concurrent manual account tests also waited on a company mutex. **Expected behavior** AI connections support concurrent runs and account tests. Cleanup for one private runtime must not prevent another private runtime from using the same account. **Steps to reproduce** 1. Prepare a managed OpenAI account and retain an earlier runtime whose cleanup has not finished. 2. Prepare another run with that account. The old implementation rejects it before provider work starts. 3. Alternatively, keep one manual account test open and start a second test with the same account. The old implementation prevents the second provider callback from starting. **Paperclip version or commit** Reproduced with two PostgreSQL regressions on `17e8bd773543d62ab3f444c55255acfd82575efd`. Both fail before the fix and pass after it. **Deployment mode** The reported deployment used authenticated Cloud. Local reproduction uses disposable PostgreSQL and fixture credentials. No live provider credentials are required for these regressions. Related search found #15952, which concerns account removal and provider application archival. It does not remove this runtime cleanup gate. This fix does not introduce the account concurrency cap requested in #14564. ROADMAP.md was checked; this change restores concurrent account use. ## What Changed - Remove the account-wide cleanup assertion and cleanup mutex from run admission. - Read ordinary subscription credentials without acquiring the company file mutex or grant and secret write locks. Only an actual durable replacement recovery enters the existing write transaction. - Remove the mutex held across manual account tests. - Retry retained cleanup without making new runtime preparation wait for it. A home-specific cleanup check considers only that exact private home. - Keep cleanup attempts for one runtime idempotent. Keep physical retirement, credential freshness, reconnect, revocation and company checks. - Update concurrency, refresh and cleanup regressions, and the implementation contract. ## Verification - Before/after: the pending-cleanup and simultaneous-probe regressions both fail on the original code and pass with the fix. - Affected suites: 186 tests passed across `ai-connections.test.ts`, `antigravity-ai-connections.integration.test.ts`, and `retained-google-credential-cleanup.test.ts`. They use real PostgreSQL, encrypted fixture credentials and controlled provider callbacks. - `pnpm -r typecheck` passed. - `node scripts/check-maps.mjs` and `git diff --check` passed. - `pnpm build` passed on the rebased branch. The server typecheck and production server build both passed again on final commit `03bd3973bdf85108bbd82355abdb217f0e1c83bd` after the runtime-home ownership changes. - Caller coverage: all 166 cases pass across native credential handoff, managed Google subscription writeback, model discovery, heartbeat configuration and hired-agent execution. The full hiring file passed all 79 cases after its admission assertion was corrected. Together with the affected suites, 352 targeted cases pass. - The final home-binding check passed all 188 cases across the account, managed Google writeback, model discovery and retained-cleanup suites. - Review follow-up: two real-PostgreSQL cases pass for quota refresh and concurrent Google account tests. The quota case observes a credential row update in progress, verifies that the runtime takes no account file mutex, and confirms that the private home receives the committed replacement token. The Google case completes a second provider callback while the first callback remains open in a separate private home. - Greptile rated the runtime change 4/5 and found only misplaced map patch notes. Those additions were removed. The implementation documentation remains in `doc/connections/AI-CONNECTIONS.md` and the implementation contract. - `pnpm test:run` was attempted. Chat partition 1 passed 351 expanded cases. Partition 2 passed 324 cases and failed one Teams file-receipt API case at its 15-second deadline: `refuses generic resolution with stale version transfer preconditions without changing receipts`. That case exercises publication resolution, not AI account execution. Its isolated rerun passed. The remaining local partitions did not run after that failure. This is not a claim that the full local suite passed. - Wider caller checks initially identified old serialization expectations and two cleanup retry cases. Updated assertions now require concurrent account use. The partial-removal correction lets a retained home finish removal after a durable credential handoff. All affected caller cases pass, including the full hiring file. - Live provider concurrency and deployment recovery have not been qualified. This PR does not claim that the reported Cloud deployment contains the fix. ## Risks - A new invocation can use the currently saved credential while another runtime still owns a refreshed copy. The existing writeback checks decide which credential can replace the saved value. - Actual credential updates and durable replacement recovery retain atomic storage transactions. This change removes execution serialization, not the integrity checks on stored credentials. - Unfinished cleanup retains its private home and its existing idle hold. This change does not certify a previous provider process as stopped. - There is no database migration, credential format change, permission expansion or ambient-account fallback. ## Model Used - OpenAI GPT-6 through Codex desktop, with source editing, shell execution, PostgreSQL fixture tests and read-only browser diagnosis. The session does not expose the exact deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 07:56:37 -06:00
DottaandPaperclip	86b365f992	feat(auth): add independent agent configuration read permission (#18766) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents need to inspect configuration and instructions when they set up teammates. > - Configuration reads currently depend on configure or suggestion grants. > - Legacy standard agents did not receive those newer grants. Some self-read routes also lose the target agent identity. > - This pull request adds an independent read grant and applies only that grant to eligible existing agents. > - The benefit is that agents can inspect and verify configuration without new write authority. ## Linked Issues or Issue Description Refs #15283. Related earlier proposals: #8064 and #12050. Those proposals target older authorization code; this change uses the current configure/suggest model and adds a read-only rollout for existing agents. Related bounded peer metadata proposal: #14160. **What existing behavior does this improve?** Agent configuration, configured skill, instruction, and revision reads through the existing APIs and native instruction tools. **Current behavior** An agent needs `agents:configure` or `agents:suggest-changes` to inspect a peer. Older standard agents can hire a teammate but cannot verify the saved instructions. Configuration and revision routes also reject self-reads because they authorize a company resource instead of the target agent. **Proposed behavior** Add `agents:read` for redacted configuration and instruction reads. Keep `agents:configure` as read/write and preserve suggestion-grant compatibility. Standard new agents receive the read grant. A one-time migration grants only reads to eligible existing standard agents. Individual routes authorize the target agent and grant scope. **Reason and benefit** Read access can be granted without configuration changes, rollback, or environment probes. Legacy agents can verify teammates without receiving new write permissions. ## What Changed - Add `agents:read` to the shared permission contract and standard-agent defaults. - Accept read, configure, or suggestion grants for configuration reads, with target scope and the responsible-user ceiling. - Fix target-aware self and scoped configuration, skill, and revision reads. - Add an audited data migration. Skip low-trust, built-in, inactive, pending, terminated, setup, and scoped configuration/suggestion cases. Preserve existing read scopes and later revocation. - Update the specifications, default-permission documentation, and agent access map. ## Verification - Focused acceptance: 276 tests passed across seven suites on commit `a37314df88`: authorization, real-key HTTP access, route authorization, canonical instructions, default grants, migration replay, and lifecycle configuration. - The HTTP journey uses a real hashed agent API key and a disposable PostgreSQL database. It verifies redaction, scoped reads, denied edits and rollback, cross-company isolation, and both agent and responsible-user revocation. - New read-grant regressions failed before the authorization change. A realistic paused/pausing/resuming migration fixture also failed before the lifecycle filter correction and passed afterward. Setup-held paused agents stay excluded. - `pnpm -r typecheck` and `pnpm build` passed on the rebased branch; the final lifecycle migration correction also passed the database package typecheck and build. The latest commit changes only a test assertion. - `pnpm --filter @paperclipai/db check:migrations`, `node scripts/check-maps.mjs`, and `git diff --check` passed. - Full current-head [CI run](https://github.com/paperclipai/paperclip/actions/runs/41641531426) passed: typecheck, build, all general and serialized test lanes, Runner checks, all eight browser shards, canary dry run, policy, and aggregate gates. The local full-suite attempt was stopped after exposing the outdated cross-company 403 assertion; interrupted local full-suite attempts are not counted as passes. ## Risks The migration deliberately expands redacted configuration and instruction visibility for eligible existing standard agents in the same company. An absent older write grant does not deny the new read capability. It adds no write grant. Scoped configure/suggestion grants and all existing read-grant scopes stay intact. Administrators can remove the independent read grant after the one-time rollout; ordinary updates and startup do not restore it. Configure and suggestion grants still imply reads within their own scopes. Cross-company configuration/history reads now return 404 through the common target-aware helper. No schema change or credential-value disclosure is added. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The host does not expose a more specific runtime model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 07:26:27 -06:00
DottaandPaperclip	44a9a043b8	fix(ui): make the full chat agent selector clickable (#18708) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat gives each person a standing conversation with an agent. > - The Coordinator chat header shows an agent avatar, name, and caret. > - Only the caret opened the agent picker. The avatar and name were separate text. > - This pull request puts all three parts in one button that opens the existing picker. > - The benefit is a larger click target with one keyboard focus stop. ## Linked Issues or Issue Description **What happened?** In Coordinator Chat, the caret opened the agent picker. The agent avatar and name did not open it. **Expected behavior** The avatar, name, and caret must form one control. Each part must open the same picker. **Steps to reproduce** 1. Enable the personal Coordinator workspace and open an agent chat. 2. Click the agent avatar or name in the header. 3. Compare the result with a click on the caret. ## What Changed - Put the avatar, name, and caret inside one button in `CoordinatorAgentSwitcher`. Keep its current-page marker for screen readers. - Add an optional trigger to `AgentChatPicker`. The shared dialog handles keyboard access and focus return. - Add an optional breadcrumb control slot. The chat owns its button, while the text label still sets the document title. - Add regression cases for both header layouts on desktop and mobile. Check all three click areas, Escape, focus return, and selection of another agent. - Add a Coordinator chat Storybook case with production components and fixture API data. Update the feature recipe. ## Verification - Passed 235 focused tests for `BreadcrumbBar`, `AgentChatPicker`, `BreadcrumbContext`, and `IssueDetail`. The header cases also check the current-page marker. - Passed UI typecheck and UI production build. - Passed token gates, map checks, and Storybook catalog checks. All 30 map and catalog regression tests passed. - The new header cases failed before the change and passed after it. - Full local typecheck could not finish because this workspace has no `cargo` command. - Full local build and the first UI typecheck attempt reached the 8 GB workspace memory limit. The separate UI build and typecheck passed. The full local test run was stopped to free memory for browser verification; it has no complete verdict. - Passed 18 browser checks in Chromium across desktop light, mobile light, and desktop dark. Each case checks avatar/name/caret clicks, Enter, Space, Escape/focus return, and selection of QAChecker. No page render errors occurred. - Captured six screenshots of the header and picker across these viewports and themes. Review images remain outside the repository. - Browser review first found stale fixture responses for recent comments, decision receipts, chat handoffs, and ordinary-task email state. Correct API response shapes removed those fixture render failures before the successful review. ## Risks - Low risk. The shared breadcrumb renderer has a new optional control slot. Callers that omit it retain their existing rendering. - Chat selection uses the existing picker and route. No database or API contract changes are required. - The Storybook API is simulated. It does not prove provider execution or server persistence. ## Model Used OpenAI Codex, GPT-6. The runtime did not expose the exact deployed model ID or context window. The agent used code editing, terminal checks, and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 07:20:20 -06:00
DottaandPaperclip	a61cf38106	docs: keep the feature map focused on substantial features (#18767) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The feature map describes product capabilities and user journeys. > - Small fixes and test changes have added long notes to that map. > - These notes make substantial features harder to find. > - This pull request removes those additions from the last week's changes and sets a clear rule for future additions. > - The benefit is a smaller map with stable capability IDs and accurate entry points. ## Linked Issues or Issue Description **Issue type** Unclear or confusing documentation. **Where is the issue?** `maps/feature-map/`, the map maintenance instructions, and repository rules. **What's wrong?** Between October 27 and November 3, 2026, 115 commits changed the feature map. Many additions describe bug fixes, performance details, small UI changes, or new tests. They do not introduce substantial product features. The old synchronization instructions also encouraged a map edit for every behavior change. **Suggested fix** Add a rule that requires a substantial new feature before expanding the map. Remove minor-update notes. Keep genuine feature additions and minimal corrections to existing instructions. The review covers master through `b22ef22d56`; it does not revert application code. Conflict resolution preserves later map changes from master outside that audit snapshot. Searched open GitHub PRs and issues for related feature-map work. No duplicate cleanup was found. Substantial feature additions retained here include Coordinator (#17983), the agent YAML editor (#18036), document image galleries (#18157), remote workspace storage (#18403), native Kimi (#18391), proactive chat messages (#18374), and the Sentry receiver added in #18714. ## What Changed - Add `rules/feature-map-scope.mdc` and register its `maps/**` scope with Greptile. - Align `AGENTS.md` and the shared map contract with that rule. Routine fixes and test changes no longer require incidental map edits. - Remove 755 net lines from 23 feature recipes. Preserve all 35 areas and 192 capability IDs. - Keep substantial features and the corrections needed for current navigation, provider availability, and access boundaries. Examples from the reviewed changes: | Removed or shortened addition | Why it does not need a feature-map expansion | | --- | --- | | Rounded composer image thumbnails (#18689) | Small UI change within existing task creation. | | Connection profile reads and review-count queries (#18687, #18688) | Performance details within existing app permissions. | | Warm-session, Retry, export-capacity, and tool-wait cases | Regression details for existing execution and recovery. | | New task-search, project-context, document-persistence, and routine test journeys | Test additions for existing capabilities. The tests remain in place. | | New task button placement (#18703) | Another entry to an existing task form. | | Local browser verification notes | Run evidence belongs in the PR or test report. | ## Verification - `node scripts/check-maps.mjs` passes: 35 areas, 192 capability IDs, valid references and contract fields. - `node scripts/storybook-catalog.mjs` passes: 328 files and 3,068 stories. Generated indexes remain ignored. - Compared capability keys and navigation metadata in every edited recipe with the original source. All are unchanged. - Checked rule frontmatter, Greptile registration, representative scope matches, and `git diff --check`. - `pnpm -r typecheck` and `pnpm build` pass on `9a60eef8eb8da9bc7c3a02203c79bec12e8c8b6b`. The final `a0801e8a939494ecde3739097a80258917428e94` commit only corrects two lines in the plugin-panel recipe; map and catalog checks pass again. - Greptile reviewed the final `a0801e8a939494ecde3739097a80258917428e94` head at 5/5 with no findings. The earlier review found one inaccurate phrase about unavailable plugin panels. The hook confirms that readiness gates automatic opening, while declared panels remain selectable. The final documentation commit corrects that phrase and the thread is resolved. Paperclip's refreshed review is advisory and remains queued. - The earlier CI run failed two assertions: [server 4/12](https://github.com/paperclipai/paperclip/actions/runs/41634761842/job/125914173960) counted three Anthropic identity calls instead of two in `connection-management.integration.test.ts`; [chat 1/3](https://github.com/paperclipai/paperclip/actions/runs/41634761842/job/125914174257) observed zero publications instead of one in the same-principal endpoint-settlement case. Both exact tests pass locally on the rebased head. This PR changes neither their source nor their expectations. These failed results remain historical evidence. - Final-head CI passed server 4/12 and all other suites except the first [chat 1/3 attempt](https://github.com/paperclipai/paperclip/actions/runs/41637834618/job/125925926131), which repeated the same zero-publication assertion. The browser gate passed. The single failed-job retry was accepted for this unchanged head in [workflow attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41637834618/attempts/2). [Chat 1/3 passed on that retry](https://github.com/paperclipai/paperclip/actions/runs/41637834618/job/125933199635), and the aggregate verification gate passed. Both required gates, `ci / verify` and `ci / e2e`, are green on `a0801e8a939494ecde3739097a80258917428e94`. The one retry allowance is consumed. - The first full local `pnpm test:run` attempt was interrupted before completion so master could be rebased without changing source under running tests. It is not a passing test result. Full current-head CI remains required. - No application code, tests, schemas, or lockfiles changed. ## Risks - Low runtime risk. This change only edits documentation and review guidance. - Detailed regression instructions become less visible in the feature map. Their tests and technical documentation remain available. - Some additions mix a new feature with a small fix. Those blocks retain the feature and shorten the incidental detail. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, Git history inspection, repository tools, and code execution. The session does not expose a more specific model deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — map, catalog, capability-ID, and rule-scope checks; full Vitest limits are stated above - [x] I have added or updated tests where applicable — no new runtime tests apply to this documentation change - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 07:16:37 -06:00
3 people	17e8bd7735	fix(runner): save native checkpoints when the sandbox disk is full (#18705) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs on a remote sandbox save two things after each turn: the provider checkpoint and the workspace export > - An agent can fill the sandbox disk during its own work, for example with a package install > - The turn still finishes, but both saves need free sandbox disk, so they fail and the task becomes blocked with no clear reason > - This pull request makes the checkpoint work on a full disk and makes a full disk visible in the export failure > - The benefit is fewer stranded checkpoints, and an operator sees at once why an export failed ## Linked Issues or Issue Description Related: #18715 added the shared-principal response-recovery control synchronized below. #18644 reduced the scratch disk that the workspace capture uses. #18172 added the disk-free streaming fallback to the Daytona directory sync-out. Both were deployed when this failure occurred. Neither changes the checkpoint copy, and neither can help when the disk has no free space. **What happened?** A native agent ran a package install for a large monorepo in a 10 GB Daytona sandbox. `df` in the run showed `10G 10G 4.6M 100%`. The agent committed its work and reported `done`. Then: 1. The provider checkpoint made a filtered copy of the provider directory inside the sandbox (`tar -cf archive; tar -xf archive -C snapshot`) to apply its exclusions. On the full disk, tar failed. The run recorded `runner_remote_checkpoint_snapshot_failed` without the reason, and the partial copy stayed on the disk. 2. Workspace finalization failed three times, and file preservation failed six times. The capture builds its archive inside the sandbox, so each attempt failed. Every failure was recorded as `remote_workspace_operation_failed` / `export_interrupted`, so nothing named the full disk. 3. The task went to `blocked`. The only copy of the commit was in the retained sandbox. **Expected behavior** The checkpoint does not need free sandbox disk when the provider can stream. A workspace export that cannot fit says that the sandbox disk is full. **Steps to reproduce** 1. Run a native agent with a provider whose checkpoint has exclusions (for example ACPX) in a small remote sandbox. 2. In the turn, fill the sandbox disk (for example `pnpm install` of a large repository, or `fallocate` of the free space) and finish the task. 3. The checkpoint fails with `runner_remote_checkpoint_snapshot_failed`, and every workspace export attempt fails with a generic code. **Paperclip version or commit** Observed on `410b0e27015`. This change is based on `5955dd4eb4c`. ## What Changed - Synchronize the shared-principal response-recovery regression with its saved comments and empty final deadline. Cover an in-flight discovery invalidation. Keep every authority lock, exact publication, error, and no-retry assertion. This changes test synchronization only. - `syncRemoteRunnerDirectoryOut`: when the in-sandbox checkpoint copy fails with a full-disk tar diagnostic, remove the partial copy and pass the exclusions to the provider sync-out (`exclude` on the directory mapping). The Daytona sync-out streams the directory without sandbox disk when it cannot write its archive. Other copy failures and timeouts still throw `runner_remote_checkpoint_snapshot_failed`. They now also remove the partial copy. - `remote-workspace-transport.ts`: the helper install and the capture scripts report `remote_workspace_sandbox_disk_full: ...` when Node reports `ENOSPC` or `EDQUOT`, or a Git or tar child reports "No space left on device". `command()` passes only that exact message through, like the existing unfinished-Git-operation message. - `native-workspace-finalizer.ts`: a remote save that fails with that message adds it to the failed operation's stderr. The failure code, the retry budget, and the recovery flow do not change. ## Verification - Current-head infrastructure interruption: the [Build job](https://github.com/paperclipai/paperclip/actions/runs/41637668555/job/125923826158) completed the actual build and production bundle boot, then the runner received a shutdown signal during the clean public Grok npm-install check. GitHub marked that step cancelled. The exact host-stop cause was not retained. One bounded rerun of only the failed Build and dependent `ci / verify` jobs passed on the unchanged head. [Build attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41637668555/job/125931400745) completed successfully, including the clean Grok npm-install check. No product code, assertion or workflow changed for this interruption. - CI failure retained: commit `6c5937fcae` failed [chat shard 1/3](https://github.com/paperclipai/paperclip/actions/runs/41634638570/job/125913957217) because the shared-principal fixture saw zero saved comments after `worker.ready` and zero active work. The other chat child passed. The exact case passed once locally and then 30 times on the same Node 24.22.2 version as CI; those passes alone do not explain the CI interleaving. - Controlled RED/GREEN: invalidate the real database discovery once while its read is in flight. The worker correctly retains a future reconstruction deadline, but the old initial-pass/active-count barrier reproduces the same zero-versus-one comment assertion. Waiting for both saved comments and final `active=0`/`nextWakeAt=null` passes. Six real-PostgreSQL controls pass on final commit `a60e5629e8`, including endpoint independence, source revocation, and external lock failure/recovery. No sleep, timeout increase, weakened assertion, or product scheduling/authorization change was added. - A local server typecheck first lacked the checkout's built Runner declarations. After the declared Runner TypeScript build, the server typecheck passes on `a60e5629e8`. Final-head remote verification is complete: 54 successful checks, 3 expected skips, no failures or pending checks, and successful `ci / verify` and `ci / e2e` aggregates. Greptile is 5/5 on this exact commit with no unresolved review threads. - Merge preparation: rebased onto master `b22ef22d56` and kept both test-import additions. Added the checkpoint fallback and capacity-failure contract to `doc/execution-semantics.md`. On rebased commit `6c5937fcae`, 47 focused checkpoint, transport, finalization and retained warm-projection tests passed across three files. The name filter skipped 1,040 unrelated cases. Map and whitespace checks passed. The final-head CI and Greptile results are recorded above. - `npx vitest run server/src/services/native-runtime/native-session-executor.test.ts server/src/services/native-runtime/native-workspace-finalizer.test.ts server/src/services/remote-workspace-snapshot-archive.test.ts` passes (1,152 tests). - `npx vitest run server/src/services/remote-workspace-transport.test.ts server/src/services/native-runtime/native-workspace-finalizer-failure.test.ts` passes (30 tests). - New tests: - Checkpoint, with real files: a full-disk snapshot leaves a partial copy. The provider then gets the exclusions, the session files are kept, the package caches are left out, and the partial copy is removed. - Checkpoint: a permission failure and a timeout still fail, remove the partial copy, and do not call the sync-out. - Transport: the real capture scripts run with an `ENOSPC` preload, for the helper install and for the archive write. Both report the full-disk message and leave no partial attempt. A message with extra text still collapses to `remote_workspace_operation_failed`. - Finalizer: the operation stderr names the full disk and keeps `workspace_sync_out_failed`. Other failures keep the bare code. - With the source change removed, the 3 checkpoint cases and the 2 transport ENOSPC cases fail. - `npx tsc --noEmit -p server/tsconfig.json` is clean. - Suites that refer to the changed error codes (`explicit-native-continuation`, `native-recovery-notice-work`, `native-run-finalizer-telemetry`, `native-workspace-export-retry`, `remote-workspaces.integration`) pass. In `chat-channels.integration`, 2 cases fail locally on unmodified `master` too (a bundled skill path outside the checkout, and a chat run timing assertion). They are not related to this change. - Manual recovery of the observed task: the retained sandbox was resumed with free space, file preservation was requested again, and the saved workspace contained the agent's commit. ## Risks - The fallback runs only after the snapshot fails with a full-disk diagnostic. The normal path does not change. - Provider exclusions are not anchored to the directory root. In the fallback, a nested entry that has an excluded name (for example a nested `tmp` or `config.toml` in a Codex home) is also left out. The fallback never copies more than the snapshot, so credentials stay out of the checkpoint. - The fallback depends on the provider honoring `exclude`, which the sync protocol requires. The Daytona and Kubernetes providers honor it. - A workspace export on a full disk still fails. This change only names the cause. A full fix needs one of these: free-space reserve for the export, removal of re-creatable caches before the export, or a capture that streams its archive. Each of these changes behavior, so this pull request does not include one. - No migration. ## Model Used - OpenAI Codex, GPT-6 family, assisted with merge preparation, conflict resolution, documentation, test synchronization and verification. The harness does not expose the exact serving model ID or context-window size. - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file edits, test runs). It did the root-cause analysis from run events, tool output, and sandbox inspection, wrote the code and tests, and ran local verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Dotta <bippadotta@protonmail.com>	2026-11-03 07:09:52 -06:00
DottaandPaperclip	dc9f387edc	fix(api): surface existing skill source import in discovery (#18757) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents discover supported API operations through the public OpenAPI catalog. > - Importing existing skill content must retain its source provenance. > - A normal source-import query misses the applicable operation on its first page because that operation has little search metadata. > - This pull request describes the existing import operation and its source boundaries. > - The benefit is clearer discovery without changing search scoring or import authority. ## Linked Issues or Issue Description **What happened?** `search_api` does not return `POST /api/companies/{companyId}/skills/import` in the first ten results for `import skill from source SKILL.md`. Source-management operations rank ahead of the import operation. The import entry has only the summary “Import a skill.” **Expected behavior** Ordinary source-import discovery should surface the existing operation. Its documentation should explain that imported content retains provenance and remains subject to company policy and source checks. **Steps to reproduce** Call `searchRunnerApi({ query: "import skill from source SKILL.md", limit: 10 })`. Inspect the operation IDs on the first page. The new regression sends this result through the real bounded MCP serializer. Related: #18717 clarifies the separate `create_skill` tool. This change covers the existing source-import HTTP operation. Neither PR is merged by this work. ## What Changed - Expand only the source-import OpenAPI summary and description. Describe allowed local skill directories, supported remote sources, retained provenance and existing access checks. - Add a first-page discovery regression through the actual authenticated MCP bridge and serializer. Keep non-import lookup, company/mode refusal and denial forwarding controls. - Update the feature and access maps with the test scope and its limits. ## Verification - Parent regression: the controlled loopback MCP response lacked the applicable import operation on the first page. - Focused current checks: 2,060 tests pass in `skill-import-discovery.test.ts` and the existing `runner-api.test.ts` owner. - `node scripts/check-maps.mjs` passes. This checks map structure only. - The first head had one server test job cancelled after a runner shutdown signal. The reason for the shutdown is unknown. The reviewed successor removes only two test comments required by contribution rules; its four focused controls pass. The remaining obsolete CI run was cancelled after that successor was published. No same-head retry was used. - The new test is under `server/src/__tests__`, which the production server typecheck excludes and the server Vitest suite includes. - No local app, database, native agent or provider model was run. Normal PR CI on `a4aaae8e320fb1d56cdbff9f376ec832a634a5da` passed with 48 successful jobs and one skipped job, including the required `ci / verify` and `ci / e2e` checks. No completed agent-import journey or reliability result is claimed. - Greptile is 5/5 on the current head with no unresolved threads. The optional Paperclip Review check remains queued. This is not merge authorization. - Keep this core PR open and unmerged. Do not enable auto-merge. ## Risks The expanded metadata can change where this operation ranks for related queries. Search weights, request schemas, HTTP routes, source parsing, authorization and company skill policy are unchanged. The synthetic 403 control proves denial forwarding; it does not qualify the real policy or source-import route. A live import still needs separate acceptance. ## Model Used OpenAI `gpt-6.1-sol`, through Codex with reasoning, source review and code execution tools. The exact context-window size is not exposed by this environment. A separate agent reviewed the source and contract before publication. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1103.0-canary.9	2026-11-03 06:29:13 -06:00
DottaandPaperclip	ef5b98594f	Explain the human primary-agent control in API discovery (#18747) ... ## Thinking Path > - Paperclip lets a person manage a company of AI agents. > - A primary agent is that person's company preference. > - Only the person's board session can change it. > - Native API discovery explains that restriction but omits the real human control. > - An agent can then suggest a Chat-picker setting that does not exist. > - This change describes the existing profile crown and replacement confirmation. > - It adds no agent authority and no new tool. ## Linked Issues or Issue Description **What happened?** API discovery for the personal primary-agent preference explains board-only access. It does not explain the supported profile control. A retained native response gave a correct board API alternative but also suggested a primary setting in the Chat picker. The picker only opens conversations. **Expected behavior** The discovered operation should direct the person to the chosen agent's profile, the crown control labeled Set as my primary, and the replacement confirmation. It must keep the current person's board authority and the agent denial. **Steps to reproduce** 1. Search the native API for set primary agent. 2. Read the primary-agent operation's description. 3. Compare its guidance with the profile crown and the Chat picker. 4. The current description has no supported UI path. **Paperclip version or commit** Master `7b8aa60b6d` and the retained evaluated application have this metadata gap. The profile control was checked in both sources. A scoped open-PR search found no matching primary-discovery fix. ## What Changed - Add general profile/crown/confirmation guidance to the existing GET and PUT operation description. - Clarify that the Chat picker does not change the preference. - Keep all other OpenAPI bytes, authority, schemas and handlers unchanged. - Update the two existing capability maps. ## Verification - The actual API search result passes through the existing authenticated OpenCode MCP bridge and its bounded JSON serializer. The parent fails because the delivered description has no profile control. The new result retains the full guidance, board-only actor and original schema. - The existing API client keeps its agent bearer/run identity and propagates a mocked board403. This is client propagation proof, not a new real-route claim. - Initial selected controls:997PASS,1061 intentionally unselected. After relocating the same test into the existing server `__tests__` directory, both complete owning files pass:2058 controls. These are static/protocol controls, not native journeys. - The first normal CI head failed with TS6059 because the test imported Runner source outside the server build root. The successor moves only that test to the already-excluded test directory and adjusts four relative imports. An independent source peer confirms identical test bodies; `--listFilesOnly` confirms build-program exclusion. The failed head is retained. Corrected-head CI41599277833 passed48 jobs/1skip on attempt2 at08:17:23UTC. Attempt1 had46passing jobs but a server4 runner-shutdown cancellation; exactly that job and dependent verify were rerun once on the unchanged head. - Maps and diff checks pass. The initial loopback listen EPERM was a setup failure; the controlled transport test then reached the actual assertion RED. - No Paperclip app, database, browser, model, external provider or real credential ran locally. Full typecheck/build passed in corrected-head normal PR CI. Fresh Greptile5/5 is tied to that exact head, with zero unresolved threads. ## Risks The added description can affect API search ranking. It changes guidance only. It does not impersonate a person, change permissions or automatically select an agent. Future model behavior remains unqualified. The prior mixed failed journey is retained; its separate versioned grading work is not part of this PR. Keep this core PR unmerged. ## Model Used OpenAI `gpt-6.1-sol` with reasoning, source inspection and tool use. Context-window size was not exposed by the host. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:28:57 -06:00
DottaandPaperclip	a2a6129ed5	Allow a new human intent to request tool review after a decline (#18742) ... ## Thinking Path > - Paperclip manages work for companies of AI agents. > - App writes can require a human review for each invocation. > - Decline and Always allow are separate controls. > - The gateway currently reuses a declined action across later human requests with the same arguments. > - A new request can therefore receive the old decline instead of a new review. > - This change uses the saved run and human comment to distinguish a new intent. > - The new intent still follows the current policy and approval gates. ## Linked Issues or Issue Description **What happened?** After a person declines an Ask first app write, a later request from that person for the same write receives `action_declined`. The gateway matches company, task, agent, tool and arguments across runs. It does not distinguish the later human request. **Expected behavior** A distinct later request from the same responsible person can open a new Ask first review. It must not execute the write or reuse approval. A retry of the declined intent, including its rejection continuation, must remain declined. **Steps to reproduce** 1. Ask an agent to perform an app write that requires approval. 2. Decline the proposed action. 3. Send a new request for the same action. 4. Observe that the gateway reuses the old rejection and creates no new pending review. **Paperclip version or commit** The behavior is present on master `7b8aa60b6d` and in a retained native journey. The focused parent control reproduces the matching error. No related open fix was found in the scoped PR search. ## What Changed - Only a rejected match can be released for a later human intent. - Require the active run, company, task, agent and responsible person to match the saved direct-human comment. Reject derived, deleted, untrusted or system-authored input and invalid chronology. - Preserve all other replay states, signed argument checks, current policy, unresolved-effect guards, approval and execution owners. - Document the distinction between decline and a standing policy change. ## Verification - Actual matching-function control: parent fails; this change passes. The control uses the unchanged owner body with synthetic database rows. It does not claim real PostgreSQL or provider verification. - 52 focused pure controls pass. These cover same-intent retries, rejection continuations, foreign scope and actor, invalid authorship, chronology and other action states. - TypeScript 7.0.2 checks pass for the new helper and its owning tests. - `node scripts/check-maps.mjs` and `git diff --check` pass. - The existing gateway service suite has a new signed-action integration test. It covers ordinary and reopened human requests. It requires the old rejection plus one new pending request, a stable retry, and no completed call. Each leaf prints one fixed success label after its assertions so CI can distinguish execution from a platform skip. No local PostgreSQL, app, model or provider execution was run. Current-head normal CI41594912212 completed with48 successful jobs and1 expected skip. Both real PostgreSQL leaves executed and emitted their post-assertion markers; the owning job passed. Full typecheck/build checks passed in CI. ## Risks This adds one indexed run/comment read only after a rejected matching action. Missing identity or ambiguous provenance keeps the old rejection. Legacy and external comments without direct-human provenance do not reset it. A new human intent is not consent to execute. This core PR remains unmerged. Candidate native journey qualification is separate; the original failed journey is retained. ## Model Used OpenAI `gpt-6.1-sol` with reasoning, source inspection and tool use. The host did not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #123` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:28:36 -06:00
DottaandPaperclip	8bd45b5b30	fix(ui): expose redacted arguments in pending tool reviews (#18734) ... ## Thinking Path > - Paperclip helps people manage AI agents and review their actions. > - Ask first tool actions require a person to inspect the proposed operation. > - The task and Apps review surfaces share the pending tool-action card. > - That card shows a short preview but omits the available redacted argument body. > - A valid preview can omit important text, such as the date at the end of a proposal. > - This pull request adds optional argument inspection to the shared card. > - The person can inspect the proposed content before using the existing decision controls. ## Linked Issues or Issue Description Refs: #16540. Related PRs: #16571 exposes reviewed arguments on formal board approvals and scrolls the task review into view. #16716 routes tool actions to their required formal board approvals. This change addresses the separate shared pending tool-action card used by the task takeover and linked Apps reviews. It does not close the other reported paths in #16540. ## What Changed - Add the existing Arguments and audit hash disclosure pattern to the shared pending card. - Show only the bounded argument summary already redacted by the server and its audit hash. Show an unavailable message for missing, malformed, or oversized fields. - Keep the current approval, decline, and remembered-scope callbacks. - Add a long-proposal story and update the tool-review feature recipe. - Add React coverage for the real task review transition, the linked Apps review, plain text rendering, invalid fields, and unchanged decision calls. ## Verification - The new task-review test failed on the parent because the disclosure was absent. It opens Review request, inspects the proposal tail and hash, then approves the same interaction. - Oversized-whitespace controls fail on the previous guard ordering because trimming runs before the length bound. The guard now checks length first for both fields. - Passed: 117 React tests across TaskChatInteractionCard, IssueThreadInteractionCard, and ReviewQueueCard. The tests use the repository UI config with the existing installed Vitest 5.0.3 dependency tree. - Passed: token gates, map checks, Storybook catalog checks, and git diff checks. - Local full UI typecheck did not pass. The linked dependency tree lacks adapter and json-schema-to-ts declarations and has stale shared exports. It reports no diagnostics in the changed files. This is not a successful full typecheck. - Full workspace build, browser, and provider checks were not run locally. Current head 50600786f7 passed all 48 normal CI jobs with one expected skip. Greptile and Paperclip Review both report 5/5 at this head; all review threads are resolved. No tool action or model call was run for this change. - Inspect the new ToolActionLongProposal story. Open Arguments and audit hash and read the final launch date. Inspection must not submit a decision. Existing destructive, running, failed, declined, and expired states keep their controls. ## Risks - This changes the pending review presentation in tasks and linked Apps reviews. - The UI displays the existing server-redacted projection. It does not request or reconstruct signed execution arguments. The existing server schema caps the summary at 20,000 characters and the hash at 255 characters. The UI uses the same limits and escapes both as text. - The server still owns redaction, scope checks, approval, and execution. Inspection makes no API mutation. The current callbacks retain the same request identity and permission scope. - The PR will remain unmerged for separate review and candidate-source integration. ## Model Used OpenAI `gpt-6.1-sol` (Codex), with source inspection, code editing, React tests, and Git/GitHub tools. The context window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:28:21 -06:00
DottaandPaperclip	07721e33f7	fix(runner): distinguish new skill authoring from source imports (#18717) ... ## Thinking Path > - Paperclip lets people manage agents and shared company skills. > - Agents can author new skills or add existing skills from approved sources. > - Source import and catalog install retain the source used by update, audit and reset flows. > - The create_skill tool described saving complete Markdown, without explaining this choice. > - An agent can therefore read an existing source and create an unrelated managed copy. > - This PR explains the distinction in the existing tool contract and tests its actual model-facing projections. ## Linked Issues or Issue Description **What existing behavior does this improve?** The model-facing description of create_skill and the choice between authoring and source import. **Current behavior** The tool says it saves a complete single-file skill. It does not distinguish new authored content from adding an existing skill. The Company Skills reference already directs external sources to import and catalog skills to install. **Proposed behavior** The tool explains that it saves newly authored content. It directs existing-source work to the existing import or install APIs through search_api and call_api. It retains the existing assignment and retry guidance. **Reason and benefit** A managed copy does not retain the original source locator. Clear guidance helps an agent choose the source-aware operation without adding a tool or changing permissions. This is a guidance repair. It does not claim that the import backend failed. ## What Changed - Clarify the canonical documentation, live descriptor and scenario descriptor for create_skill. - Test the actual discovered contract and provider projection, including API claim and company-boundary negatives. - Refresh generated descriptions and the seeded catalog digest and manifest entry. ## Verification - A focused regression fails on the parent because the actual discovered contract lacks the authoring versus import distinction. - Three focused pure suites pass: 48 tests. They include existing skill creation, retry, schema and authorization controls. - The source-owned serializers and protocol fixture validator regenerated the inventories without an app or native build. - Both generated inventories change only create_skill description. Arguments, required claims, modes, idempotency and all other tools remain byte-equivalent. - Current head `bca441726a6cba80e6e7da0321db59e27183c5b8` passes required `ci / verify` and `ci / e2e`; the full main CI run has 49 successful jobs. Windows, Docker, security and review checks also pass or are intentionally skipped. Greptile and the final Paperclip review are 5/5, with zero unresolved threads. This PR remains deliberately unmerged for the separately reviewed evaluation candidate. - Original head CI preserved: one unrelated UI clipboard-confirmation assertion failed after the clipboard call passed; 47 other executions succeeded and the final verify check failed as its dependent. The two UI owners are unchanged from the base. Exact React scheduler/timer timing is unknown. No unchanged-head retry was used before the necessary master conflict sync. - The conflict sync preserves incoming task-assignment guidance and regenerates metadata. It is not a fix for the UI test failure. The synchronized current head passes CI and review. No same-head recovery was consumed. ## Risks The description changes model context and the catalog digest. It may affect tool choice, but actual same-request and same-model acceptance is still pending. The change does not alter authorization, API behavior or grading. Local checks did not run the app, database, native runner, full graph typecheck or build. Intermediate test-fixture claim and placement mistakes were corrected; the final focused run passes. Do not treat these pure controls as a feature pass. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through Codex, with reasoning, source inspection and tool execution. The host does not report a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs; narrow create_skill/import and skill-import-guidance searches found no duplicate. - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:27:50 -06:00
3 people	a497bae83a	test(e2e): verify Cloud workspace access isolation (#18655) ... ## Thinking Path > - Paperclip keeps agent work inside company boundaries. > - Cloud users switch workspaces through the tenant account menu. > - A company-bound credential must work inside its permitted company. > - A missing resource cannot prove an access denial. > - This journey uses real controls in two registered test workspaces. > - It checks reads and task creation in both directions and revokes both keys. ## Linked Issues or Issue Description Refs: #18728, #18729, #18652. **What existing behavior does this improve?** Product E2E coverage for Cloud workspace selection and company-bound agent API access. **Current behavior** The journey catalog lacks a two-workspace boundary check with real allowed and denied controls. **Proposed behavior** Use the visible Cloud menu to switch workspaces and return. Read a real task and create one task in each credential's own company. Require real foreign reads and the same task-creation operation to return 401 or 403 in both directions. Verify that no foreign task was created and that protected content was not returned. Revoke both credentials and require their earlier allowed reads to return 401. ## What Changed - Add a browser journey and a narrow target-provider fixture contract. - Require independent owner reads, exact agent authorship, zero-to-one task creation, stable run history, no unexpected work and no foreign creation. - Validate observed workspace entry links against the exact registered targets. - Add wrong, missing, leaked, foreign and invalid-entry evidence controls. Saved passes reject collected runs or nonempty model usage. - Register all seven required checks, definition-hash inputs, feature associations. The feature-map documents are unchanged, as requested. - Base the final tree on verified repository master `7b8aa60b6d79b6cd9e4d46ca1369515c33e2d5d4` plus only 15 access journey and catalog files. Preserve all other master files and modes. The held completion journey is outside this PR's diff. ## Verification - 851 catalog, journey, completion-helper, artifact and browser-action support tests passed across 40 files. - E2E TypeScript, credential-free discovery, feature-map validation, full workspace recursive typecheck, full build and `git diff --check` passed. - The broad local `pnpm test:run` attempt stopped in chat integration shard 1 before any assertion. Embedded Postgres initialization exited with code null. No chat tests or later unit shards ran. The startup cause is unresolved, the failure is retained, and the command was not retried. The database/chat/test-runner source is unchanged from the fixed master snapshot. The separate hosted full CI result is recorded below. - The first local E2E typecheck read stale compiled plugin SDK declarations after the merge. The SDK source already declared the field. Rebuilding its declarations cleared the diagnostic without a source edit. - The complete tree comparison proves exactly 17 changed paths before the documentation removal against the fixed master snapshot. Every other master file and mode is preserved. All four access flow, contract and test blobs match the previously passing source `fab9a1fb2fefb41a7248e4fc1aa239d77455f5e2` byte for byte. - The historical staging attempt on source `fab9a1fb2fefb41a7248e4fc1aa239d77455f5e2`, the earlier Core42 deployment and the earlier adapter passed all seven checks: owner resource controls, visible Cloud switch and return, scoped data, both restricted-agent positive controls, cross-workspace read/task-creation denials, credential revocation and no agent work. Both credentials were revoked, owned credential agents had terminal receipts, and evidence cleanup was complete. No agent or model work was recorded. - The earlier attempt failed before journey checks while reading the borrowed fixture agent's historical details. That failed result remains preserved. The provider then supplied a separately owned primary identity. The public history assertions remain unchanged. - The strict root catalog expectation first failed because it omitted the new access case. Adding only that expected entry made the test pass. The ordered list and default-selection assertions remain unchanged. - The first combined-head full CI run failed two unchanged attention-retention tests at their original 15-second limits. The catalog fix does not change those tests or their timing. The initial failures are retained, their cause is unresolved, and that failed run was not retried. - Normal full CI passed on the new commit in [run 41595755161](https://github.com/paperclipai/paperclip/actions/runs/41595755161), attempt 1: 48 successful jobs and one intended skip. The two unchanged attention-retention cases passed at about 12 seconds; the earlier timeout cause is still unknown. The fresh review on commit `4c0ea682444738ad6fa17017a6afd25a49037559` is 5/5 with no open findings. The fresh staging result is recorded below. This PR changes no daily plan, private configuration, model or live workspace. - Fresh staging qualification on commit `4c0ea682444738ad6fa17017a6afd25a49037559` passed all seven checks and all five phases. Both ordinary same-workspace task creations returned 201. Both cross-workspace task creations returned 401. Both credential agents reached terminal state. Both keys were revoked, and both revoked-key reads returned 401. The manifest records complete cleanup and no active operations. No model or task run was recorded. All three workflow jobs passed. - The coordinator verified TestOps case 719 and result 3761 as passed, then closed launch 1987. Its report shows model none and the current qualification run. The automatic Slack report recorded one pass and zero failures. - The original normalized result was passed. The first evidence collector left its own qualification false because its allowlist omitted the standard `dashboard.html` artifact. The adapter already declared that file, and the public launcher generated it. An exact-object read matched the 118,791-byte object, ETag and SHA256 archive receipt. This completed the three-artifact inventory. The original result, collector proof and seal remain unchanged. No HTML was executed or rendered, and no test was rerun or regraded. | Fresh staging provenance | Verified value | | --- | --- | | Public commit | `4c0ea682444738ad6fa17017a6afd25a49037559` | | Qualification workflow | `41597774599`, attempt 1 | | Serving Core commit | `09162fe893b464dd2ccdafd14bd3ef1611202a86` | | Target adapter commit | `7367cf6fd0ee1cf52c2651a8a6671e087728a9c1` | | Actual workflow bootstrap commit | `55351ffba2704a4e13c1ead3e50665acd51aef87` | | Exact 117-file QA contract SHA256 | `14d95c29059c82563ab0e1d65a01aa4ac6c4f60f77a236803cf06ad5d9e7d27b` | | Provider-pack configuration version | `9` | - At the user’s request, follow-up commit `7e0dbc4bdba8a25ff6486648b0ca0d0db2616a4c` removes both feature-map additions. The final diff contains 15 paths and no `maps/feature-map/` changes. An entire-tree comparison against the QA-passed commit proves these two Markdown files are the only changes; every executable file, test, fixture, registration and file mode is unchanged. `node scripts/check-maps.mjs` and `git diff --check` pass after the removal. The live QA run above was not repeated for this documentation-only change. Fresh current-head full CI passed in [run 41633385961](https://github.com/paperclipai/paperclip/actions/runs/41633385961), attempt 1. The current-head Greptile review is 5/5 with zero unresolved threads. The normal merge preserves every newer master file outside the 15 PR paths and keeps all 15 PR files byte-for-byte intact. ## Risks The private target provider must register two exact test workspaces and preserve their separate ownership and cleanup. The case verifies company-bound agent credentials. It does not certify human membership roles. It tests task creation because ordinary agent task edits require a real execution context. No execution context is forged and no product authorization guard changes. The local broad unit run could not start embedded Postgres, so it does not establish a full local unit pass. The earlier hosted attention-retention timeouts also remain unexplained. Passing a later run would not prove the source of those timeouts. The four case files retain their historical passing bytes. The fresh qualification verifies the executable test code retained in this PR against the deployed staging stack. Daily admission remains a coordinator action after the normal source merge. The case does not cover future deployments without another run. ## Model Used OpenAI GPT-6 through Codex with reasoning and code execution. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com>	2026-11-03 06:24:08 -06:00
DottaandPaperclip	b22ef22d56	fix(runner): show provider failure details on tasks and runs (#18622) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner starts providers and records their run outcomes. > - Provider failures can contain useful details below a generic protocol error. > - Those details were lost at the sidecar, command receipt, and run projection boundaries. > - A blocked task therefore showed a generic failure even when the provider gave a clear reason. > - This pull request carries a bounded, sanitized provider failure through those boundaries. > - Operators can read the actual cause on the task and run pages. Existing recovery policy still controls retries. ## Linked Issues or Issue Description Refs #17543 for the related closed JSON-RPC classification work. Refs #17272 for provider recovery in the legacy OpenCode adapter. This change adds display evidence for native Runner providers and keeps recovery policy separate. **What happened?** A native Claude session returned an HTTP 429 message below an ACP `Internal error`. The task and run showed only a generic startup failure. Operators had to inspect the provider response to learn that the account was rate limited. **Expected behavior** The task and run show the sanitized provider message, provider name, startup or turn phase, and available HTTP status. Unknown categories keep the message. Display evidence does not grant a retry or replace a recovery decision. **Steps to reproduce** 1. Start a native Claude run whose account is rate limited during model selection, or return a structured provider failure from a Codex or OpenCode turn. 2. Let the run fail and open its task page. 3. Before this change, the page shows a generic failure. With this change, the page shows the provider explanation. The run page shows the same explanation after reload. **Paperclip version or commit** Source checkout based on `f8a888a072`. ## What Changed - Add one JSON Schema for `ProviderFailure` and generate matching Runner and shared TypeScript readers. Keep the Runner package independent of the app workspace at runtime. - Normalize known ACPX, Codex, and OpenCode error fields. Preserve unknown messages. Redact known credential values and credential patterns before applying the 4,000-character limit. Do not copy response bodies, headers, stacks, or prompts. - Carry startup failures through the sidecar, Rust Runner, durable command receipts, and private transport error details. Keep provider text out of generic transport error strings and Sentry failure messages. - Keep turn failures in authenticated run events. Recover the same evidence from persisted events during server finalization. Store the cause in the company-scoped run result. - Show one shared notice on failed tasks and run details. Keep existing retry permissions, scheduled retry state, and resolved-task behavior. - Reapply registered run-secret and current-user redaction to task display data. Batch secret lookups for only the visible provider causes. - Document the contract and update the capability maps. Add regressions for nested Claude 429 errors, OpenAI quota and rate limits, OpenCode errors, redaction, durable receipt replay, restart finalization, and UI rendering. ## Verification - `pnpm -r typecheck` passed. `pnpm build` passed. All GitHub Actions build, test, package, and validation jobs passed on `d49c79ab4a`. The security checks passed. Greptile gave this head 5/5 with no open review threads. Paperclip Review remains pending. Its review worker exhausted six workspace recovery attempts with export_interrupted. A fresh review request was accepted and is waiting for that saved workspace. The ordinary Retry action rejected reuse of the stopped request. This is an external review-worker blocker; no check was overridden. The duplicate local `pnpm test:run` was stopped after the CI test shards passed. - Focused server settlement, provider evidence, and restart tests: 93 passed. Runner admission and normalization tests: 103 passed. Focused Runner wire and terminal tests: 29 passed. The real Rust/sidecar command receipt replay test passed. - The new projection test failed before implementation because the task had no provider cause. The startup producer regression also failed before extraction moved ahead of the generic error-code replacement. Both pass with this change. - Live check: a rebuilt local sidecar received an actual Claude HTTP 429 during startup. The new cause kept the full provider message. A synthetic task in an isolated local app then reached the same provider error through the native Runner path. The dedicated run result, task API, task banner, and run detail all showed the same message after reload. The later reporting fix keeps run.error generic; a regression proves the display cause still survives restart and stays out of Sentry capture. - The first isolated app attempt failed earlier in `admission_credential`. An ordinary manual Retry reached the provider and produced the 429 above. This PR does not claim to fix that separate admission failure. An initial parallel local database test setup also failed; the serial restart suite passed. - Security review found that copying display text into run.error could forward it to Sentry. A regression failed with that behavior. The fix keeps the cause in its display field. All 38 settlement/restart tests and the server typecheck pass. - All 29 protocol contract tests pass, including bounds and extra-field rejection. An earlier full Rust core run passed. The later full run hit one 5-second receipt-limit timing assertion; its isolated rerun passed. The full Linux and Windows Rust CI jobs passed on this head. - Review regressions pass for explicit Codex driver identity, native Rust startup and resume receipt propagation, oversized OpenCode/ACPX errors, task read-time registered-secret redaction, and the standalone source closure. The focused Runner run passed 103 tests; projection/restart tests passed 16. - Read-time redaction edge cases passed: a missing decryption key omits only the unsafe optional explanation; expanded redacted messages remain visible within the Unicode character limit. All 14 task projection tests and server typecheck pass. - The package Build check found an old Pi assertion against Error.message. The probe now checks the exact closed classification in the authenticated receipt and requires the new provider cause. Both polling transport cases pass. A local credential-free Pi probe stopped at admission_installation because this checkout lacks the Pi distribution; the installed Linux package probe passed in CI on this head. - UI coverage includes startup and turn text, unknown categories, literal rendering of provider text, read-only retry behavior, and resolved-task suppression. Token gates and map checks passed. - The live check covers Claude startup on macOS. Codex, OpenCode, terminal failures, and restart paths have deterministic regression coverage. The change is not deployed. ## Risks - Provider messages are untrusted text. The boundary selects specific fields, redacts credential patterns and known environment secret values, bounds the message, and renders plain text. New provider-specific secret formats may need added redaction patterns. - Optional fields preserve compatibility with older producers and stored runs. Old runs without the field keep their current fallback text. No database migration is required. - The cause stays in company-scoped run data. This change does not send provider text to first-party Telemetry or add it to OpenTelemetry span attributes. - Normalized categories and retryability are display evidence. Existing recovery classifiers, retry limits, ownership checks, and approval gates keep their authority. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser verification. The host does not expose the exact serving variant or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:13:26 -06:00
DottaandPaperclip	718336eb04	fix(native): refresh prior warm controller projections (#18718) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native remote sessions keep their controller state in a database checkpoint. > - An admission can also leave a full local copy of a prior run's state. > - The prior owner can commit more state without changing its run ID. > - Warm admission skipped a refresh when the local and saved run IDs matched. > - This pull request refreshes that prior-run copy when no retained warm owner exists. > - The next turn can use the committed state while existing ownership checks remain effective. ## Linked Issues or Issue Description **What happened?** A prior warm run can complete after an earlier admission reads its controller checkpoint. The local copy can still contain a pending command or lack the final terminal event. A later admission sees the same run ID and skips restoration. It can then hold work against that old copy even though the database checkpoint has settled. **Expected behavior** Read the canonical prior-run checkpoint before reuse when no retained in-memory owner exists. Keep current-run state with its controller. Keep unresolved canonical work and invalid checkpoint bindings blocked. **Steps to reproduce** 1. Create a full prior-run local projection while a command is pending. 2. Complete that command and record the terminal in the canonical checkpoint under the same run ID. 3. Start an ordinary remote warm continuation without a retained in-memory owner. 4. Before this change, the local command remains pending. With this change, the completed command and terminal replace the stale copy. **Paperclip version or commit** Reproduced on master `36df8933f0` before changing production code. Related work: #18018 restored per-turn checkpoints; #18343 added controller replacement recovery; #18554 restored missing remote suspension receipts. This change handles a full prior-run projection whose run ID already matches the saved supervisor. It does not change the settlement guard or MCP compatibility policy. ## What Changed - Refresh a prior-run warm controller projection even when its run ID has not changed. - Keep current-run and retained warm owner authority on the existing path. - Add five executor cases for a settled source, a pending canonical command, a current run, legacy file-backed state, and an invalid checkpoint binding. - Add an opt-in Linux continuity case with a stale full local projection and two follow-up turns. - Update execution documentation and the paired session verification recipes. ## Verification - RED before the production edit: the executor fixture retained `pending` in its local command record when the canonical record was `completed`. - GREEN on the final commit: 48 focused tests passed across executor restoration, retained warm owner rotation, per-turn restoration, and remote settlement guards. - `pnpm --filter @paperclipai/server typecheck` passed. The server TypeScript check also passed after the integration fixture was added. - `node scripts/check-maps.mjs` and `git diff --check` passed. - The unit fixture uses the real durable controller serializer and the real executor entry. It stops at current tool authority before provider/backend creation. It proves restoration and guard behavior, not full task acceptance. - Actual protocol RED/GREEN: the same opt-in continuity fixture uses real PostgreSQL, TLS, the current Linux Rust Runner, and a deterministic fake Codex backend. With only the executor restored to its parent revision, continuation fails with `prior_owner_terminal_state_indeterminate` in the settlement guard after one provider turn (20.16 seconds). The PR source completes all three turns with one provider thread and final suspension (34.77 seconds). Hard recovery permits one provider-process replacement; the healthy final follow-up starts no additional process. Both runs use the same Runner binary and owned container. - The exact PR executor source was restored after the baseline comparison, its Git blob was verified, and the worktree is clean. The disposable local container was removed. This fixture proves the controlled stale-projection recovery path; it does not establish a live model or Daytona API result. - Full current-head CI passes, including typecheck, build, general and serialized tests, Rust/Runner checks, all eight E2E shards, and the `ci / e2e` and `ci / verify` aggregate gates. Greptile is 5/5 on the current commit, with no unresolved review threads. No original task was retried and no staging sandbox lifecycle was changed. ## Risks - Admission without a retained owner performs one additional scoped checkpoint read when a full prior-run projection already exists. - The canonical checkpoint must still pass its company, task, session and record validation. A pending canonical command stays held. Current-run and retained owner state are not replaced. - The regression proves a stale local projection defect. It does not establish that a historical incident had those same local bytes. ## Model Used OpenAI Codex, GPT-6 family. The harness does not expose the exact serving model ID or context-window size. Capabilities: repository inspection, code generation, tools, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:12:22 -06:00
DottaandPaperclip	db8a273562	fix(native): publish fresh retry sessions for later turns (#18713) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native tasks save a provider session for later comments and monitor checks. > - The goal projection also records the run that owns the session event sequence. > - A fresh retry uses a new native session, but the projection accepted only the old session. > - The accepted checkpoint then rejected session publication because the goal source still named the old run. > - This pull request lets the current task executor advance that source after a terminal predecessor with no unfinished goal. > - Later turns can then use the fresh retry's saved session. ## Linked Issues or Issue Description Related: #12669, #18343. I searched open and closed PRs for source epochs and native task-session publication. I found no duplicate repair. **What happened?** A successful fresh native retry could retain its predecessor's task-session pointer. A later comment or monitor selected the old provider session. The goal source transition required both runs to have the same native session. The terminal checkpoint then saved a `suppressed_newer` publication result. **Expected behavior** The current task executor can publish its accepted fresh session. A later turn resumes that session. Delayed events cannot restore the old source or revive a cleared goal. **Steps to reproduce** 1. Complete a native turn and retain its task-session row. 2. Admit a fresh session on the same task without deleting the row. 3. Complete that turn, then send an ordinary follow-up. 4. Before this change, the saved pointer can still name the first session. **Paperclip version or commit** Reproduced on source commit `36df8933f0` with a PostgreSQL regression before the production change. **Deployment mode** Source checkout. The affected producer runs for local and remote native sessions. ## What Changed - Permit a different native session to advance the goal source only when its run holds the task execution lock and assignee, and the task is in progress. - Keep the terminal predecessor, native runtime, runner identity, absent or completed goal, source sequence, and conversation reset checks. - Test accepted session publication, a fresh heartbeat followed by an ordinary resumed turn, stale events, current-owner loss, and unfinished or unreadable goals. - Add an opt-in live Codex file-persistence check. Update execution documentation and both recovery maps. ## Verification - Red: the new PostgreSQL publication regression failed before the producer change. The saved session, display ID and goal source still named the predecessor. - Green: 72 distinct tests passed across the final focused runs of `runner-goals.test.ts`, `native-task-session-publication.test.ts`, `native-session-resumption.test.ts`, and `remote-warm-session-recovery.test.ts`. - The assembled heartbeat test admits three turns, opens two provider sessions, resumes the second session once, and stores three results. It uses a deterministic provider backend. - `node scripts/check-maps.mjs` and `git diff --check` passed. - Real local Codex acceptance passed on macOS arm64 with `gpt-6.1-sol` and explicit warm mode. Three heartbeat turns saved three exact file lines. The fresh second turn published its new normalized and provider session IDs. The ordinary third turn retained both IDs. The fixture closed its warm owners and removed its private credential copy and workspace. The production code was at `450e15e`; the final test-only commit selects explicit warm mode and uses the existing owned-home cleanup helper. The production change is identical in the final head. - The earlier per-turn fixture preserved all three file steps and selected the correct normalized session, but provider-thread reuse failed. `run.attach` reported that the Codex provider had pending events; the safe fallback used another provider thread. This remains a separate failure. The first fixture cleanup also failed on immutable instruction assets. The owned-home helper fixed cleanup, and the second per-turn attempt exposed the provider identity assertion directly. - Full current-head CI passes, including typecheck, build, general and serialized tests, Rust/Runner checks, all eight E2E shards, and the `ci / e2e` and `ci / verify` aggregate gates. Greptile is 5/5 on the current commit, with no unresolved review threads. Broad local workspace checks were not duplicated; the focused regressions and actual provider acceptance above were run locally. No merge, deployment, or recovery of an original task is included. ## Risks - Source epoch changes affect saved goal state. The fresh-session path requires current task authority and refuses unfinished or unreadable saved goals. - The change starts no provider work and changes no prior-owner recovery guard. Unknown external outcomes retain their reconciliation hold. - A prior session remains held when it lacks the required authority or state proof. - Per-turn provider-thread reuse remains an open coverage gap described above. This change does not alter that attachment guard. ## Model Used OpenAI Codex, GPT-6 family. The exact model ID and context window are unavailable in this session. The agent used code inspection, tools, and executable tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:12:00 -06:00
DottaandPaperclip	89e3c23062	fix(daytona): stream files when export copies cannot fit (#18712) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native task must save its files before finalization can release its writer. > - Remote workspace capture can produce an archive that fits on the sandbox disk. > - Daytona then makes another copy before it downloads that archive. > - The second copy can run out of space and leave completed work behind a save hold. > - This pull request uses the existing bounded stream when that copy cannot fit. > - The benefit is a saved archive without a second archive-sized sandbox allocation. ## Linked Issues or Issue Description Related: #18644 reduced capture scratch space. #18172 added the disk-free directory export. #18705 preserves provider checkpoints on a full disk and reports capture failures. This change handles the later single-file download copy. It does not depend on #18705 and does not replace its checkpoint fix. **What happened?** A native provider turn returned a successful result and captured its workspace archive. The Daytona outbound file guard then failed with exit 43 and `No space left on device` while it copied the archive to a reserved download file. Finalization and file-preservation retries could not pass this boundary. The workspace kept its writer hold. **Expected behavior** Download an existing archive through a confined sandbox process when its second copy cannot fit. Keep the source and all finalization holds until the normal save commits. **Steps to reproduce** 1. Capture a workspace archive in a Daytona sandbox. 2. Leave less free space than the archive needs for another copy. 3. Finalize the task. The outbound single-file guard fails before download. **Paperclip version or commit** Reproduced against `6d72243d8a`. The incident used the native sandbox runtime. This branch is based on current master. **Deployment mode** Cloud with native Paperclip Runner execution and remote workspace storage. ## What Changed - Remove partial outbound file copies after a failed snapshot command. Only copy capacity failures select the stream path. Permission, confinement, and transport errors still fail. - Use the existing acknowledged terminal stream to read each file as the sandbox user. Recheck canonical confinement and regular-file status. Use a no-follow open. Keep the sandbox disk limit, host free-space reserve, shared transfer deadline, and bounded teardown. - Keep host files in private temporary files until the complete batch has transfer receipts. Never pass a mutable live source to the privileged download API. - Add real-file and real-terminal regressions, a disposable Daytona acceptance case, and workspace verification notes. ## Verification - Red before repair: a real 5 MiB copy under a controlled file-size limit failed with exit 43. The same test passes after repair and compares every source and destination byte. - The ten capacity cases pass. They cover producer failure after output, source replacement, stream size limits, permission errors, timeout teardown, cleanup, and a successful retry. A two-file case completes the first stream and fails the second. Both old host files stay intact, both source files survive, and retry saves both files. An isolated early-publication mutation makes that test fail. - The initial complete Daytona package suite passes: 537 tests passed, 8 skipped. After the test-only review follow-up, all 60 focused transfer, stream, and diagnostic tests pass. The package typecheck also passes. - Real Daytona acceptance passes: an owned Ubuntu 24.04 sandbox stops and resumes, a copy to `/dev/full` reports ENOSPC, production export saves the exact archive hash, and a second save after reconnect passes. The test deletes only its disposable sandbox. This is provider-boundary acceptance, not a UI journey. - Assembled finalization acceptance passes with real Postgres and a live Linux x86_64 Daytona sandbox. The production control-plane port accepts one fixture result. The production workspace finalizer encounters the failed copy, saves the files, and commits the task as `done`. A new database client reads the committed result and exact file hash. The workspace has no writer or running save, and there is still one run and one result. No model turn runs. The target resolver is bound to the live provider fixture. Object storage uses the production encryption and local-disk service with an S3 admission label; this does not prove an S3 network transfer or a UI journey. - The live tests used the fixed transfer source with SHA-256 `857b3d8c9ad75170707d95277c17c051ce4582fa79a9c38331d51d2800b36af5`. That source is identical in this PR head. - `pnpm -r typecheck` passes locally. The duplicate full local test run passed all six chat shards before it was stopped during the server lane in favor of completed current-head CI; a duplicate local build was skipped. Full current-head CI passes, including typecheck, build, general and serialized tests, Rust/Runner checks, all eight E2E shards, and the `ci / e2e` and `ci / verify` aggregate gates. - Greptile is 5/5 on the current commit with no unresolved review threads. - `node scripts/check-maps.mjs` and `git diff --check` pass. Map checks prove structure only. ## Risks - The fallback requires the Bash and GNU `dd`/`base64` tools already required by the existing stream path. - Terminal streaming is slower than the normal bulk download. It runs only after a copy capacity failure. - The workspace capture itself still needs sandbox space. This change does not fix a full disk before an archive exists. - The test injects a failed copy sink. It does not exhaust a shared host disk. No original task was retried, and no deployment or merge is part of this PR. - No schema, API permission, writer ownership, or retry-budget change. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, shell tools, browser diagnostics, and automated tests. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:11:41 -06:00
DottaandPaperclip	1bc3ed6dd5	fix(opencode): keep native tool calls pending until Runner settlement (#18711) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native OpenCode sends semantic tool calls through the durable Runner protocol. > - The Runner retains a call until its result is known. > - OpenCode and its HTTP bridge each stopped waiting after 30 seconds. > - A slow response could leave a call pending after the provider had moved on. > - A later run then failed the pending-call guard during attachment. > - This pull request keeps native calls under their Runner owner until settlement or cancellation. > - The benefit is one delivered result and a safe next run. ## Linked Issues or Issue Description Related: #18343 applies the durable wait policy to native ACPX calls. This change applies that policy to native OpenCode. **What happened?** An assigned tool call took more than 30 seconds. OpenCode reported an MCP timeout while the Runner still retained the request. The source turn could finish, but suspension failed. A later `run.attach` failed because provider tool calls were still pending. **Expected behavior** The native proxy must wait for the authoritative result under the Runner owner. Stop, bridge close, and lease expiry must still end the wait. Unknown effects must still block attachment. **Steps to reproduce** 1. Start a native OpenCode run with an assigned semantic tool. 2. Hold the controller's result for 31 seconds. 3. Observe the provider's MCP result and attach a later run to the session. **Paperclip version or commit** Reproduced against `6d72243d8af0362d73fbcb5331fd42a384800d4e`. The branch is based on `36df8933f0`. ## What Changed - The native OpenCode proxy enables a driver capability for durable tool requests. - That capability removes the bridge's separate request timer and sets OpenCode's fixed MCP client ceiling to one hour, consistent with the native ACPX sidecar. - Ordinary driver calls keep their 30-second deadline. Assigned external MCP connections keep their existing budget. - Cancellation, result replay, tool authorization, and the pending-call attachment guard keep their existing contracts. - The native regression holds a result for 31 seconds, then completes a second run with the same Runner process and no pending provider calls. - The deterministic provider fixture honors the emitted MCP budget and retains the initial completion contract across ordinary follow-ups. - Execution semantics and the session verification maps describe the wait and its safety checks. ## Verification - Red before production edits: the bridge aborted a durable request at the ordinary 30-second boundary. The driver emitted `30000` instead of the expected `3600000` client budget. - The final combined bridge, driver, proxy, and native protocol run passed: 95 tests in five files (71.17 seconds). - The native delayed-result probe passed with the current Rust Runner and current bundled proxy. It completed the later run with the same Runner PID and an empty pending-call ledger. - Actual provider check passed on macOS arm64 with `opencode-ai@1.18.34`, the changed source, and a local deterministic OpenAI-compatible model. One assigned write waited 31 seconds and saved one file. Both turns completed. The second turn retained the saved tool receipt. The write ran once across five model requests. - Bridge safety cases cover result replay without another action, ordinary timeout behavior, MCP cancellation, and teardown of an unanswered durable request. - Both runner TypeScript projects passed typecheck. The first attempt lacked the eval-kernel build; it passed after building that declared dependency. - Map alignment, module boundaries, and `git diff --check` passed. - The first native follow-up probe failed because its synthetic provider forgot the initial completion-contract revision. This caused a repair turn and another discovery call. The fixture now retains the contract, and the same probe passed. The failure is retained here. - Runtime acceptance used the implementation at `583d316ad4`. The later clarification commit changes comments, documentation, and a test name only. It does not change runtime behavior. - Full current-head CI passes, including typecheck, build, general and serialized tests, Rust/Runner checks, all eight E2E shards, and the `ci / e2e` and `ci / verify` aggregate gates. Greptile is 5/5 on this commit, with no unresolved review threads. No deployment or retry of the affected historical task was performed. The actual provider check used a local model; it does not qualify a deployed Linux sandbox or a live model service. ## Risks - A native call can wait longer while its Runner owner remains valid. Stop, teardown, and lease expiry still end the wait. The MCP client has a separate fixed one-hour ceiling. Runner ownership can renew beyond that ceiling. If the client deadline expires first, the operation outcome remains unknown and the existing hold must still block replay and run attachment. - This prevents new premature timeouts. It does not classify or replay an old request whose effect remains unknown. - No schema, migration, credential, or permission change is required. ## Model Used - OpenAI Codex, GPT-6 family. The harness does not expose the exact model ID or context window. Capabilities used: code generation, tool-assisted diagnosis, local execution, and test analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:11:24 -06:00
DottaandPaperclip	dd02595390	fix(db): retain hosted census construction callers (#18725) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use a bounded database census to find expensive read paths. > - The hosted server combines application code and capture wrappers in generated files. > - Generic HTTP owners suppressed caller capture. Generated paths also failed the caller allowlist. > - This PR retains trusted query construction locations without source maps. > - Operators can distinguish application query sites while exact owners, privacy limits and capture bounds stay intact. ## Linked Issues or Issue Description Refs #17522, #17564. **What happened?** A real map-free bundle with two application query sites produces one callerless census group. This affects unowned queries and queries under the generic `http.request` owner. **Expected behavior** The census should distinguish trusted query construction sites. It must keep unknown calls unknown and preserve specific semantic owners without stack capture. **Steps to reproduce** Run the committed bundle test and pinned comparison helper. The helper bundles the actual capture module and real Drizzle, with a fake postgres-js client. It does not assign a fake path to an isolated mock caller. **Paperclip version or commit** Control: `09162fe893b464dd2ccdafd14bd3ef1611202a86`. The branch is rebased onto `1e3ab690b072bbf6a1c1f1d0454ea49315b2074c`; neither intervening commit changes this owner. ## What Changed - Keep the exact generic HTTP owner and add bounded caller capture only during the active census window. - Exclude capture wrapper frames with a local V8 stack boundary. Keep one static generated construction anchor for trusted root builders. - Keep unscoped bundled driver and relational paths unknown. Their first frame can belong to the ORM. - Cover exact application lines, alternate parameters, prepared builders, transactions, nested savepoints, privacy and inactive windows in a real bundled scaffold. - Add a portable pinned comparison helper and retained measurements. Existing group, page, byte, expiry and output-rate caps remain in force. ## Verification - PASS: 54 focused unit, real-bundle, existing runtime bundler and real-PostgreSQL tests. - PASS: database TypeScript check. - PASS at final head `b270b92821`: full local `pnpm -r typecheck` and `pnpm build`. - The first local `pnpm test:run` was stopped after two chat partitions passed (668 cases), because source changed during the run. This is partial evidence only. The fresh final-head full local command passed all six chat partitions (1,829 cases). Its general phase reported one failure in the unchanged workspace port-conflict test. The exact case then passed alone (1 pass, 178 skipped), and the unchanged full file passed 179/179. The original failure did not reproduce; its cause is unknown. The general phase was stopped after its remaining serial queue was measured at roughly three hours. Cancellation did not print the original assertion diff. The full command is failed/incomplete, not passed. Remaining general files, workspace-project phases and 164 serialized files were not reached. Fresh final-head CI passes all partitions. - Final-head code CI passes. Greptile is 5/5 with its thread resolved. Paperclip Review is delayed by an external review workspace recovery failure; it has not completed on this head. - Five serial alternating rounds per mode, 1,000 queries per sample, 40 retained samples, Node v24.22.2. Generic and unowned attribution rises from 0/1,000 to 1,000/1,000 across two exact application anchors. Specific owners stay stack-free. - HTTP median wall cost is 10.528 → 15.169 µs/query; process CPU is 15.461 → 17.662 µs/query, with noisy overlapping ranges. This is census overhead, not a product speedup. - Review failure retained: the existing runtime bundler test first failed because the public parser accepted arbitrary generated frames. The fix restores default rejection and permits generated frames only from the trusted builder boundary. The unchanged existing test now passes. - Full ranges, raw samples, proof boundaries and reproduction commands: https://github.com/paperclipai/paperclip/blob/b270b92821ae47c456a1b61d78c1593c5348ede7/doc/performance/2026-11-03-db-capture-hosted-callers.md ## Risks - Active generic HTTP census now captures stacks. More groups can reach the existing 2,048-group cap sooner. - Anchors identify construction sites in one exact generated build. They do not identify later async consumers, TypeScript lines or CPU owners. - Unscoped driver calls can remain unknown. This is intentional. - The no-network local fixture excludes deployed Sentry/OTel stack formatting and traffic effects. It does not establish exact hosted overhead. - Wrappers remain installed after expiry. Disabled, future and expired windows do not capture stacks. No schema, authorization, provider, or query behavior changes. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code execution and repository tools. The host does not expose the context-window size. Independent review used the same model family. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:04:42 -06:00
DottaandPaperclip	bf7f77d8cd	fix: avoid response recovery self-contention (#18715) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server must present accepted agent responses as comments under current chat authority. > - The recovery worker can check four tasks at the same time. > - Different tasks can share a chat endpoint or external principal. Their authorization transactions then compete for endpoint or identity locks. > - A failed transaction leaves the accepted response pending and repeats its source queries on a later retry. > - This pull request groups checks that share a task, endpoint or principal before dispatch. It keeps every transactional authority check and lock. > - The benefit is less failed recovery work and prompt, exactly-once presentation of valid accepted responses. ## Linked Issues or Issue Description Refs #17714 for a separate publication-commit lock-order fix. Refs #18686 for separate run-list JSON work. Neither changes this recovery admission path. **What happened?** A response-presentation worker reported `could not obtain lock on row in relation "chat_endpoints"`. Four real PostgreSQL recovery fixtures reproduced three such failures in one pass with no external ingress. The worker grouped by task, while authorization locks shared endpoints and principals. Review also exposed two valid Slack endpoints in one workspace that share one principal and compete for its try-advisory lock. **Expected behavior** Recovery checks that share an endpoint or principal should retain task order without competing with one another. Unrelated endpoints should make progress. The server must still reject stale authority and retry legitimate external contention. **Steps to reproduce** Run the shared-endpoint regression or the portable comparison helper linked below. The fixture creates four accepted historical responses for distinct tasks and principals on one actual endpoint. ## What Changed - Read optional endpoint and principal scheduling hints through a company-scoped join to the canonical wake action. The join does not change eligibility or grant authority. - Extend existing bounded check groups to merge intersecting resource keys. Preserve task order, transitive conflicts, independent-group progress, shutdown, drain and pending scopes. - Add real PostgreSQL controls for shared endpoints, a principal shared across two endpoints, an observed blocked transaction with independent endpoint progress, external locks, exact selected output, and endpoint revocation despite a forged hint. - Add a portable helper that materializes the pinned baseline with `git show`, runs the existing real fixture, retains results, and restores temporary files. ## Verification - Final principal-extension head `57047441b28765ed1cf49b8924efc69f5c03dcb4`: 34 focused scheduler/chat controls passed on Node 24.22.2 / pnpm 9.15.4. Independent exact-head source review is green. Full local `pnpm -r typecheck` and `pnpm build` pass on this head. Fresh Greptile and Paperclip reviews are both 5/5 with no findings. All 59 final-head contexts are complete: 55 success, 3 skipped, and the neutral Paperclip review at 5/5. There are no failures or pending checks. - Earlier endpoint-only head `eea437e7edbe7eb41eb880d4d8fd6d99a9c247da`: full local typecheck/build and all remote CI passed. The redundant local full suite was interrupted for this review fix after 956 passing chat cases; it is not a full local pass. - Final shared-principal counts-only comparison completed the same two exact responses: product calls 165 → 147, rows 114 → 100, outer contention failures 1 → 0. SQL query errors stayed zero; advisory-lock refusal is not a failed SQL query. - The earlier endpoint-only study completed five alternating pairs / ten measured runs: median calls 443 → 293, rows 314 → 200, query errors 6 → 0 for four exact responses. Its instrumented shared-host CPU data predates the principal extension and is not a final-head or hosted CPU claim. Retry clocks were accelerated; wall time is not hosted latency. - [Raw samples, final principal work counts, portable commands and failure accounting](https://github.com/paperclipai/paperclip/blob/57047441b28765ed1cf49b8924efc69f5c03dcb4/doc/performance/2026-11-03-response-presentation-contention.md). - Initial server4 CI reported three provider calls instead of two in the unchanged Anthropic identity fixture. The exact local case passed; the full 88-case file passed with observations showing the two expected identity-route calls. The third CI call's origin remains unknown. One bounded rerun of the failed job and its dependent CI aggregate passed. No assertion or source change was made for that failure. - The causal benchmark baseline worker/query is `82c033ed127d07862e6765aa4b69a3cf339bde81`. The PR base is `36df8933f0`. They are not claims about the currently serving revision. ## Risks - Connected resource groups can reduce parallelism when tasks link several endpoints. This preserves the existing task ordering contract. - Missing or stale endpoint/principal hints retain current transactional checks. Other workers and ingress can still cause legitimate NOWAIT retries. - There is no schema, API, permission or provider-transport change. Endpoint revocation and company isolation remain enforced by their existing owners. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), with tool use and code execution, produced and reviewed this change. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:04:25 -06:00
DottaandPaperclip	76ee5d0886	fix(evals): preserve singleton reports and pin reviewed chat repairs (#18749) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat eval lane runs selected cases from one immutable candidate. > - The finalizer must authenticate every original projection and publication. > - The pinned artifact action places one matched artifact at the download root. > - The finalizer expects named child directories, so a valid one-case run fails before final coverage verification. > - This PR supports both valid layouts and binds one reviewed repair candidate. > - The lane keeps exact selection, source, digest and publication checks. ## Linked Issues or Issue Description Refs: #18740, #18736, #18742, #18734, #18717. This PR is a proposed replacement for the source pin in #18740. That PR remains open. The evaluated candidate contains all five repairs from that prior cut and five further reviewed repair conditions. The product PRs remain unmerged. Merge authority for this replacement is pending; this PR does not dispatch a run. A completed one-case run has a verified original PASS and a verified publication. Its final aggregate fails because the pinned artifact action flattens a single matched artifact. The same path assumption also affects the later publication verifier. The final checks must still run; the repair must preserve their identity and digest checks. ## What Changed - Bind the finite candidate `ab2019e976076ec0eb60b026844f3d9211cb0208` through the manifest and exact descriptor snapshot. - Retain the original 244 case identities and the explicit cumulative definition overrides. Keep the historical definition source separate from the current candidate. - Add the reviewed source dependencies for real hire-review readiness and the strict later-human-intent review overlay from unmerged #18742. Preserve all original clocks, queries and model settings. Keep the earlier Studio144 accounting change explicit. - Document one current target and identify the prior Source5 cut as unmeasured. - Preserve named singleton download destinations for both bundles and publications. Authenticate every bundle before gathering its safe projection. Reject mixed, foreign, missing and symbolic-link layouts. Accept only the producer's exact closed once-marker schema. Do not export or upload results again. ## Verification - Candidate controls: 217 pure checks in 13 files; 242 real factory declarations with zero fixture callbacks; 42 admission controls; unchanged 506-input SDK registry; focused actual database-row type check. - Combined trusted pin and layout controls: 124 Node checks in seven real files, plus source-byte closure for all 244 singleton evidence policies. The unchanged parent reproduces both layout failures. Actual producer-marker, multi-case and malformed-layout controls pass. - The two actual PostgreSQL route regressions on core #18742 head `61996a96dd09e90a5788e8711a94fc5ae631f425` ran and passed. That core PR remains unmerged. This is not chat-case acceptance. - The independent layout logic peer and the whole final eight-path PR source peer passed. Current-head CI completed with 48 successful jobs and one skip. Both required checks passed. All 60 exact-head checks completed: 55 success, two neutral and three skipped. The separate Windows Rust check passed. Greptile and Paperclip Review both scored 5/5, with no unresolved threads. - The optional raw-order note is [disposed with source evidence](https://github.com/paperclipai/paperclip/pull/18749#issuecomment-6598680442). The admitted selector writes canonical order before hashing. A changed raw order fails closed before copying. The checked code remains unchanged. - Full local candidate typecheck and build are not claimed. Installed workspace dependency gaps prevent that full local check. The existing remote source and build preflight must pass before a native query. - No new model run or TestOps upload was used to prepare this PR. Prior attempts and grades stay unchanged. ## Risks A permissive layout repair could hide missing or foreign publications. Exact selected names, singleton counts, source identities, hashes and final readback checks must remain enforced. The layout helper must reject symbolic links and mixed shapes. The active default-branch ruleset requires CODEOWNERS review for the workflow. Normal merge is currently blocked on that review. This PR does not enable auto-merge, change branch rules or dispatch an eval. The candidate contains explicit unmerged product overlays. It is not a core release or a full-catalog qualification. The other five SDK cases remain held. SDK115 completion remains unknown. Provider quota failures are not fixed by the skill observer change. No credentials, caps, runner permissions or workflow admission references are added. ## Model Used OpenAI `gpt-6.1-sol`, with code execution, tool use and independent agent source review. The host does not expose the exact context window. No model eval was run to prepare this PR. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub links) - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1103.0-canary.8	2026-11-03 06:03:25 -06:00
DottaandPaperclip	7b8aa60b6d	test(e2e): add repository continuation journey (#18729) ... ## Thinking Path > - Paperclip keeps agent work in project workspaces. > - A later user request must continue from the saved repository state. > - A successful reply alone cannot prove file or Git history preservation. > - The repository journey in #18652 passed against QA staging with independent byte and commit checks. > - Its separate completion journey remains unpassed and must stay open. > - This PR delivers only repository continuation and reuses the merged shared support from #18728. ## Linked Issues or Issue Description Refs: #18652. Builds on merged #18728. This is the repository-only delivery from #18652. The original completion implementation and all failed attempts remain on that open PR. ## What Changed - Add `repository-continuation` with the exact measured source fixture, two native turns, and eight required checks. - Start from measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189`. Keep the repository flow and Git/byte oracle unchanged. Clarify the two file-content instructions to require the word first or verified followed by exactly one LF newline; retain the exact expected bytes and the original random-reference contract. - Document the two-turn proof boundary: final saved hidden Git metadata is not independently read. Keep all eight success assertions. - Use the original-deadline and routine failure-reporting repairs from merged #18728. Integrate its actual master squash `e5713d95c6d9a258882649789ea5eb071cc17dc0` through one normal merge. - Use a fixed public read-only repository commit. Verify downloaded Git objects, parent commits, changed paths, saved workspace identity, and exact source bytes across both turns. Reopen the registered outputs through the browser. - Keep the shared artifact reader behavior unchanged and narrow its type to the API capabilities it uses. - Add only this case to the catalog, feature association, dispatcher, source hash, types, run/check contract, strict catalog expectation, and journey documentation. ## Verification - Current head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed fresh [QA staging run 41589522430](https://github.com/paperclipai/paperclip-cloud/actions/runs/41589522430) and its [full report](https://pages.paperclip.ing/evals-qa-smoke-41589522430-1/). All eight checks and five phases passed. Both native turns succeeded and committed in the same workspace. The independent same-buffer Git audit verified that the second commit has the first commit as its parent. The saved original files and browser downloads matched their expected bytes. This does not add a final hidden `.git`/HEAD assertion. - The fresh run used OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on QA staging/Daytona. It used Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`, Cloud `cloud-41570439002-1`, and environment configuration version 9. Actual checkouts matched public source `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31`, adapter `7367cf6fd0ee1cf52c2651a8a6671e087728a9c1`, and protected workflow bootstrap `a9ad6877de62fe3f65aacadc8a9dfba7d90729cf`. The bootstrap matched the accepted 117-file QA contract. - Normal cleanup completed. The original manifest is complete, evidence cleanup is complete, no operations remain active, and the owned agent is terminal. The coordinator verified ACTIVE TestOps case 685 / result 3695 and closed [launch 1921](https://paperclip.testops.cloud/launch/1921). This records normal cleanup closure; no separate physical provider-stop acknowledgement was collected. - Measured source `ac4efb7c6f4782ea806661683c656fd2b4cc3189` passed [QA staging run 41573331189](https://pages.paperclip.ing/evals-qa-smoke-41573331189-1/) against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch 1723 / result 3501 maps to case 685. All eight checks and five phases passed. The two run bindings, independent saved-byte/Git audit, and completed cleanup were verified. - Initial extraction `54103639850d08d4d125ebe0493d8d3d49b58211` preserved the measured case files. The current head also clarifies the two LF instructions, documents the final Git-metadata limit, and inherits the reviewed helper repairs. This changed the fixture conditions. The current head now has its own live pass, listed below. The original ac4 result remains a pass under its original conditions; it was not used to qualify this head. - The emitted-prompt control failed on the old quoted backslash+n instructions, while 14 sibling controls passed. All 47 focused repository/artifact controls then passed after the two-sentence repair. Local: 891 related support tests, runner E2E typecheck, recursive typecheck, build, maps, token gates, and `git diff --check` passed. The inherited routine source has 62 causal read/cleanup controls and 26 unchanged real component controls. - The initial repository head's normal full CI run 41578686494 passed. Its 4/5 review and all original findings remain recorded. The later stacked head `037b979477161881a4877230500e6f3de8faea5c` also passed full CI run 41581342503 and a fresh 5/5 review. Current integration head `86808b7ec17e1727bd55a3fd46bc7e0409ca0e31` passed normal full [CI run 41584981651](https://github.com/paperclipai/paperclip/actions/runs/41584981651), attempt 1. Its fresh exact-head Greptile review is 5/5, with zero unresolved threads and no open findings. The broad local suite was not repeated. The earlier full local run recorded in #18643 had four failures and did not reach 164 serialized groups; this PR does not claim an all-local pass. - Earlier repository failures remain recorded: provider-pack transfer timed out in one attempt; a later attempt could not read saved workspace files. The later pass does not regrade those failures. The product repairs are separate from this unchanged case delivery. - The fixed actual #18728 squash is now integrated. The PR targets master. Its aggregate diff has exactly 14 repository-only paths, each identical to `037b979477161881a4877230500e6f3de8faea5c`; all other master changes are preserved. No later master head was chased. The current CI, review, and fresh staging qualification all passed for this integration head. ## Risks - Git bundles and workspace bytes are private evidence. Public reports retain closed checks; the fixture never pushes to the public source repository. - The second committed output must retain the first commit as its parent, the same workspace binding, and original saved file bytes. Final hidden `.git`/HEAD after the second save is not independently exposed by the supported remote workspace API; this test does not claim third-turn Git metadata durability. - Only JSON metadata observations retry transient transport failures. Binary attachment and browser downloads remain single attempts. Transport failures and digest mismatches remain visible failures. - The existing artifact reader is shared. Its runtime code is unchanged and its regression coverage remains included. - Daily source pins and plan selection stay with the coordinator. This PR does not register or bypass the unpassed completion case. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment ID and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1103.0-canary.7 nightly/v2026.1103.0-nightly.0	2026-11-03 00:46:13 -06:00