PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-31 22:24:16 +01:00

Author	SHA1	Message	Date
DottaandPaperclip	dbff3747b8	feat(server): persist run logs and provider traces in S3 (#18327 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use run logs and restricted provider traces to inspect agent work. > - Local log and trace files depend on the web server's persistent volume. > - That dependency prevents overlapping volume-backed Railway deployments. > - This PR reuses the instance S3 store for batched log and trace payloads with conditional publication. > - Existing Postgres rows retain their contracts; native export recovery reuses events already committed before Runner acknowledgement. > - S3-configured servers automatically copy and verify retained history before a later deployment cutover. ## Linked Issues or Issue Description Subsystem affected Server run logs, restricted provider traces, storage providers, and migration tooling. Problem or motivation Replacing the web server and its disk must preserve committed log output and retained traces. The current S3 mirror can repeatedly upload a growing file. Moving those payloads into Postgres would add database writes, WAL, replication, and backup work. Proposed solution Reuse the existing instance storage configuration, S3-compatible provider and credential chain. Store new run logs under run-logs/ and opted-in raw traces under provider-traces/, below the existing prefix. Cloud keeps its existing bucket and bucket-scoped credentials per stack. Add no new configuration switches. Store immutable payload segments and bounded binary range indexes directly in S3. Publish each batch with an ETag-conditional head. Keep existing database ownership and lifecycle records. Recover canonical native exports from their existing validated control event source, checkpointing export progress in S3. Add no new payload writes to Postgres. Preserve local defaults and migrate settled history with verified byte counts and digests. Alternatives considered Postgres payload chunks add control-plane load. Repeated whole-file mirrors amplify transfer work. The first index prototype left superseded index nodes; the final binary index keeps all normally published nodes reachable. Roadmap alignment This is a storage prerequisite for replacing the main web/API server. It does not enable deployment overlap, detach a volume, or change native Runner protocol acknowledgements. The maintainer approved this implementation scope. ROADMAP.md has no duplicate implementation item. Related PR: #16770 improves the existing in-flight mirror. This PR preserves that legacy backend and uses the existing instance S3 profile for new journals. ## What Changed - Automatically select S3 segments for new logs and opted-in traces when instance file storage uses S3. Reuse bucket, endpoint, region, prefix, path style and credentials. Remove separate backend switches. Keep local-disk instances local. - Share the existing storage configuration resolver with product files. Preserve config-file settings, environment precedence and the existing root prefix. Keep pre-existing run-log mirror settings for legacy history. - Add bounded buffering, conditional ownership fencing, immutable range reads, stable byte offsets, and completion hashing. - Dispatch all existing log consumers through the recorded backend. Coalesce output progress writes and publish live log messages after storage receipts. - Recover interrupted canonical native exports from their existing durable source. Exclude separate credential, cleanup, and reprojection records. Replay missing callback ranges and deduplicate late delivery. Repair settled archives after controller lease expiry. Bound shared repair work and stop it when all readers cancel. Serve the published prefix when best-effort repair fails. - Add encrypted optional trace channels, replacement reads, durable deletion tombstones, and deadline-driven local/external expiry cleanup independently of heartbeat scheduling. Stream historical source and remote verification. Propagate worker cancellation through trace open, conditional-write probing, upload, restore and reads. Fence deletion even before the first object head exists; abort and drain owned captures. - Retry transient storage errors, distinguish 409 from failed preconditions, and backpressure bounded log admission. Track interrupted trace imports explicitly so ordinary local deletion stays independent of S3. - Automatically migrate retained history through the existing delivery coordinator. Process one log and one trace per batch. Retry unresolved history with backoff and stop scans after completion. Keep the inventory/import command as optional diagnosis and repair. Verify source and remote bytes before switching references. Verify finalized native size and digest before setting its export cursor. Require ordered source coverage when completeness metadata is missing. Preserve original trace expiry. Block active, missing, corrupt, and unproven sources. - Preserve identity and run-secret redaction in live and recovered native logs. Register runtime log redaction secrets once through the existing encrypted registry. Keep agent private keys in their existing owner. - Add storage, migration, retention, UTF-8, and route-privacy coverage, plus operating documentation and qualification tooling. - Add no schema, payload table, per-record SQL, production dependency, or lockfile change. ## Verification - Final fixes at `b3a44c898e` passed all 116 selected startup, retention, attribution and cancellation controls. The disabled-scheduler cleanup and concurrent-repair defects reproduced red before the fixes. A timed-out repair still prevents later storage reads. - The merged source passed 124 storage and real-Postgres Cloud configuration tests. These include automatic history migration, existing environment/file vault keys, source-directory removal, five real-SDK stalled-request cancellation cases, shared capability-probe cancellation and multi-chunk trace ordering. Saved Cloud S3 settings work after the legacy config file is gone. - Workspace typecheck and full build passed at the merged parent `19f702d780`. Server typecheck and build passed again at `b3a44c898e`. - Actual app acceptance: finish a process-agent run through the normal API, stage its retained local-file input, then restart. Automatic migration reports one migrated item and zero blockers. Remove the original web home and restart at `b3a44c898e` with an empty home, the existing S3 profile and the existing vault key file. The normal log API preserves all five producer lines and exactly 788 bytes, with SHA-256 `74d9ae469da28bfe89b9b50a0e132aec24fe9e6d686496ed5e4ffa6167151036`. No log-specific storage settings are present. - Actual expiry acceptance: with heartbeat scheduling disabled, a trace remains unexpired after startup and later reaches its fixture deadline. The worker marks the row deleted and tombstones both S3 channel heads. The retained run log stays byte-identical. The process fixture emits no provider protocol frames, so nonempty encrypted payloads are qualified separately through the real SDK round trips. - Isolation tests use identical run identities in two stack buckets. Cleanup in one existing prefix leaves another prefix, another company and product files intact. Read-only Cloud source inspection confirms a bucket and bucket-scoped credentials per stack. Four existing Cloud storage environment-contract tests passed. No Cloud provisioning changed and no new live vendor permission probe was performed. - Earlier qualification passed 1,014 affected storage/native tests and 27 privacy-route tests with isolated Docker Postgres. An abrupt process test commits 1,000 records, receives receipts, dies by SIGKILL and loses its directory; another process reads every byte and continues. The real Rust Runner experiment acknowledges 32 records while the export cursor is zero, then continues the same Runner PID after controller loss. All 64 records export once with exactly 64 existing control rows. Repeated snapshot/Stop commands completed in 162/81 ms. - Emulator workload: 1,000 burst records used two batch PUTs. Eight steady streams produced 240 records with 104 conditional PUTs and 72 GETs over 6.9 seconds. These are synthetic operation counts, not fleet sizing or provider prices. - Failure evidence is retained. Compact native events over 64 KiB, missing file-key support, disabled-scheduler retention and repair failure handling all reproduced before their fixes. Fixture repairs addressed cold module setup, localhost/IPC permissions, the S3 bucket-name limit and a mode-0700 crash-test directory. Full local `pnpm test:run` was stopped after host PostgreSQL semaphore exhaustion; affected suites used isolated Docker databases instead. Native package library symlinks were restored with the package's standard hydration helper. - The `19f702d780` browser session-recovery case timed out while saving a task title. All eight browser shards passed on `b3a44c898e`, including that case, with no UI behavior or assertion change. Fresh-head CI passed at `b3a44c898e`: 55 successful checks, three skipped checks and one neutral check. The branch is conflict-free. - Map and module-boundary checks passed. Current-head Greptile is 5/5 with zero unresolved review threads. Reproduce the opt-in storage suites with `PAPERCLIP_LOG_TEST_S3_ENDPOINT` and a dedicated `PAPERCLIP_RUN_EVIDENCE_MIGRATION_DATABASE_URL` ending in `_test`. See `doc/run-evidence-storage.md` for operating limits, automatic migration, optional diagnostics and rollout qualification. ## Risks - Keep the existing per-stack bucket, prefix and vault key stable across replacements. Relocating that storage is a separate whole-instance operation. - This release automatically changes new log storage for existing S3-configured instances. Local-disk instances remain local. Canonical native source replay is qualified with a deterministic real Runner; actual remote execution and production-provider qualification remain rollout gates. Generic server-only diagnostics and optional raw trace tails without a surviving source can be incomplete after a crash. This does not establish full zero-downtime deployment. - Before deploying this release, qualify the actual S3 provider. It must enforce conditional writes and pass read, version, deletion, workload, and latency qualification. A capability probe rejects providers which ignore preconditions. Moto is emulator evidence only. - Dirty output is bounded and unconfirmed until S3 publication. Storage failure leaves persistence incomplete while task results and usage can settle. - Trace expiry is not yet represented by Cloud's external wake contract. Outstanding trace deadlines conservatively block sleep. - Automatic history copying requires the original source volume. Keep it until all retained resources pass verified import and clean-filesystem reads. Stop old binaries from creating new local-file runs. Missing or corrupt history remains blocked and visible in server logs. Rollback requires a compatible reader against the same external state. Ordinary object deletion does not erase retained historical object versions or backups. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, tool use, code execution, and browser verification. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 15:17:35 -05:00
DottaandPaperclip	6dcf4160dd	perf(chat): load recent messages and page history on scroll (#18358 ) ## Thinking Path > - Paperclip lets people manage work done by agents. > - A standing agent chat can hold months of conversation. > - The chat route forced full-history mode, bypassing compact task loading. > - A cold visit fetched lifetime run summaries and hydrated historical transcripts. > - This PR starts with 20 recent messages and loads earlier pages on upward scroll. > - It preserves the reader's position, current questions, and draft while history grows. ## Linked Issues or Issue Description What happened? Standing agent chats forced full-history loading. A long conversation fetched full activity and run history before it became readable. Earlier comments also used an explicit load control. Expected behavior Open a small recent conversation quickly. Load older messages when the reader scrolls upward. Show an inline loading state. Keep the visible message and composer mounted through paging, live updates, and Back navigation. Steps to reproduce 1. Open a standing agent chat with hundreds of messages and completed runs. 2. Use a fresh browser query cache. 3. Observe full activity and run reads plus transcript requests for historical runs. 4. Compare the recent window and upward paging after this change. Related work: #18312 and #18323 restored compact task and collection loading. #14848 added reopening of visited agent chats. A GitHub search found no open PR for this chat-history optimization. ## What Changed - Return 20 contiguous recent comments for standing chats, with a stable oldest cursor. Keep the existing compact task exchange contract. Use the existing company/issue/time index for cursor reads. - Keep chats compact even if their task view previously expanded history. Retain observed messages through live catch-up and cache-backed return visits. - Keep fetched history pages, acknowledged sends, and the observed recent cursor separate. Reconcile live replies without losing history after a send or deleted cursor. - Hydrate metadata only for newly revealed message/run IDs; preserve prior batches while paging. Slow-send recovery refreshes the recent window. Receipts failures have an independent retry. - Hydrate run summaries only for loaded messages and the latest turn. Validate batches of at most 100 run IDs and keep the existing issue/company access rules. - Reuse hydrated terminal native streams when only history membership changes. Preserve retry, resume, mode changes, reconnect recovery, and interrupted recovery reads. - Load older messages, decision receipts, and task handoffs through upward scrolling. Support wheel, touch, and keyboard input. Keep pending questions visible. - Reserve a status slot while loading. Preserve a logical message anchor and the mounted composer. Pause failed page reads until an inline Retry. - Add database, route, controller, and scroll regressions, plus an opt-in production-build browser rig for long chats on desktop and mobile. Update design and feature/access documentation. ## Verification - The real PostgreSQL cold-open suite passes all 12 tests, including contiguous pages with equal timestamps, complete history, company denial, and scoped run IDs. - Activity route and OpenAPI suites pass 53 tests. The route run allowed local cold-import overhead; CI retains the repository's normal timeouts. - Three new paging regressions fail against the unchanged source. The candidate's focused paging/window suite passes 26 tests. Broader UI verification is recorded below when complete. - Desktop and mobile browser acceptance covers recent-only startup, per-frame reveal geometry, delayed and failed older pages, stable reading anchors and drafts, Back, older comment links, and one persisted live reply. - Full workspace typecheck and build passed locally. The final UI source also passes its typecheck and production build. - Affected UI suites pass: 212 controller/history/paging tests, 352 thread/scroller tests, 26 native-stream tests, plus metadata batching and live-cache hydration coverage. The issue mutation/access route suite passes 179 tests. - All 22 final desktop/mobile browser journeys pass, including delayed image growth. The isolated second-page request proof also passes six repeated desktop/mobile runs. A separate 6,000-message, 3,000-run fixture passes cold-open and delayed-paging checks on both screen sizes, still opening 20 messages and loading one 50-message page. - Final matched production timings use baseline `aa58fa1073` (`index-BaRiMbxd.js`) and candidate UI source `f9a26b9783` (`index-DPWTbsFF.js`, identical application source at final head `bf71ffd5af`). Chromium 151.0.7922.34, 1440×900, 4× CPU throttle, no added network latency, browser HTTP cache disabled, fresh application cache for each cold visit. Both variants block service workers and use the same warm local API server and immutable 600-message/300-run fixture. Only document HTML is intercepted for both variants; API requests are real. The common API revision is `109981f8c1`; subsequent commits change only UI and tests. \| Usable conversation \| Baseline median (range) \| Candidate median (range) \| \| --- \| --- \| --- \| \| Cold direct URL \| 6.57 s (5.69–7.45) \| 3.07 s (2.50–3.28) \| \| Warm in-app return \| 3.76 s (3.54–4.76) \| 0.67 s (0.58–1.46) \| \| Back return \| 2.53 s (2.12–4.20) \| 0.51 s (0.47–1.18) \| Five pairs alternate order. Every paired difference favors the candidate for all three journeys. Readiness requires a revealed, fully visible latest reply above the composer and an editable composer. Return timing starts after the Tasks route renders. Cold API requests fall from 953–954 to 71; API wire bytes fall from 517–559 KB to about 49 KB. Separate CPU profiles and request waterfalls confirm the removal of lifetime event hydration and are excluded from the five timing pairs. Both variants have the same expected missing optional plan document (404) and no browser exceptions. Control recent-window API reads have medians of 37 ms / 32 ms, with shared-host outliers up to 376 ms. The 6,000-message recent-window SQL uses `issue_comments_company_issue_created_at_idx` (22 index rows read for a 21-row probe). Geometry assertions retain a 2-pixel tolerance for the reading anchor and first-reveal layout. These are local synthetic measurements, not deployed staging latency. Earlier timing probes are excluded because they did not isolate service workers or composer clearance. Final head `bf71ffd5afb5684ba9105a3127f617e7adf5139c` has 55 successful checks, three optional skips and one neutral result, with no failures or pending checks. Greptile is 5/5 with no open review findings. ### Failure record Early local attempts exposed an unconditional scroller wrapper that changed the old task layout, and automatic retries on chat page failures. Both were corrected. The original task scroller structure stays in place. Metadata request assertions wait for initial connection recovery to settle, then verify only the new page’s reads; retry/resume/reconnect revalidation remains intentional. The initial review caught a send-before-scroll cache defect; its controller regression failed before the fix. Later rapid-paging checks caught scroll readiness being gated by metadata after the first reveal; readiness now stays enabled once revealed. Live replies use the separate acknowledged-message cache, then catch up from the server cursor. Visual review exposed a pagination wrapper bypassing existing composer clearance. The body is again a direct scroller child; cold-open browser checks require the full latest reply above the composer. A resize-observer regression failed before the fix and now passes. Controlled delayed inline images prove both pinned following and a stable reading anchor on desktop/mobile. The first performance fixture lacked valid native event envelopes and caused missing-log fallbacks. It is excluded from latency claims. The final fixture uses saved native assistant events and conversation session generations. The live test uses a temporary fixture-agent key, attributes the reply to an existing synthetic run, and revokes the key. It starts no model or agent process. Local database attempts failed during a Docker shutdown. The retained disposable database was restored and its suite passed. Early fixture errors included a missing conversation identity field and session generation. Mobile test setup also selected an offscreen sidebar link and reused queued human input. Optional announcements could intercept input; the fixture now dismisses them before the journeys. The scroll observer now waits for the delivered wheel event and settled animation frames, including when the viewport is already at the top. A concurrently running live test briefly changed another probe’s row count; the final acceptance and timing runs were sequential on separate fixtures. The final rig uses the visible mobile navigation and a saved agent reply. Anchor observation waits for the user's scroll to settle before measuring the effect of live updates. The full local `pnpm test:run` attempt hit repeated embedded PostgreSQL initialization failures and failures in untouched native-session executor suites; it was stopped after the repeated environment failures. The targeted real-PostgreSQL suite passed, and final-head CI is required before handoff. The first CI run also caught an exact mock expectation missing the added company scope; the expectation is updated and its 179-test suite passes. The final-head first CI attempt also stalled an assigned child task in the existing plan-approval scenario, then hit a heartbeat-run row lock timeout during fixture cleanup. The failed shard and aggregate check passed on an unchanged rerun of the same commit. Scenario assertions and timeouts were unchanged. The failure and rerun are retained in [CI run 41116120430](https://github.com/paperclipai/paperclip/actions/runs/41116120430). ## Risks - Historical transcript diagnostics no longer load on a cold chat open. Saved messages and conversation events remain available; diagnostic run views keep their full records. - Cursor paging must not omit comments with equal timestamps. Database tests cover timestamp/UUID ordering and the complete set. - Prepended or late content can move the reader. Browser checks cover retained row identity, viewport offsets, mounted drafts, and Back. Previously loaded pages stay in the query cache for its existing lifetime. - Local synthetic evidence does not establish deployed staging latency. This PR does not merge or deploy the change. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact backend deployment ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected checks; limits are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 15:12:24 -05:00
Dotta	a4308b9791	test(e2e): add browser project and task context journey (#18353 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People group tasks into projects and use those projects to keep work in context. > - Deployment journeys already test task creation and assignment. > - They do not yet prove that a new project and its task keep the same saved relationship after reload. > - This pull request adds an explicit browser journey for that gap. > - Independent API reads and visible task properties must agree on the exact records and selected settings. > - The journey uses no model runs. Each deployment must pass before a daily plan can include it. ## Linked Issues or Issue Description What existing behavior does this improve? The deployment-targeted Product E2E suite can now check project creation and task context through the production dialogs. Current behavior The suite covers standalone task creation and assignment. It does not prove that a newly created project contains the task selected in the composer, or that the relation survives reload. Proposed behavior Select `project-task-context`. Create a unique project and a self-assigned task with that project selected. Reopen the project, open its task, and reload. Require exact saved company, project, title, prompt, user assignment and standard work mode. Require no agent runs before or after reload. Reason and benefit The same journey can qualify different deployments. It catches lost project links and incorrect saved context without a model call or a separate QA-only test implementation. ## What Changed - Add the browser flow and an independent saved-state oracle with positive and negative cases. - Scope project and task navigation to unique links in the main content so sidebar links cannot bypass list coverage. - Bind reopened routes to saved project and task references. Verify the main project heading and visible task content instead of a hidden duplicate sidebar label. Keep browser failures as closed stage codes. - Register the journey, its closed success checks, and its source files in the definition hash. Default model-matrix selections stay unchanged. - Associate qualification-report coverage with project lifecycle, project task context and task creation. Reject retained browser-only passes containing run IDs, provider/model metadata or native/legacy execution. - Add `project` to the existing durable ownership resource contract. A target provider remains responsible for cleanup of its own project fixtures. - Update discovery expectations, the deployment journey guide and the projects feature map. ## Verification - Focused runner tests: 110 passed across the new oracle, discovery, journey contracts and existing report subprocess checks. The first sandboxed attempt blocked the existing report subprocess IPC socket; the same selection passed with local IPC enabled. - Late review regression checks: both report association and unexpected browser-only model work reproduced as failing tests before the fix. The final contract, saved-state oracle and report catalog selection passed all 72 tests. Runner typecheck passed again after these changes. - Browser readback: the old project-name selector failed in Chromium when a hidden sidebar label preceded the production project heading. All six browser regression cases now pass, including missing rows, wrong project/task destinations and a wrong project property. The fixture uses the production heading and task-row overlay conventions. Credential-free Node 24 imports pass. - Current-master integration: 208 focused catalog, project/search, campaign and reporting tests passed. All nine project/search browser regressions passed. Runner typecheck, map checks and Node 24 imports for both journeys passed. The combined catalog, dispatch, source hashes, success checks, browser-only guard and feature associations retain both selections. - The first deployment qualification created both owned records but failed during later readback with a generic browser code. Authentication, setup, collection and cleanup passed. That failed grade is retained. The corrected source subsequently passed a fresh attached QA campaign; the original failure remains preserved. - New oracle alone: 7 tests passed after the final catalog mapping change. - `node scripts/check-maps.mjs` — passed; structural checks only. - Credential-free `--suite product-journeys --list` — passed and lists this browser journey. - `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json` — passed after building required plugin, runner and eval-kernel declarations. The initial fresh-checkout check failed because those declarations were not built. - All current-head CI gates passed, including repository typecheck, tests, build and browser checks. Greptile reviewed the exact head at 5/5 with no unresolved findings. Fresh attached QA passed at `f19557c9eebde14c2def37195d6b154c23eb9690`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41119722363-1/). All five independent checks and authentication, setup, journey, collection and cleanup passed; zero agent runs started. ## Risks - This adds an opt-in test and changes no production behavior or schema. UI control changes can fail the journey. - The target provider must support durable `project` ownership before it can run this case. Project cleanup must settle owned tasks, reject foreign linked tasks and archive the project while preserving history. - Source registration alone does not include the candidate in any saved daily plan. A fresh pass against that deployment is required first. ## Model Used - OpenAI Codex, GPT-6 family, with code inspection and tool execution. The host does not expose the exact underlying model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicate journey found) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-10-30 15:07:21 -05:00
DottaandPaperclip	a0e3ac62a7	test(evals): add reusable task search and filter journey (#18355 ) ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - The task list lets people find saved work and open its context. > - Product journeys can exercise the same task UI on local and Cloud targets. > - The shared suite lacks a search and status-filter journey with control tasks. > - This pull request adds that journey with exact task identity checks. > - The benefit is reusable evidence that search, filters, and task navigation work together. ## Linked Issues or Issue Description What existing behavior does this improve? Product E2E coverage for task search, status filters, and saved task navigation. Current behavior The shared journey suite covers task creation and assignment. It does not prove that a search excludes an unrelated task, that a status filter excludes a matching completed task, or that the selected task retains its saved content. Proposed behavior An explicit `task-search-filter` selection creates three owned tasks. It searches through the production task list, applies Backlog, and opens the exact target. It verifies reload and Back navigation, saved task fields, and zero agent runs for all three tasks. Reason and benefit The same journey can run against a managed local stack or an attached Cloud target. Independent assertions reject missing evidence and plausible wrong results. Deployment qualification and daily-plan admission remain separate gates. Breaking changes None. Existing journey selections keep their behavior. Related work: #16979 fixes status filtering before pagination. #15403 covers saved task views. This journey tests the existing search/filter interaction and does not add a new UI or saved-view feature. ## What Changed - Add a zero-run browser journey with one target and two owned controls. - Compare exact visible task identities after search, filtering, reload, and Back navigation. - Verify saved company, owner, title, description, status, and zero agent runs through the public API. - Register dispatch, source hashing, closed success checks, and browser reporting validation. - Add adversarial oracle tests and a campaign dispatch test. Document the selection and its feature-map scope. - Exercise the real browser flow with a stateful UI/API port across Coordinator, combined Tasks, and legacy shells. Inject missing ownership, wrong destinations, lost filters, changed saved context, and unexpected native work. - Use the combined Tasks shell's named Backlog view for reload persistence, as required by its existing view-preset contract. Preserve the successful polled result snapshot and link the case to feature coverage. ## Verification - First attached Cloud attempt failed during Backlog selection. Its cleanup passed and it started no model work. The original failed result is preserved; the corrected candidate passed a fresh attached campaign below. - Local Chromium reproduces the cause: the legacy StatusIcon and text label produce the accessible checkbox name `BacklogBacklog`, so the prior exact `Backlog` lookup finds no status control. - The selector now stays inside the Status group and accepts the explicit and duplicated names. Three real Chromium regressions pass, including a separate assignee called Backlog. - Latest focused journey checks: 130 passed. Runner typecheck and map checks passed. Unexpected browser/API failures retain only closed source-owned stage codes. Screenshot privacy guards and all exact identity, content, filter, and zero-run checks are unchanged. - Fresh attached Cloud qualification passed at `e3dfc0ae37e8ac74e9a07ca24421ea6c96f9bcfc`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41114233380-1/). All five independent search/filter/reload/reopen/no-agent checks and all five phases, including cleanup, passed. The browser journey started zero agent runs. - Runner typecheck passed: `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Focused oracle, contract, and campaign tests passed: 101 unique tests. The campaign suite was rerun after its new dispatch test: 30 passed. - Direct browser-flow tests passed: 23 tests. These call the real `taskSearchFilter` function through a stateful UI/API port. They do not establish live browser or Cloud qualification. - Latest focused search, oracle, catalog, and report checks passed: 127 tests across five files. - Full runner Vitest completed with 5,204 passes, two skips, and three failures outside this change. Two local embedded PostgreSQL fixtures failed at bootstrap. The Pi controls test expects profile version 35 while the unchanged base declares 37. These failures remain recorded; the PR does not change those fixtures or pins. - The remaining Node support suite passed separately: 137 tests. The standard unit launcher stops before these tests after a Vitest failure. - Credential-free discovery passed: `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/launch.ts --suite product-journeys --list`. - `node scripts/check-maps.mjs` passed. `git diff --check` passed. - All 58 current-head check runs are green, the additional security status passed, Greptile reviewed this exact head at 5/5, and no review threads remain unresolved. Typecheck, test, build, browser CI, and fresh attached Cloud acceptance passed. Daily-plan admission follows merge. ## Risks - Task-list control labels and link structure can change. Such drift fails the journey and needs an update. - The journey creates three board-assigned tasks. It records ownership before creation and records each returned ID immediately. The existing target provider owns cleanup. - Local assertion tests do not prove that a deployed Cloud target passes. The target owner must record a fresh live pass before admission to a daily plan. ## Model Used OpenAI Codex, GPT-6 family. The host does not expose an exact model version or context limit. Used repository search, TypeScript tools, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 14:26:53 -05:00
DottaandPaperclip	8f822bfcf7	fix(cloud): persist configuration and load release-image packages (#18335 ) Land the approved Cloud configuration and release-image package stack. Includes the separately reviewed adapter settings (#18335), external adapters (#18333), bundled plugins (#18336), startup configuration (#18339), and migration/package checks (#18341). Includes the final image-owned loader and linked-dependency inventory corrections. Verified stack head: `462716170d`. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-30 19:21:33 +00:00
DottaandPaperclip	aa58fa1073	perf(ui): restore fast Coordinator task and inbox loading (#18323 ) ## Thinking Path > - Paperclip lets people manage work done by agents. > - Tasks, Inbox, and Needs You must stay usable as work history grows. > - The Coordinator views bypassed some existing collection optimizations. > - Navigation fetched complete decision cards for a count, and Tasks mounted every loaded row. > - This PR uses the canonical attention rules for small count and task-ID responses. It also restores bounded rendering and the optimized Inbox selection. > - The benefit is less data and less initial rendering work without a second definition of pending work. ## Linked Issues or Issue Description What happened? Coordinator navigation fetched the full attention feed on every cold collection visit. Tasks rendered the entire loaded collection. Coordinator could also select the old Inbox loader when Streamlined UI was off. Expected behavior Fetch only the projection needed by each surface. Start with a bounded set of rows. Keep complete task membership, accurate counts, current permissions, and stable loaded rows. Steps to reproduce 1. Enable Coordinator and open Tasks or Inbox with an empty browser query cache. 2. Observe a complete attention-card download for the navigation badge. 3. Open a large task list and observe that every loaded row mounts at once. 4. Disable Streamlined UI while keeping Coordinator enabled, then open Mine and observe three overlapping compact task reads. Related work: #17756, #3744, #17983, and #18312. ## What Changed - Add board-only attention count and complete task-ID projections. Reuse canonical company, privacy, addressee, deduplication, dismissal, snooze, and archive rules. Skip card bodies, plan/image reads, and presentation roster lookup. - Share the small count read across desktop and mobile navigation. Reuse the page-owned response on Needs You and Needs attention. Keep the newest known count through navigation, including zero. - Use the task-ID index for Needs attention and keep task row hydration in batches of at most 100. - Render 100 task rows initially, across the whole expanded tree. Reveal 150 more at a time through scrolling or a button. Keep existing rows through refresh and append. Defer offscreen leaf layout with shared tokens. - Keep Coordinator on the optimized Inbox when Streamlined UI is off. Use index reads for expanded project recents. - Add real-database, component, and production-build browser regression checks. Update the access and feature maps. ## Verification - All required CI checks pass on `1c68ac8957490a6c8585ba0939355f536c8120bf`: typecheck, build, server/workspace tests, serialized database suites, and eight E2E shards. Greptile is 5/5, with its row-retention finding fixed and the review thread resolved. Storybook checks were intentionally skipped; Paperclip Review completed neutral. - Workspace typecheck and production build passed locally. The final rebased source also passed UI typecheck and production build. Token gates and map checks passed. - The attention service suite passed all 34 tests on disposable PostgreSQL. Final targeted checks cover compact projections, complete-set counts, quality approvals, snoozes, private tasks, and board/company access. Database response capture confirms compact projections do not read plans or images. All 31 OpenAPI tests passed after registering both endpoint contracts. - Six production-build browser journeys passed: desktop, mobile, Coordinator Inbox with Streamlined UI off, Needs You, Needs attention, and an older server page followed by task open/Back. They verify the initial 100-row budget, scrolling, retained DOM identity/height, one owner per attention read, complete task membership, 220 retained rows on Back, and less than 2 px of return-position difference. - The row-budget and Inbox regressions fail against original source and pass with this change. The final three affected UI suites passed 40 tests, and the broader Inbox/IssuesList suites passed 101 tests serially. ### Controlled production-build comparison Baseline `b31ec0a2dc079e32272a8f618425fe92b68bbaec` versus PR head `1c68ac8957490a6c8585ba0939355f536c8120bf`. Isolated baseline asset `index-CS9uPxMx.js`; candidate asset `index-BxJeTQ30.js`. Both UI builds used the same warm disposable API server and database, with 220 tasks and 110 pending board approvals. Chromium 151.0.7922.34, 1440 × 900 viewport, 4× CPU throttle, loopback with no added network latency, alternating baseline/candidate order. Cold cases use a new browser context with network cache disabled; warm cases navigate within the same context. Other host workloads are uncontrolled. Readiness requires the destination collection's first visible task row, then two animation frames. Inbox readiness waits for its canonical Mine route so previous-page rows cannot satisfy the measurement. Five paired samples per case; cold Inbox has a second five-pair confirmation wave because the first result was slower. Times below are milliseconds; parentheses show min–max. Paired savings are the median of individual before-minus-after pairs, not the difference between medians. \| Journey \| Pairs \| Baseline median (range) \| Candidate median (range) \| Median paired saving \| Faster pairs \| \| --- \| ---: \| ---: \| ---: \| ---: \| ---: \| \| Cold Tasks \| 5 \| 2752 (2542–3025) \| 2300 (2275–2524) \| 328 \| 5/5 \| \| Cold Inbox \| 10 \| 2536 (2283–2927) \| 2582 (2117–2856) \| 37 \| 5/10 \| \| Warm Tasks navigation \| 5 \| 2054 (1826–2479) \| 1448 (1419–1548) \| 635 \| 5/5 \| \| Warm Inbox navigation \| 5 \| 2020 (1485–2254) \| 1568 (1289–1848) \| 350 \| 5/5 \| \| Task → Back to Tasks \| 5 \| 1747 (1609–1948) \| 1327 (1238–1355) \| 472 \| 5/5 \| Cold Tasks, warm Tasks, warm Inbox, and Back improved in every measured pair. Cold Inbox latency is inconclusive: the two waves pointed in opposite directions, with the candidate faster in 5/10 pairs. This PR claims less badge work there, not a proven cold Inbox latency improvement. The canonical badge response shrank from 200,039 to 359 decoded bytes (encoded HTTP transfer: 8,125 to 598 bytes). The compact response also skips card payloads, plans, images, and presentation-only roster work. These are measured mechanism reductions, separate from end-to-end latency. Needs You still owns its full card feed; Needs attention owns its complete task-ID projection. Reproduction setup and assertions are in `tests/perf/collections/README.md`. <details> <summary>Individual paired readiness measurements (ms)</summary> \| Journey \| Pair \| Baseline \| Candidate \| \| --- \| ---: \| ---: \| ---: \| \| Cold Tasks \| 1 \| 3025 \| 2283 \| \| Cold Tasks \| 2 \| 2542 \| 2275 \| \| Cold Tasks \| 3 \| 2752 \| 2424 \| \| Cold Tasks \| 4 \| 2783 \| 2300 \| \| Cold Tasks \| 5 \| 2641 \| 2524 \| \| Cold Inbox \| 1 \| 2927 \| 2702 \| \| Cold Inbox \| 2 \| 2422 \| 2629 \| \| Cold Inbox \| 3 \| 2409 \| 2606 \| \| Cold Inbox \| 4 \| 2621 \| 2769 \| \| Cold Inbox \| 5 \| 2539 \| 2856 \| \| Cold Inbox \| 6 \| 2283 \| 2117 \| \| Cold Inbox \| 7 \| 2466 \| 2558 \| \| Cold Inbox \| 8 \| 2674 \| 2286 \| \| Cold Inbox \| 9 \| 2696 \| 2389 \| \| Cold Inbox \| 10 \| 2533 \| 2183 \| \| Warm Tasks navigation \| 1 \| 2479 \| 1423 \| \| Warm Tasks navigation \| 2 \| 2228 \| 1548 \| \| Warm Tasks navigation \| 3 \| 2054 \| 1419 \| \| Warm Tasks navigation \| 4 \| 1826 \| 1448 \| \| Warm Tasks navigation \| 5 \| 1954 \| 1518 \| \| Warm Inbox navigation \| 1 \| 2020 \| 1661 \| \| Warm Inbox navigation \| 2 \| 2198 \| 1848 \| \| Warm Inbox navigation \| 3 \| 1733 \| 1497 \| \| Warm Inbox navigation \| 4 \| 2254 \| 1568 \| \| Warm Inbox navigation \| 5 \| 1485 \| 1289 \| \| Task → Back to Tasks \| 1 \| 1721 \| 1330 \| \| Task → Back to Tasks \| 2 \| 1747 \| 1275 \| \| Task → Back to Tasks \| 3 \| 1948 \| 1355 \| \| Task → Back to Tasks \| 4 \| 1609 \| 1327 \| \| Task → Back to Tasks \| 5 \| 1790 \| 1238 \| </details> ### Failed attempts and limits - The local full repository command hit embedded PostgreSQL startup failures and 16 failures in the untouched native session executor suite; it was stopped. The full local suite is not claimed green. CI on the final revision provides the broad test result. - A concurrent local UI run hit timing failures under host contention. Inbox/IssuesList subsequently passed all 101 tests serially. No tests were weakened to address these timeouts. - CI caught two integration gaps introduced during this work: missing OpenAPI entries for the new endpoints, and an attention fixture incompatible with the routine filters added on master. Both were fixed; the affected 31 OpenAPI and 40 UI tests pass. - The final revision's first CI attempt failed two untouched tests: scheduled-retry batch cancellation and the tool-access audience/archive race. Both failed jobs passed when rerun without changing the revision. Their original failed attempts remain visible in [the workflow history](https://github.com/paperclipai/paperclip/actions/runs/41083379875). - Browser evidence uses synthetic disposable data. This PR is not deployed staging or production verification. ## Risks - New attention sources must preserve canonical eligibility. Both projections call the existing collector, so they share its permissions and complete-set behavior. - The count and task-ID caches must not stand in for full decision cards. Each projection has its own key under the existing attention invalidation prefix. - A row budget can omit descendants if applied per parent. The tests cover one global budget across a large expanded tree, refresh, append, reset, and incremental loading. - Local browser evidence uses synthetic data and is not staging deployment proof. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact backend model deployment ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected suites; broad-run limitations are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 11:58:53 -05:00
DottaandPaperclip	75666224d4	fix(native): use the deadline for remote health retries (#18324 ) ## Thinking Path > - Paperclip manages agent work and its execution state. > - Remote Runner processes use health reads to check their saved process identity. > - A failed health request cannot prove that the process stopped. > - Three fast gateway errors can exhaust the old retry limit in half a second. > - The caller already gives these reads a shared 30-second recovery budget. > - This pull request uses that budget with backoff for read-only failures. > - A short gateway outage can then clear without stopping the existing turn. ## Linked Issues or Issue Description What happened? A transient HTTP 502 during sandbox process supervision caused an active native Codex turn to fail. The probe stopped after three attempts, even when most of its 30-second budget remained. The run reported `notification_transport_failed` and required workspace recovery. Expected behavior Retry read-only transport failures within the existing deadline. Keep process identity conflicts authoritative. Do not repeat a signal after a lost acknowledgement. Steps to reproduce 1. Start a read-only process verification with a 30-second deadline. 2. Return HTTP 502 from its first three transport reads, then allow the next read to succeed. 3. The old helper rejects after three reads. The fixed helper returns the successful observation. Related repairs: #16451 introduced bounded remote probes. #16835 and #17799 require proof of process exit before replacement. This change retains those guards. ## What Changed - Use the existing absolute deadline instead of a three-attempt limit for read-only probe failures. - Back off from 250 milliseconds to two seconds. Clip each request to ten seconds or the remaining budget. - Preserve the original error and diagnostic when the deadline is exhausted. Reject non-finite deadlines before an RPC. - Add a red/green regression and a repeated-gateway-failure case around a real listening process. - Update execution semantics and both recovery maps. ## Verification - Before the fix: the new probe suite failed two behavioral assertions. The gateway case returned its HTTP 502 error instead of the later successful observation. - After the fix: 75 focused probe, diagnostic, process-supervision and adoption tests pass. The diagnostic failure tests use a controlled clock for their full 30-second budget. Their error and redaction assertions stay unchanged. - Map alignment and module-boundary checks pass. - Full workspace typecheck and build pass at final commit `ddd42f66378ebc36ec1d7e0ca1f45943473561b3`. - The broader affected suite against disposable PostgreSQL passed 994 of 997 tests; its three startup/database timeouts passed unchanged in isolation. The initial full default local run encountered embedded PostgreSQL startup failures and did not complete. The four diagnostic tests that assumed the old short retry timing now use a controlled clock without weakening their assertions. - Incident evidence came from Linux AMD64, Node 24.22.2, Core commit `0c70cedce9a803bdc201324528abb35e423e7cd0`, native Codex `gpt-6.1-sol`, and a shared ChatGPT subscription. Two separate runs recorded the same supervision HTTP 502 within one second. - A disposable live Daytona sandbox check passed on Linux x86_64 and Node 24.22.1. Three injected gateway failures recovered in 3.31 seconds on production source from `e54a355fa179022caaa6854302ff56bd85de0229` (unchanged by the final test-only commit). The real Node listening process had one launch, retained its exact identity and saved file across a second read, and produced a confirmed Stop receipt. Sandbox deletion succeeded. This is process-supervision integration evidence, not a Codex task or model execution. - Fixed-revision hosted acceptance and recovery of the original conversation require rollout. This PR does not claim either. The duration of the original provider outage remains unknown. - Final commit `ddd42f66378ebc36ec1d7e0ca1f45943473561b3`: required `ci / verify` and `ci / e2e` checks pass; Greptile is 5/5 with no inline findings. The separate Paperclip Review service has not completed an assessment and remains queued. It is not a required branch check. ## Risks - Persistent fast transport failures cause more read-only requests within the same deadline. Capped backoff limits their rate. - Missing or changed identity still returns immediately. Failed health reads still cannot authorize replacement, process signals, or provider replay. - Signal requests still have one attempt. The timeout budget and execution permissions do not change. - No schema or package changes. ## Model Used - OpenAI GPT-6, through Codex, with reasoning, repository inspection, and shell tool use. The exact deployment model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 10:50:49 -05:00
DottaandPaperclip	b31ec0a2dc	feat(ui): link routines to a filtered task list (#18317 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines create tasks that people need to inspect and manage. > - The routines list sends its View all runs link to Audit. > - Tasks already supports routine origin filters in its API, but the page has no control for them. > - This pull request adds a Routine tasks checkbox and links to that selection. > - People can inspect routine work in the normal task list, search, and board views. ## Linked Issues or Issue Description What existing behavior does this improve? The routines list link and the Tasks filters. Current behavior View all runs opens Audit. People cannot select routine-created tasks in the Tasks filter menu. Proposed behavior View all tasks opens `/issues?view=all&originKind=routine_execution`. Routine tasks is selected. People can toggle or clear it from Tasks. Reason and benefit People can use normal task controls to inspect work created by routines. The filter includes manually started routine runs. It excludes manual tasks and other automatic work. Breaking changes The routines list link changes its label and destination. No API, schema, authorization, or routine execution contract changes. ## What Changed - Add Routine tasks under Visibility in the current, streamlined, and legacy Tasks filters. Keep Live runs only in the same group and use Show agent-created tasks with proper capitalization where that control is available. - Keep the selection in the URL through reload, search, task navigation, and checkbox changes. - Apply the existing API origin filter before pagination. Include routine tasks even when other automatic tasks are hidden. - Keep source filtering in the shared task filter module and collection owners. Reuse the existing company-scoped issue-list contract. Hide the live-run control in the design sandbox, which does not load live-run IDs. - Update regression coverage, product docs, feature/access maps, and the toolbar preview. ## Verification - All 58 final-head checks are complete with no failures for `f46c2e409b20fa372d166f4721601a0471894fb0`, including successful `ci / verify` and `ci / e2e` gates. Greptile and Paperclip Review rate this commit 5/5 with no actionable findings or unresolved review threads. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41078093110). - Local checks pass: full repo typecheck/build for the feature, UI typecheck/build after the final refinements, token gates, feature-map checks, and Storybook catalog generation. - Regression coverage exercises both real list implementations, filter grouping and selection, search and all board columns, URL/pagination state, and mixed pending-decision batches. The Visibility update passed all 89 relevant tests across separate runs. The final sandbox correction passes all 10 filter-menu tests, including three cases that reproduced the bug before the fix. The earlier broad UI run passed 10,043 tests across 845 files. - Browser proof used the actual local app with disposable PostgreSQL data: three tasks created through the routine-run API and one manual task. Followed View all tasks; verified the selected Routine tasks checkbox under Visibility, toggling, reload, search, and board view. Checked current, streamlined, and legacy filter presentations. Screenshots show the link, routine-only results, and requested capitalization/grouping. - Failure accounting: the initial broad UI run found two preference expectations that needed the new false default; those were corrected before the full UI pass. Scrolling tests hit five-second timeouts in combined runs and passed in isolated reruns without timeout changes. A duplicate full local UI run was stopped after the earlier final-commit CI UI lanes passed, with no failures reported in that stopped run. - Full local `pnpm test:run` could not complete because native PostgreSQL initialization exhausted the host semaphore pool. A direct initdb probe confirmed `could not create semaphores: No space left on device`. Browser verification used a disposable container; no host settings or other workspace processes were changed. - Review found that the newly exposed live-run control could empty the design sandbox, which has no live-run IDs. The sandbox now disables that unsupported control; its regression test failed before the fix and passes after it. ## Risks - Low risk. The change uses the existing routine origin and read permissions. A routine task remains automatic work, including when a person starts it manually. - The routine checkbox is exposed only on the Tasks collection. Other filter consumers retain their controls and normalize the new field to false. - Routine histories and Audit run links inside routine details keep their current behavior. ## Model Used - OpenAI GPT-6, through Codex, assisted with implementation and verification. The exact runtime model ID and context window are not exposed in this session. Capabilities used: repository editing, command execution, tests, and browser interaction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 10:28:58 -05:00
DottaandPaperclip	5c807d11fa	fix(github): prevent duplicate review score headings (#18321 ) Clarify the shared GitHub review submission contract for new and existing bots. Remove copied score headings while preserving the structured score and saved assessment. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-30 10:19:31 -05:00
DottaandPaperclip	59e191c368	Limit first onboarding to Claude and Codex (#18309 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - First onboarding helps a new user connect the first agent. > - Its provider picker currently includes GitHub Copilot. > - The first setup should offer only Claude and Codex. > - This pull request limits that picker to those two providers. > - Users can still choose other providers when they add more agents. ## Linked Issues or Issue Description What existing behavior does this improve? The model-source picker in first-agent onboarding. Subsystem affected `ui/` — the onboarding wizard and its tests. Current behavior First onboarding offers GitHub Copilot. It can also expose other providers when their display metadata marks them as recommended. Proposed behavior Show only enabled Claude and Codex choices in first onboarding. Keep their automatic native-runner routing on qualified execution environments and the silent legacy fallback on native Windows, with the same provider UI. Keep all supported providers in the later Add Agent flow. Require a visible provider choice when an old draft names a provider that is no longer offered. Reason and benefit Keep the first setup focused while preserving provider choice for an existing company. Breaking changes Old Copilot or other provider drafts require a new choice in first onboarding. Existing agents, provider APIs, and the Add Agent flow keep their current behavior. Additional context Related provider setup work: #15562. This change only narrows the first-onboarding picker. Searches for onboarding/Copilot and first-onboarding PRs found no duplicate change. This is a small improvement to the existing setup flow, consistent with ROADMAP.md. ## What Changed - Limit first-onboarding adapters to Claude and Codex, with existing availability checks. - Remove the Copilot tile, login card, connection state, model discovery, and creation guards from first onboarding. - Retain only migration handling for old Copilot drafts: clear the retired selection/model/flag, restore automatic runner choice, and avoid reusing GitHub bindings when the user selects Claude or Codex. This also works when both onboarding providers are disabled. - Update coverage for provider choices, disabled adapters, and old Copilot and Gemini drafts. - Clear the model when an unavailable saved provider changes to Codex. - Explain how to proceed when both first-onboarding providers are unavailable. - Update the onboarding feature and access maps. - Document the Windows fallback beside automatic runner discovery, with an explicit note to default Windows to native when its runner support is qualified. This inline migration note is an explicit requirement of the change. - Extend both providers' setup coverage for native and legacy availability without forcing a runtime from the browser. ## Verification - 282 tests passed across all onboarding component/route tests and `NewAgentDialog.test.tsx`. This includes the separate adapter-draft suite that CI identified as still expecting Copilot in first onboarding. - The existing server runner-selection suite passed: 231 tests, including native Claude/Codex runtime translation and Windows legacy fallback. This is automated routing coverage; no native Windows end-to-end run is claimed. - UI TypeScript checks and the production UI build passed. - Token gates, feature/access map validation, module-boundary checks, and `git diff --check` passed. - Reviewer command from `ui/`: `pnpm exec vitest run src/components/OnboardingWizard.test.tsx src/components/OnboardingWizard.adapters.test.tsx src/components/OnboardingWizard.step.test.tsx src/components/OnboardingWizardVariant.test.tsx src/components/NewAgentDialog.test.tsx src/lib/onboarding-route.test.ts`. - A separate browser walkthrough on staging completed the Add Agent flow with Codex and confirmed Copilot remained available. That walkthrough used an existing deployment; it does not prove this branch's first-onboarding change. - Ponytail Paperclip Review and Greptile both scored the latest commit `b470f7023ecb6657bd77edefbcaf2ab12fd89414` 5/5. All review threads are resolved. - All latest-head PR checks are complete: 55 passed and 3 neutral/skipped, with no failures or pending checks. Two server shards passed on a single retry after an unchanged timing-sensitive test exceeded its limit and a self-hosted runner lost its connection. No code or test changes were made for the retry. ## Risks - Low scope: the provider filter belongs only to the onboarding wizard. Native Windows runner support is still unmerged (#18250 is only its build/CI baseline), so automatic selection must retain the existing legacy fallback. Platform selection is based on the execution target, not the browser OS. - Users with an old draft for another provider must choose Claude or Codex. Existing agent configuration is not removed. - Instances that disable both providers have no first-onboarding choice. The wizard explains how to enable a provider or use New Agent from the Agents page. ## Model Used OpenAI GPT-6 through Codex. The exact runtime model ID and context-window size are not exposed in this session. Capabilities used: reasoning, repository edits, shell commands, and browser UI verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 10:04:38 -05:00
Nicky Leach	fba0ab8162	fix(runner): preserve GitHub helpers and Codex probe policy (#18212 )	2026-10-30 07:42:10 -07:00
Devin Foley devinfoley Paperclip	e345b9217d	fix(runner): paginate native task searches without hidden truncation (#18282 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents search tasks to find work that already exists. > - The search schema permits 200 results, but the server returns at most 100. > - The response does not tell the agent that more results exist. > - This pull request adds pages and reports the page size and continuation cursor. > - Agents can read the full accessible inventory before they decide that a task does not exist. ## Linked Issues or Issue Description What happened? A native `search_tasks` request with `limit: 200` returns only 100 tasks. Historical scans recovered 167 and 173 tasks with status partitions. This proves missing inventory and extra work. It does not prove duplicate task creation. Expected behavior Honor the declared limit. Report more results and give the caller a continuation cursor. Steps to reproduce Create 205 matching tasks in a test company. Call `search_tasks` with `limit: 200`. The baseline returns 100. The fixed path returns 200, then five when the caller uses `nextCursor`. Paperclip version or commit Baseline: `81ebe479e8db8a3d59c86cdfb894bd917734492f`. Related changes: #16979 changes UI status filters before pagination. #13699 adds public task URLs. Neither repairs this native limit. An exact `search_tasks` plus pagination PR search returned no result. Pattern: [Task searches omit cases beyond the first hundred](https://www.braintrust.dev/app/Paperclip/p/Paperclip%20Staging/patterns/c20462d6-05fd-4862-bfbd-7bec1f2bc159). ### Hypothesis and thesis for a fix The native authority uses `boundedLimit` with its default maximum of 100. The tool schema permits 200. The response contains only `tasks`. The raw request and result counts agree with this cause. Fix the native authority and its canonical tool contract. Keep the existing `tasks` field. Add `limit`, `returnedCount`, `hasMore` and `nextCursor`. Use ascending task IDs so task activity does not move results between pages. Apply current company, privacy and Muse read checks before each page. A cursor grants no access. Callers must keep the same filters and read every page. Concurrent task changes require a fresh scan. ## What Changed - Honor the native search limit of 1 to 200, with a default of 50. - Add ID cursor pagination and page metadata. - Describe complete inventory reads in the tool contract and access map. - Refresh the generated semantic contracts and seeded catalog identity. - Add regression coverage for complete inventories and access-filtered pages. ## Verification - Red: the database integration test requested 200 of 205 persisted tasks. Baseline returned 100. - Green: the same regression returns pages of 200 and five without duplicate IDs. - The native authority and Muse suites cover company exclusion, status filters, empty pages, invalid cursors and readable-only metadata. - Command: `pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/paperclip-runner-tool-authority.test.ts src/services/native-runtime/muse-native-visibility.test.ts`. - The test uses the real run-bound native authority and disposable PostgreSQL. It does not use an external model or change a deployed scanner. - Both native authority and Muse suites pass: 71 tests. Server direct `tsc --noEmit`, Runner `typecheck:typescript`, `node scripts/check-maps.mjs` and generated semantic contract checks pass. - Full typecheck and build cannot finish here because `cargo` is absent. The full local test command was stopped after about five minutes when the complete CI matrix was running. It has no complete verdict. Current-head CI passed, including typecheck, build, all general/serialized/Runner test suites and E2E. Greptile is 5/5 with no actionable findings at `392cb7424a1fa0ec73401d5e8a10a22b504fde32`. Optional Storybook and manual Docker workflows were skipped and are not executed coverage. The success-example correction also passed all 82 contract tests locally. ## Risks Task order changes to ascending ID. Pagination is not a snapshot. New or changed tasks can require a fresh scan. Each page retains the existing full company-list read and Muse authorization cost. This change adds no database query loop or new background work. Existing callers can still read `tasks`, but must follow `nextCursor` for a complete inventory. No scanner implementation was supplied. No migration or deployment is included. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact provider deployment ID and context limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge The task requires the supplied branch to stay unchanged. All applicable CI and review checks pass at the current head. Unmerged and not deployed. --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 09:23:10 -05:00
DottaandPaperclip	5a7a6b32ba	fix(ui): restore compact Coordinator task loading and responder continuity (#18312 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People open tasks from their personal workspace and send replies to agents. > - Cold task opens should load recent conversation before older history. > - Classic settings could expand history before the Coordinator preference arrived. > - Coordinator lists also read full tasks, and response startup used the previous assignee. > - This pull request restores compact reads and keeps the selected responder visible through runtime discovery. > - People can read and reply without waiting for unrelated history or losing response feedback. ## Linked Issues or Issue Description Related: #17593, #17751, #17756, #17885, #17983, and #18092. What happened? A cold task could select Classic full history while its personal workspace preference was pending. The full-history choice stayed cached after Coordinator became active. Coordinator Tasks used full task reads. Needs attention fetched one enriched task per pending task. Sending to a newly selected agent could show no responder, show the previous assignee, or clear feedback when assignment changed. Legacy runtime discovery then replaced the startup identity with a generic activity line. Expected behavior Wait for the workspace preference before selecting Classic history. Show the task instruction and recent conversation with More messages for older content. Read collection indexes and fetch diagnostics for visible rows. Keep the selected responder mounted from Send through saved input and runtime startup. Steps to reproduce 1. Enable Classic task UI and Coordinator. Delay the personal workspace preference on a cold task open. 2. Open Coordinator Tasks or its Needs attention view with many pending tasks. 3. Open a task assigned to a person. Select another agent in the composer and send a message. 4. Let the message save before runtime reads return. Then allow a legacy run to appear. ## What Changed - Wait for task interface settings before persisting Classic history expansion. - Use lightweight indexes for Coordinator Tasks. Resolve pending task IDs in batches of at most 100. Keep search and participant filters on the server. Enrich at most 50 visible rows through the existing hook without replacing list membership or order. - Preserve an error when a pending task cannot load, instead of showing a false all-clear. - Register board cards for row enrichment. Observe newly mounted cards after column expansion or pagination, and track visibility by element so hidden list rows cannot clear visible board cards with the same task ID. - Track the recipient of each submitted message through reassignment. A later draft's recipient does not change the response already starting. - Share a stable response header across startup, native runtime, and legacy runtime rendering. Keep the existing transcript renderers. - Add regression tests, production-browser checks, design guidance, and feature-map notes. ## Verification - Red/green: delayed Coordinator preference started a full comments read before the fix. It now retains compact history and still expands on request. - Red/green: sends from human or other-agent assignment showed no identity or the wrong identity. Both now retain the selected responder through assignment, acknowledgement, later draft selection, and native/legacy startup. - In a 105-task Needs attention fixture, the previous code made 105 enriched detail requests. The fixed code makes two bounded index requests. - All 659 final affected tests pass across task detail, thread, runner turn, history window, Inbox, board, row enrichment, and Coordinator collection suites. - Review follow-up: the production board lost its next-step badge before card registration. The regression now passes, along with column pagination. The focused 82-test enrichment run and 371-test header/thread run also pass. - All 12 compact-startup browser checks pass against the production UI and an isolated PostgreSQL container. They cover delayed preferences, list navigation, saved-card geometry, document/comment anchors, history expansion, draft retention, retry, and actual legacy process startup. The final deterministic process case recorded 118 frames with zero missing or replaced responder identities. It does not call a model provider. - Full workspace build and typecheck pass. Final UI build and typecheck pass after the shared-header change. Token gates, map checks, module boundaries, and diff checks pass. - The full UI rerun before the board review follow-up passed all 10,034 tests across 844 files. The 659 affected tests pass after that follow-up. The first full UI run passed 10,030 tests and timed out in one unchanged IssuesList large-tree test. Its two variants pass in an isolated rerun and in the full rerun. Those later passes establish recovery; the earlier timeout's cause remains unproven. - The full `pnpm test:run` attempt was stopped after repeated embedded PostgreSQL initialization failures in server fixtures. No complete local server-suite pass is claimed. Browser verification recovered with a disposable container database. - A later avatar browser rerun reused the previous same-title task and its cancelled run through the existing 48-hour title deduplication. That fixture never reached fresh runtime startup. The test now uses a unique title and checks its initial assignment/status; a full 12-case rerun and a second avatar run on the same instance pass. The failed trace is retained locally. - Final commit `7a18372da3` has 55 successful checks, two intentionally skipped Storybook checks, and one neutral Paperclip Review. All required verification gates are green. Greptile is 5/5 with no actionable findings or open review threads. The advisory Paperclip review paused without a complete assessment; no passing Paperclip assessment is claimed. - These checks establish bounded reads and stable rendering. No reliable staging latency benchmark or staging deployment is claimed. ## Risks - Collection projections omit task bodies. Detail navigation must retain its normal core/full task reads. Search remains server-side. - Startup feedback must end on rejection, unavailable agents, terminal outcomes, or a visible run. Existing cases cover those transitions. - The shared header changes the legacy live-turn layout to retain identity beside its existing status line. - No migration, mutation permission, provider protocol, or execution authority change. ## Model Used OpenAI Codex, GPT-6 family. The exact serving deployment ID and context-window size are not exposed in this session. The model used reasoning, code editing, shell tools, automated Chromium checks, and manual browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites and browser checks; full-suite limitations are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 09:03:03 -05:00
DottaandPaperclip	cd5f930558	Add shareable image galleries to task documents (#18157 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/247/changes/paperclip/247 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task documents hold plans, reports, and image evidence. > - Task attachments already have an image gallery. > - Inline document images had no gallery navigation or image links. > - This change gives each document a gallery that uses the existing modal. > - People can review the image sequence and share a link to one image. ## Linked Issues or Issue Description What existing behavior does this improve? Image review in task documents, including the side panel and Classic interface. Current behavior Inline document images do not open a gallery. People cannot move through them with arrow keys or share a link to one image. Proposed behavior Select an image to open the gallery. Use arrow keys or Previous and Next. The URL records its one-based position, for example `#document-report&image=2`. Reloading that URL opens the selected image. Revision links keep their exact saved content. Reason and benefit People can inspect a document's image evidence in reading order and share the exact view. No matching issue was found. The latest 300 open PR titles were checked for related document gallery work. ## What Changed - Add a document renderer that collects rendered images in reading order, including reference-style and repeated images. - Reuse the gallery modal with optional index-change and focus-return controls. - Keep the document owner in the saved tab identity when people switch between local and other-task documents. - Match explicit current-task document owners with local document tabs. Keep review and viewer parameters during gallery navigation. - Treat another Classic review receipt as a new request while image navigation retains the selected revision. - Make inline image controls work with a keyboard. Keep focus in the modal and restore focus on close. - Keep document, revision, review, and maximized-view link parameters during image navigation. - Fix a Classic revision-cache race found by the browser test. Recheck a pinned revision when the saved document head changes. - Update the feature and agent access maps. No new server API or permission is needed. ## Verification - Passed 119 focused tests for document galleries, the shared gallery, document rendering, the revision panel, and hash routing. - Passed all 3 Chromium journeys: desktop, mobile, and Classic. These use real uploads and saved document revisions. - The browser journeys also check explicit current-task owners, review parameters after reopening, other-task links after tab switches, and a second Classic review link after Return to latest. - Browser checks cover keyboard and button navigation, wraparound, shared-link reload, exact historical content, invalid positions, focus return and containment, scroll locking, and Escape, Close, and backdrop dismissal. - UI typecheck, token gates, map checks, and diff whitespace checks passed. - The UI production build passed in the browser harness. - Earlier browser runs found and then verified the Classic revision-cache fix. A later build exited 137 while the full test suite ran. The browser run passed after that suite stopped. - Full repository typecheck and build could not complete because Cargo is absent. The full repository test run did not complete within this sandbox. These checks remain subject to CI. ## Risks - Image positions follow document reading order. Editing an unpinned document can change those positions. A revision link retains the saved sequence. - The shared modal now uses Radix Escape handling. The existing gallery tests and nested mobile dialog journey pass. - No schema change or migration. ## Model Used OpenAI GPT-6 through Codex. The runtime did not expose a more specific model ID or context window size. Used reasoning, code editing, shell commands, and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 07:52:15 -05:00
DottaandPaperclip	ed8fae8dbb	fix(workspaces): preserve completed shallow sandbox exports (#18221 ) ## Thinking Path > - Paperclip coordinates agent work and preserves its results. > - Remote execution copies completed workspace changes back to the host. > - An agent can fetch a published branch with shallow Git history. > - A full Git bundle omits the shallow boundary metadata and can fail host connectivity checks. > - This pull request carries the boundaries through the full-bundle fallback and imports through normal Git fetch. > - Recovery must also recognize completed native work on a reusable lease without treating it as ordinary executor reuse. > - Completed work can be saved without another provider turn, while existing host history remains connected. ## Linked Issues or Issue Description What happened? Sandbox finalization can fail with a missing parent commit after an agent checks out a separately fetched shallow branch. Re-exporting a full bundle repeats the failure because Git bundles do not encode shallow boundaries. Expected behavior Import the completed branch and saved files. Preserve existing host ancestry and concurrent host edits. Keep normal Git connectivity and workspace merge checks. Steps to reproduce 1. Prepare a sandbox from one host commit. 2. Fetch a different published branch with depth one and check it out. 3. Save a result file and restore the sandbox workspace. Paperclip version or commit Reproduced against master before this repair. The regression fails with the missing-parent error before the production change. Related public context: #15268 covers rewritten history integration. This repair handles shallow bundle transport before integration. Searches found no equivalent open repair. ## What Changed - Export shallow boundaries beside the full bundle when the delta import lacks history. - Unpack into a temporary shared bare repository. Keep parent connections already available on the host and import with Git's shallow fetch protocol. - Reject malformed boundary values and retain existing connectivity, branch, and concurrent-work merge guards. Keep boundary probes within 15 seconds without consuming the separate 60-second Git import budget. - Let source-bound preservation recover reusable leases only for the finalizer-selected, accepted successful native result and matching completion contract. Retain provider-stop, physical ownership and host-writer checks; ordinary reusable sources remain blocked. - Classify invalid Git metadata separately from omittable unsafe archives. Keep native finalization failed and preserve service-owned file recovery; carry the sender's dedicated rejection across the command boundary without transient retries. - Share the failure allowlist across bounded run/activity projections and cancelled-preservation selection. Migration `0416` rebuilds the matching partial index; task chat retains the non-retryable recovery notice. - Add real Git, filesystem sandbox and database recovery regressions. Update execution semantics and affected verification maps. ## Verification - Red: real filesystem sandbox restore failed with the same missing-parent/revision-walk error before the export repair. - Green: 240 focused Git, workspace restore, sandbox transport and native-finalization tests pass. Coverage includes both transfer paths, malformed/oversized metadata, a genuinely shallow host with its original boundary intact, and concurrent host work with both merge parents verified. - Red/green: a 14-second simulated probe left the final real Git fetch only 45,893 ms before the budget split. It now retains over 55 seconds after the probes; a 16-second probe exhausts its own budget before host import. - Red/green: an oversized read progress report was retried three times before its distinct metadata classification. It now aborts the first read before payload consumption, leaves the host tip unchanged and does not retry. - Red/green: the real metadata error was swallowed by best-effort export and native finalization incorrectly returned success. Both regressions now pass; all 240 focused Git/restore/native-finalization tests pass. A PostgreSQL/filesystem fault-injection test also verifies retained files, failed run, service-owned recovery barrier, no preservation receipt and no new wake. - Red/green: duplicate metadata produced 47 Git process starts before deduplication; the bounded-work regression now passes. Oversized byte and boundary inputs fail before Git probes. - Recovery red: both reusable lease policies exhausted recovery despite a selected accepted successful result. The assembled PostgreSQL/filesystem path now passes for retained and both reusable policies, including idempotent re-admission after exhausted automatic recovery, saved files, no agent wake, and once-only existing-result finalization. - Source verification controls reject ordinary reusable leases, rejected/failed/unselected results, another physical allocation owner, and an active host writer without changing recorded state. - The 78-test PostgreSQL/filesystem recovery/source suite passed before the metadata classification correction. Its new persisted metadata-failure regression also passes. Full workspace typecheck and build pass at final head `69f10daab2`. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Full local `pnpm test:run` attempts did not finish. They are not counted as passes. Early focused runs exceeded 5-second and 30-second fixture deadlines; the final serial suite passed all 176 tests with a 30-second per-test budget in 56.8 seconds. - Red/green: both bounded PostgreSQL projections dropped the new code in oversized results. All 19 source/projection cases now pass, plus an actual index-predicate test that excludes a metadata repair hold and still selects ordinary cancelled work. - Red/green: an oversized sender `.git/shallow` lost its error type before host validation. The real transport test now verifies one typed rejection, no shallow-payload read, and an unchanged host tip. All 465 command/sandbox/task-chat tests pass, and token/migration gates pass. - Full typecheck/build pass after these final contract corrections at `69f10daab2`; all 20 PostgreSQL source/projection/index contracts pass. All current-head CI checks passed; Greptile is 5/5 with no unresolved review threads. Merged #18230 supplies the CI-cache and Linux shell-fixture baseline repairs; their duplicate changes have been removed from this PR. The final Runner lane passed after one justified rerun of an unchanged authentication fixture timeout (all 11 fixture tests also pass locally). Previous CI failures and incomplete local runs remain recorded; live deployment and retained-run recovery remain separate verification. ## Risks - The fallback creates a temporary shared bare clone. Metadata is limited to 64 KiB and 256 unique boundaries, duplicate/parent probes are cached, and Git import has a 60-second command budget plus a separate 15-second boundary-probe budget. It runs only after a missing-history import failure. - Incorrect boundaries could hide real host ancestry. The importer retains available parent connections, and regressions cover that case. - The full-bundle fallback can transfer a bundle twice. This preserves the existing fast delta path; removing that extra transfer is outside this correctness repair. - Preservation eligibility expands only for completed native exports on reusable leases. It creates no provider turn and leaves ordinary reusable leases ineligible. Database/filesystem regressions cover result selection and competing physical/host ownership. - The migration rebuilds one partial index; no tables, columns or user records change. There are no agent execution admission, model, trust, recovery permission, workflow or lockfile changes. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 20:02:53 -05:00
DottaandPaperclip	ebed7f5237	test(evals): protect Antigravity setup and two-turn delivery (#18190 ) ## Thinking Path > - Paperclip manages agents and their work. > - A provider integration must work through ordinary agent setup and task controls. > - The recorded Antigravity walkthrough exposed setup, delivery, and continuation failures. > - A repeatable check must verify real work and retain failed attempts. > - This pull request converts all 47 recorded steps into one fixed Product E2E journey. > - Maintainers can rerun the case and inspect independent delivery, cleanup, and accounting evidence. ## Linked Issues or Issue Description Builds on merged #18187, #18188, and #18189, and consumes the shared repairs from merged #18230 and the equivalent manual-policy declaration from #18222. Refs #18214 for the CI task wrapper. Refs #12503 and #14938 for related legacy adapter work. Agent or provider Google Antigravity native Runner with a saved managed Google subscription and the exact Low model. Why this adapter is useful A user must be able to add an agent, connect its account, and receive real task results across a follow-up. Picker visibility alone is insufficient. How the agent is invoked A fixed Playwright driver uses ordinary product controls. The agent runs through native ACP. Independent checks use public APIs, actual downloads, owned process observations, and native lifetime fences. ## What Changed - Register one explicit-only case: `antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn`. - Map all 47 source steps and declare saved-account prerequisites, finite limits, and same-host macOS ARM64 scope. - Create an agent through the ordinary picker, connect the saved account, select Low, run Test, and wait for readiness. Save an instruction marker, deliver a file, and update it through an ordinary follow-up. - Require each run's visible completed file activity, exact downloaded bytes, committed Done result, same tagged conversation, distinct physical owner birth, released lease, and free native fences. Verify reload persistence. - Stop failed native runs that still own a retry. Replay an existing Stop with its durable request identity and wait for the matching audited acknowledgement. A terminal reservation without a native intent remains unconfirmed. - Keep private evidence in the existing result, report, dashboard, and accounting pipeline. Retain the paused QA agent and financial history while preserving the shared account and budget policies. - Journal only the ordinary wizard's future ID, target, name, intended harness and expected creator before forwarding its POST unchanged. Bind the response and interrupted recovery to exact identity reads; a pending 404 stays unconfirmed. - Serialize private resource snapshots through temporary-file fsync, atomic rename and directory fsync. Abort forwarding on storage failure and preserve legacy receipts. If later journal or diagnostic writes fail, retain the primary error, always close the browser, and attempt the ordinary failed report. - Add production-component browser controls and harden task-menu readiness, alias selection, scroll restoration, failure packaging, and one hermetic auth-test budget. - Wait for the existing GitHub recovery fixture to register its next deadline before asserting the original range; preserve its nested-transaction and zero-request checks. In the decision-expiry fixture, verify the exact saved expiry separately from the earlier global wake of another queue. ## Verification - Fresh registered acceptance passed on eval source `9060f3cd01b7c3445ecdf703d72e9cf30fe42b59` and merged serving product `61bd4ac6919c2309f766a25e62273b074cee3261`. It completed ordinary setup and two turns with native `.5` and `gemini-3.8-flash-low`. All 16 matchers, 14 checkpoints, and cleanup passed in 165,613 ms. Actual downloads were 61 bytes and 119 bytes. Both turns passed their own run-bound browser activity checks and reload persistence. - Independent read-back confirms Done, a paused retained agent, the same connected saved default account, and no active owned run. Both accepted results have the same full tagged conversation and distinct primary process births. All 21 observed processes retired; both leases released and all three native fences were free after each turn. The selected `.5` checkpoints also showed equal agent/backend session IDs; this is an observed case requirement, not a generic ACP invariant. - Receipts report 339,273 input tokens, 7,224 output tokens, and 202,846 cached input tokens. Subscription prices are unavailable. Billing coverage remains incomplete, not free. - The normal report CLI passed. All 310 files from nine original attempts and two separate recoveries retained their hashes. The twelve current screenshots are byte-identical, and 194 local report links are contained and valid. The eight earlier reports were copied without regeneration. - Delivery source `1bf4fb04df04710b82844b3c9355767f46b5f68f` branches from `d3f2a60e795fc60d64c8b19d1eed4d48657d5492` and merges cleanly with current master `1d571de5f9f886d66e520800baec8fc2f67d0948`. All 41 prior `c8b332` authored file blobs remain identical; the 42nd path is the decision-expiry fixture contract correction. Creation ownership, the strict durable writer, and the ordinary successful workflow remain unchanged. Of 43 measured file blobs, 30 remain identical. The Antigravity oracle, artifact, billing and report gates and its suite-specific validation block remain unchanged. The actual retained agent matches its original creator; its original request `setupId` was not captured and is not reconstructed. - Later master changes include model-visible workspace guidance and rejected-terminal-report handling. Their bounded source assessment is separate: the first actual finish failed, then a same-turn replacement was accepted and committed. Later read-only results match both original native-result hashes, and the complete event catalogs have no `session.failed` event. Master #18228 also expands detection of explicit JSON file requests. The measured create/update-and-attach `.txt` task already required publication under both parsers; the publication validation gate itself is unchanged. Master #18240 changes remote workspace cache staging and adds an OpenCode index refresh. The selected local lane returns before those paths. Master #18213 adds optional numeric turn timing and trace boundaries. Its selected Antigravity model, text, IDs, result and retirement semantics remain unchanged in the bounded source review. Local diagnostic rows and awaits can change event ordinals and latency; the oracle checks relative semantic order. Collector coverage and latency improvement remain outside this proof. This does not claim that the whole current runtime is byte-identical to serving `61bd`, or relabel the measured result as a new run. - The adopted upstream cache controls passed 3/3 with pinned real Turbo. Forced work disables local and remote reads and writes; ordinary work reuses local results. The original genuine-pnpm/Turbo controls remain historical to `9060`'s earlier write-only policy. The duplicate cache patch is superseded by #18230. Its qualification-only pnpm addition is removed because the adopted upstream controls do not require it. - The E2E typecheck, map checks and 277-control replay are carried from the unchanged eval source paths. That replay passed, including the 183 Antigravity controls, catalog/connection checks and the separate product-journey validator. The carried creation controls include 91 config/recovery cases. The final launcher suite passed 19/19, including five real campaign finalization fault cases; the report suite passed 7/7. These final checks did not rerun the earlier 277-control group. The canonical production-component browser spec passed in 27.4 s under its unchanged 60 s limit before the replay. Four paused-composer cases and the product base's build/typecheck plus 376 focused regressions were verified earlier. - The shipped wizard and actual API client cover persistence before forwarding, lost response JSON, foreign/malformed requests, wrong response identity/creator and cleanup-error preservation. Real filesystem controls failed 5/5 before the writer repair and passed 5/5 afterward, preserving the old journal on partial-write, temporary-file sync and rename faults. Three persistent-sync campaign controls failed before the finalization repair and passed afterward. All five final campaign fault cases preserve the primary failure, cleanup failure, usage and failed report while attempting browser closure. The strict journal still blocks the next mutation. These are provider-free ownership controls; the Google task proof remains the original `9060` → `61bd` acceptance. - The decision restart check passed in the local 20-case fixture run. A current-source real-PostgreSQL rollback control reproduced the exact 90,000 ms earlier global wake: the old equality rejected it, while the exact durable expiry and new future-wake bound passed. The original 60 s open-state, exact-timestamp, continuation and quiet checks remain. The broader local run was FAILED: 19 passed / 1 failed, with a 15 s timeout and a 30 s cleanup timeout in an unchanged timestamp-pagination case. That case passed in canonical `c8b332` CI; the local timeout cause remains inconclusive. An extra local server typecheck failed against stale cached Runner declarations; it did not check this excluded test file. Both limits remain recorded for fresh canonical CI. - A broader local GitHub recovery group passed 21/22. Its unchanged pagination case observed an unexpected config request; the log does not establish the cause. That result remains retained, and fresh current-head CI is the final broader gate. - Exact delivery-head CI completed with 55 successes, two skips, and one optional neutral review. Required `ci / verify` and `ci / e2e`, build, typecheck, and Runner checks passed. Fresh Greptile is 5/5 with zero unresolved threads. Optional Paperclip is neutral 4/5; its carried recommendations are dispositioned against the public Stop contract and selected live identity evidence. Seven earlier converted attempts retain their failed grades and partial evidence. Historical Paid8 and its later read-only per-run UI observation remain separate from the fresh acceptance above. Unknown prices and the older partial startup accounting remain explicit. Full lineage and raw evidence remain private in the normal report bundle. ## Risks - This case requires an authorized saved personal default Google subscription and finite operator candidate admission. It does not test fresh OAuth or establish global qualification. - The physical observer supports a same-host loopback macOS ARM64 target. Cloud, Linux, and other models remain outside this case. - A known-cost stop cannot cap missing prices or in-flight subscription charges. Existing operator policies remain unchanged. - Raw account/native identities stay private. Missing retirement proof keeps resources retained. The selected cleanup mode does not test agent purge. ## Model Used OpenAI Codex (GPT-6). The host did not expose the exact model revision or context limit. The model used tools for code edits, tests, browser automation, and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 19:51:32 -05:00
DottaandPaperclip	d3f2a60e79	test(e2e): reuse core journeys across deployment targets (#18222 ) ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests verify work through the browser and public APIs. > - The classic matrix already tests several common task stories. > - Its older stories depend on local fixture setup and workspace access. > - Deployment tests need the same story checks with a separate target adapter. > - This PR shares those checks and adds five explicit journey selections. > - A deployment pass still requires evidence from that deployment. ## Linked Issues or Issue Description What existing behavior does this improve? Product E2E journey reuse across managed instances and deployed Cloud instances. Current behavior The deployed-target catalog covers task creation, assignment, Ask, and fresh onboarding. Plan revision and Everyday stories run through the classic fixture driver. Several story steps assume a local workspace. Proposed behavior Add explicit Plan revision, artifact revision, protected-action approval and decline, Skill Studio, and Stop/redirect journeys. Share the main browser sequences and content checks with the classic driver. Let target adapters provide authentication, native agents, a reachable service fixture, and owned cleanup. Reason and benefit A local pass cannot verify the deployed application. Shared story code lets a deployed target exercise the same behavior without copying an entire test or starting a replacement server. Breaking changes No product API or default matrix selection changes. Passed deployment results must satisfy the declared story checks and work-run count. Unsupported target capabilities fail before a story runs. Related public work: #17262 and #18098. Upstream prerequisite: #18230. ## What Changed - Extract shared Plan, artifact download/oracle, protected service, Skill Studio, and Stop/redirect helpers. Keep classic callers on those helpers. - Add five explicit paid journey selections with owned-task observations and native company, agent, model, and environment checks. - Include classic prompts, delivery/decision code, and the decline grader in the source-definition hash. Keep classic transient-read retries and reject deployment binding failures immediately. Match cookie directory boundaries in sensitive fixture requests. - Require complete story checks and expected run receipts before retaining a passing result. Keep failed evidence and private downloads. - Add a secret-safe request method for deployment fixture credentials. Add the missing declaration for the existing manual evidence policy module so the E2E project can typecheck. - Document target responsibilities and promotion after a real deployment pass. This source catalog does not enroll candidates in a daily plan. ## Verification - E2E TypeScript check: passed, including the review follow-up. - Review follow-up: 30 tests passed across four files, covering transient Plan reads, immediate binding rejection, cookie-path scope, agent receipts, and artifact evidence. Stop now waits for a saved, still-active native turn before the browser click and retains its startup events. Counterexamples reject claimed-but-unstarted, already-completed, interrupted, and non-native work. Shared observation and API helpers also enter the definition fingerprint. Cancellation snapshots are captured before lease cleanup. A direct terminal-boundary assertion rejects native work after cancellation even if it is already in the first snapshot; five cleanup-window counterexamples are covered. The Stop-output regression first failed for both native `item.delta` and `item.completed`; the fixed check detects both while allowing accounting-only updates. A report-catalog test first failed because the sandbox denied the tsx child IPC pipe (`listen EPERM`); the unchanged test passed outside the sandbox. - Affected journey, catalog, API, service fixture, and delivery checks: 313 tests passed across 19 files. - Workspace build: passed after rebase. - Workspace typecheck: passed. The default repository run was attempted. Repeated embedded PostgreSQL startup failures prevented local database coverage; it was stopped without claiming a full pass. Linux CI must supply that coverage. - Broad E2E support run: 4,793 passed, 23 failed, 2 skipped. Failures include existing stale image/Pi assertions and process/startup timeouts under host load. The affected tests pass separately. This is not a claim that the broad suite is green. - Upstream CI prerequisites: this branch is rebased on the merged #18230 repair. The earlier conflicting Turbo flags no longer apply. The shell-hook fix is also supplied by that upstream change. Linux shell-hook investigation: 19 focused tests and adapter TypeScript passed. The exact staged-config sequence reproduced CI's 500 in a pinned Node 24 Linux container. Node's default stdin socket at shell level zero selects Bash remote-shell startup and reads `.bashrc` instead of `BASH_ENV`. Ignored stdin and a nested shell each produce 1000. The fixture now ignores stdin; the production hook and 1000 assertion are unchanged. See [Bash startup rules](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files). - GitHub CI on the earlier head passed most suites but failed two browser tests (a fixture cleanup lock timeout and a fixture run staying queued) plus the shell-hook assertion. The lock-timeout case received one retry, which was cancelled by a later source push and does not establish recovery. These failures remain part of the verification record. - Real Cloud qualification: the private target adapter is reviewed and merged. These candidates must pass there before daily-plan promotion. Current source review: Greptile 5/5 and Paperclip Review 5/5 on `a9df1d6e8d880710fce39b87e689e07e1c8e6edc`, with zero unresolved review threads. The full [GitHub CI run](https://github.com/paperclipai/paperclip/actions/runs/40974946081) passed on that exact head, including both required aggregate checks. ## Risks Shared helper changes affect classic callers. The independent ZIP oracle and visible delivery requirements remain in place. Stop requires native acknowledgement, terminal event evidence, resource cleanup, and no later original-turn activity. The managed local target still lacks paid-agent fixtures and reports these journeys as unsupported. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, terminal use, and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 18:58:26 -05:00
DottaandPaperclip	61bd4ac691	feat(agents): set up Antigravity with saved Google subscriptions (#18189 ) ## Thinking Path > - Paperclip manages agents and their work. > - Users choose providers and managed accounts while adding agents. > - Antigravity was missing from the ordinary additional-agent picker and its saved Google path did not complete native readiness. > - A user needs a successful Test, an enabled task action, and a stable account binding. > - This pull request connects the ordinary setup flow to the native candidate. > - It also preserves retired credential refreshes across database failures and server restarts. ## Linked Issues or Issue Description Built on merged #18188 and #18187. Refs: #12503, #14938 for related legacy CLI proposals. Agent or provider Google Antigravity ACP 1.4.3. Why this adapter is useful Users need to set up an Antigravity agent and run real work through their managed Google subscription. How the agent is invoked Native Paperclip Runner uses the pinned Antigravity ACP runtime. Exact operator admission remains required; the local candidate result does not enable general release. ## What Changed - Show Google Antigravity in the ordinary adapter grid and preserve the draft choice when setup resumes. - Reuse the same user’s saved personal default Google subscription and discover the exact low model through the native account. - Run a prompt-free native Test and wait for authoritative readiness before enabling task assignment. - Carry the managed account through native runs and recover retired refreshes through the existing encrypted, fsynced journal with exact generation checks. - Retain the latest rotated credential through verification, storage, and partial home-removal failures. Retry the exact account’s retained cleanup before another provider use. - Reconcile cleanup before taking the credential mutation lock, fence owners arriving during lock acquisition, and support nested lock holders without a join cycle. - Bound sign-in retirement outside database transactions and preserve claims for the existing cleanup owner; one pending sign-in does not block unrelated accounts. - Cover Gemini and Pi API-key recovery, keep API-key bulk manual-model editing usable, and keep adapter discovery available after candidate expiry. - Skip taskless native timer work before run or credential admission. Preserve current master monitor and deadline scheduling. ## Verification - The registered local saved-Google-subscription journey passed all 16 matchers in 196,520 ms using eval source `12aba` against actual serving source `4a37ee7c`. Both turns stayed in one conversation, produced exact downloaded files of 61 and 119 bytes after reload, and used distinct native primary processes. All observed owners retired, leases released, and credential fences settled. Independent readback confirmed Done work, no active owned runs, a connected saved account, and the retained agent paused for inspection. - The measured runtime is production-equivalent to pre-restack `65ef31dc` except its test-only transport fixture correction. Current-master head `ff7c9249` preserves all 99 product paths: 98 exact blobs and only eight inherited plugin-panel lines in `agents.ts`. Its native binary was rebuilt for inherited Codex-only diagnostic changes; the changed Rust behavior is outside Antigravity's ACPX execution path. Shared native/UI changes have focused regression coverage. The recorded live result is not represented as a measured replay of `ff7c9249`. - Full workspace build and typecheck passed on `ff7c9249`. All 376 focused tests in 14 files passed, covering Google credential/retirement controls and the inherited native-tool, redaction, route, and UI recovery changes. Token gates, maps, and diff checks passed. The required-close fixture was separately demonstrated red, then passed all 317 selected provider-wiring cases. - [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/40959387972) is green: 55 successful checks, two skipped workflow checks, and an optional neutral review check. Required `ci / verify` and `ci / e2e` passed. Greptile and Paperclip Review both report 5/5 on exact `ff7c9249`, with zero unresolved review threads. - This qualifies the selected operator-authorized local saved-subscription candidate. The original exploratory journey remains historical evidence; the repeatable fixed-driver eval is in the next PR. ## Risks - Google refresh recovery must preserve the exact grant, account, and credential generation. An uncertain native owner or failed encrypted handoff retains the private home and blocks cleanup. - A successful local saved-subscription candidate does not qualify fresh OAuth, API keys, Cloud, Linux, other models, or general release. - Timer filtering changes only taskless native Runner admissions. Existing task-bound monitors and legacy adapters retain their owners. ## Model Used OpenAI Codex (GPT-6). The host did not expose the exact model revision or context limit. The model used tools for code edits, tests, and review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 16:51:50 -05:00
Nicky LeachandPaperclip	35ab90ad0f	Map runtime activity to authorized task plugin panels (#18131 ) ## Thinking Path > - Paperclip manages agents and tasks. > - Plugins contribute views to the task sidebar. > - A ready view can belong to an earlier task. > - Runner records activity under its authenticated run. > - This PR maps that activity to installed task-panel declarations. > - The sidebar checks readiness after current-run use and restores use after reconnect. ## Linked Issues or Issue Description Builds on merged contract #18129 and Runner publisher #18050. This PR targets master and is ready for review. Earlier combined verification is retained in [the verification archive](https://github.com/paperclipai/paperclip/pull/18050#issuecomment-6489369676). Subsystem affected Run-event reads, plugin task contributions and the task sidebar. Problem or motivation Readiness polling can show a view left ready by another task. Reload and reconnect must recover current-run use without losing concurrent live activity. Proposed solution Map recorded activity to installed task-panel declarations. Merge authorized history and live activity, then perform bounded readiness checks for the running agent. Preserve the existing read-only view. ## What Changed - Add optional `{ namespace, activityKey }` activation mappings to task-panel declarations. Legacy declarations remain valid but contribute no task panel. - Add `view=plugin-panels` to the existing run-events route. Preserve run-read and telemetry-read authorization, secret redaction and private-run denial. Map activity to installed contributions and return at most 128 panel identities. - Scope activation caches to current panel versions and activity declarations. Preserve unchanged panels during unrelated plugin updates and history reads. - Merge history and live hints without losing concurrent activations. Filter by company, task, run and agent. Restore history on mount and reconnect. - Replace periodic availability checks with up to three readiness attempts after activity. Fresh activity restarts settled transient failures. Pending checks, successful readiness and HTTP 4xx failures are left alone. - Reuse live run updates with the panel cache key. Recover only panel history after a pending subscription read. Mark expected not-ready query results so the global error reporter skips them; real failures remain reportable. - Preserve current-run dismissal and the read-only plugin view. Update manifest comments and producer/consumer documentation. ## Verification - Current head: `5e04d757c`, based on master `09c05c550`. Range comparison confirmed all six feature commits were unchanged by the rebase. The final follow-up keys readiness by the plugin installation ID and availability path. - `cd ui && node ../node_modules/vitest/vitest.mjs run src/hooks/useTaskPluginPanels.test.tsx` — all 45 cases passed. `cd ui && node ../node_modules/typescript/bin/tsc -b` passed. - The shared query-client and live-update recovery selection passed 96 cases on `49c544cd2`. Token gates passed before the final hook-only changes. - Earlier focused verification passed 57 manifest cases, 69 database/history cases, event-registry cases and route authorization cases, including private-run denial. - Regression coverage includes subscription/history races, task changes during recovery, unfocused run completion, bounded readiness retries, denied reads, expected not-ready responses, changed activity mappings, unchanged panel preservation, old cache scopes, and same-version route or installation changes. - [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/40942001707) passed: 54 successful checks, two skips, no failures or pending checks. - [Standard and final review](https://github.com/paperclipai/paperclip/pull/18131#issuecomment-6489842168) both scored 5/5 on `5e04d757c`. The final review used the Apex tier, added no comments, and left no unresolved findings. The PR is conflict-free and ready for review. - Local checks stay targeted. GitHub CI provides the broad build, test, typecheck, browser and Runner coverage. No deployment or live-provider acceptance is claimed. <details> <summary>Retained failures and recovery evidence</summary> - `ac9329548` twice failed a browser reload assertion while conversation history was concealed. Master fixed its readiness wait in #17923. The approve journey then passed locally on `fea63a418` with installed Chrome (1 passed, 39 seconds). All eight browser shards passed in later CI. The browser test is outside this diff. - A canary checkout failed with Git HTTP 408; its unchanged-head retry passed. - `de07d09fa` failed the mobile Cases card click after a ten-second stability wait. The same read-only journey passed locally on that exact head (1 passed, 26 seconds). Its unchanged-head CI retry passed 22 cases, with one skip. The original artifact is retained; the cause of that click timeout is not established. - [CI on `49c544cd2`](https://github.com/paperclipai/paperclip/actions/runs/40903679075) passed 54 checks with two skips. CI also passed on `de07d09fa` after the recorded retry. - The `1e59ab441` merge checkout passed all browser and server suites. Runner shard 2 passed 2,252 cases and timed out waiting for a controller-recovery command. The test had landed on master in #18161 after the former branch base. Rebasing aligned the local checkout with that code; Runner shard 2 then passed on `fcf7380d3`. The original timeout cause remains unproven; no unrelated timeout was changed. - [CI on `fcf7380d3`](https://github.com/paperclipai/paperclip/actions/runs/40940235193) recorded runner shutdown signals in typecheck and a workspace shard. A chat recovery test observed a null wake deadline, and one Runner transport fixture failed during close. Those separate assertion failures remain unexplained. All of those jobs passed on the final readiness-key head. Those later passes establish recovery; they do not prove the cause of the earlier assertion failures. - Local browser setup first stopped at pnpm version-download verification, then a sandbox IPC restriction, then a missing bundled browser. Installed pnpm 9.15.4 and Chrome completed the journeys. No dependency files were changed. - Earlier optional-property TypeScript errors were corrected. Embedded PostgreSQL needed its install hook run from its package directory. Older CI also recorded fixture-cleanup lock timeouts, attention-fixture timeouts, and canceled server shards after runner shutdown. The parent fixed its FIFO snapshot-copy failure before merge. The linked verification archive retains earlier combined evidence. </details> ## Risks Activity is a hint and grants no access. Missing plugins, denied reads and unavailable views contribute no panel. Older runs without activity records show no task plugin tab. A used view stays available for the running task; reconnect reconciles readiness. Matching producer and consumer builds are required. No migration, dependency, image publication or deployment is included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact serving snapshot is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 13:56:48 -07:00
DottaandPaperclip	9e09d8f51f	fix(runner): clarify native approval argument recovery (#18205 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - First-task onboarding requires a saved human proposal decision before child work can start. > - Native proposal tools reject interaction metadata placed outside their advertised schema. > - The existing rejection text focuses on question forms and does not explain the misplaced approval metadata. > - This pull request explains the field placement and keeps approval recovery in its original confirmation or checkbox kind. > - It also retains safe schema guidance in the run log so operators can identify the rejected field. ## Linked Issues or Issue Description Refs: #18025 for native first-task proposal recipes and #16889 for safe native schema guidance. Related open PR #18105 changes separate server API-client guidance. What happened? A native first-task confirmation copied `version` and `supersedeOnUserComment` to the tool root. Both belong inside `payload`. The runner correctly rejected the call before execution, but its feedback described question forms. Later calls kept the misplaced field and changed the approval into a question. No proposal card was created. The run log retained only a generic schema error. Expected behavior Keep strict validation. Explain the nested field locations. Keep a confirmation or checkbox approval in that kind while correcting its arguments. Save the normal pending card and wait for an actual human decision. Retain bounded schema guidance without storing rejected input. Steps to reproduce 1. Send an otherwise valid native confirmation with root-level `version: 1` and `supersedeOnUserComment: false`. 2. Observe the pre-dispatch `invalid_tool_call` rejection. 3. Move both fields into `payload` and keep `interactionKind: "confirmation"`. 4. Verify that the corrected call creates one pending confirmation, exact replay creates no duplicate, and child creation still needs saved human acceptance. Paperclip version or commit The observed argument shape and initial regressions came from public source base `c32e1ba6`. The repair is based on maintained master `4a6f6a65`. Public tests use synthetic prompts and identifiers. ## What Changed - Give rejected confirmations and checkboxes approval-specific guidance. It names `payload.version` and `payload.supersedeOnUserComment` without suggesting a question-form recipe. Other human-input errors retain complete question-form guidance and stable question/option ID instructions. - State the allowed native root fields in the first-task skill and put the metadata inside both executable proposal recipes. - Retain the existing safe guidance in `semantic_tool_denied` diagnostics. Keep the original rejection identity and `paperclipExecuted: false`. - Preserve only the exact bounded scalar diagnostic payload after verified event admission. The rest of the envelope and all API request/result bodies keep their existing redaction. - Add a real native Codex transport regression and strengthen advertised-schema, card replay, human-admission, persistence, and privacy controls. Update the execution contract and onboarding maps. ## Verification - Red: the native transport reproduced the malformed confirmation and showed that the old hint omitted the metadata correction. A second red confirmed that approval guidance still included the question-form recipe. The corrected transport case now rejects before execution, retains a safe hint, and dispatches one corrected confirmation that returns a pending card. - Red: both shipped recipe checks lacked explicit nested metadata. All 49 first-task admission tests now pass, including advertised-schema validation, idempotent card replay, actual human acceptance, agent-only rejection, and question-answer controls. - Updated native checks pass on the current repair: 95 native Codex transport tests with 2 existing ignored cases, all 32 provider-bridge tests, and the schema-feedback unit control. - API/case persistence regressions were red before the event-boundary correction. All 71 affected port/redaction tests pass in a clean run at frozen head `3be3a70f`, including the corrected envelope privacy regression. The focused 16-test persistence/privacy group and server typecheck also passed earlier. - Full local `pnpm build` and `pnpm -r typecheck` pass at exact head `3be3a70f` in a clean detached checkout with Node 24.22.0 and pnpm 9.15.4. Exact-head CI passes: [full PR CI](https://github.com/paperclipai/paperclip/actions/runs/40950360142) and [isolated native Runner readiness](https://github.com/paperclipai/paperclip/actions/runs/40950359835). - The full local unit attempt recorded a 15-second timeout in the unrelated Microsoft Teams 100,000-character existing-comment publication case. That failure remains recorded. The exact isolated rerun passed (1 test; 1,814 unselected). The aggregate timeout cause is unproven. That long local run straddled subsequent source edits and also observed the two port regressions during their red phase; it was stopped and is not final-head evidence. The clean affected-suite run above and current-head CI provide the final verification. No timeout or assertion was weakened. - Release-registry CI initially rejected the native test's compile-time schema include outside the release crate's hashed input closure. The guard was reproduced locally. The test now reads the current canonical schema at execution. All 10 crate-cache controls and the real native approval regression pass afterward; the cache guard and workflow keys remain unchanged. - Current-head Greptile review is 5/5 with no unresolved review threads. All required checks pass (56 success, 4 skipped). The advisory Paperclip Review remains neutral/incomplete after one normal re-review request; no completed assessment or score is claimed. - `cargo fmt --check`, map structural checks, and `git diff --check` pass. - Source and scripted-provider checks do not prove that a model will follow the guidance. The original live onboarding attempt remains failed; acceptance at a deployed revision is separate work. ## Risks - This changes repair guidance and shipped instructions. It does not make malformed arguments valid or supply a human decision. - Diagnostic retention adds no rejected arguments, values, or unknown input keys. The API receipt allowlist is unchanged. The validated-event exception restores only the safe scalar payload; optional debug and other envelope metadata still pass through redaction. - The existing 512-byte guidance bound, strict tool schema, company/run binding, idempotency checks, and proposal admission remain effective. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) through Codex, with tool use and code execution. The context-window size is not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 15:48:37 -05:00
Devin FoleyandPaperclip	88a35cf4e7	fix(recovery): retain workspaces when restart recovery blocks (#17839 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native restart recovery decides whether an interrupted provider session can continue. > - An unusable checkpoint must stop provider replay. > - The failed run can still own a remote sandbox and files that need saving. > - The blocked restart path recorded failure but omitted the existing workspace preservation obligation. > - Saved user comments then waited for cleanup that had no file-recovery owner. > - This change records that obligation in the same transaction as the failure. ## Linked Issues or Issue Description What happened? A native run failed after its provider reported a terminal connection error. Restart recovery rejected its failed checkpoint and marked the run failed. Its prepared workspace and active sandbox lease had no preservation record. A later user continuation waited for the old environment to stop and could not start. Expected behavior When restart recovery blocks a run, preserve its exactly bound workspace through the existing stop-and-save service. Keep provider replay blocked. Stop proof and saved files must still precede any successor. Steps to reproduce 1. Prepare a native run with a remote workspace descriptor and its matching lease. 2. Persist a permanently failed provider checkpoint. 3. Run native restart recovery. 4. Observe a failed run with an active lease and no required workspace-recovery record. Refs #17801 and #17439. Those PRs add lease-settlement notifications and remote controller adoption. This change records the missing preservation obligation in the current restart failure producer. ## What Changed - Call the existing native workspace preservation service before the blocked restart transaction commits. - Keep the existing company, coordinator ownership, source lease and descriptor checks. - Add real PostgreSQL cases for exact binding, an unrelated source, a live owner, and rollback when preservation cannot commit. - Verify that preservation creates no replacement run or wake and retains the provider attempt count. - Document the transaction and recovery contract. ## Verification - Updated the existing branch with master at `09c05c55050fede448196637cbaa543a154da76e`. Both sides of the overlapping test additions remain. The production fix is still two added lines. - Current source verification: macOS arm64, Node 25.6.1, and disposable Homebrew PostgreSQL 16.14. No deployment or live task replay was performed for this branch update. - 113 tests pass across `native-failed-workspace-recovery.test.ts` and `native-restart-recovery.test.ts`. Run them with `PAPERCLIP_TEST_DATABASE_URL` set to a disposable database ending in `_test`. Coverage includes exact source binding, live-owner protection, transaction rollback, no replacement run or wake, cancellation preservation, and orphan preservation. - Full local `pnpm -r typecheck` and `pnpm build` pass. Map checks, module-boundary checks, and the PR diff whitespace check pass. - Local verification limits remain explicit. The embedded PostgreSQL probe fails during `initdb`. The full `pnpm test:run` invocation hit that startup failure and was stopped. A broader restart integration invocation against disposable PostgreSQL did not finish while checkpoint-backlog work remained pending and was stopped. Neither invocation is a passing acceptance result. Hosted CI must qualify the updated head. - Prior-head evidence retained: the preservation regression first failed because its required fields were absent. The original verification passed 105 locally available cases and skipped nine real-runner process cases because the local binaries were unavailable. Previous-head hosted CI passed full typecheck/build, test shards, Runner checks, and browser shards. Those results do not qualify the updated head. - Prior live-repair evidence retained: an audited call to the existing preservation service registered the missing obligation for an exactly bound source. Normal workers confirmed termination, released the source lease, and preserved the prepared files before an assigned continuation started. This was an earlier repair, not acceptance of the current branch update. - Updated-head hosted CI passes on `a22feeb54daa2e9325a55d0ccd86515e64cbd47d`, including typecheck/build, all test shards, Runner checks, and browser shards. Apex is 5/5 on the same commit with no actionable findings or unresolved review threads. GitHub reports a clean merge state. The patch is not deployed. The original provider connection failure remains unexplained by this change. ## Risks - Preservation adds database work only when a restart classification becomes terminal. It adds no recurring scan or timer. - A failed preservation write now rolls back failure settlement. The existing owner can retry instead of leaving cleanup without an owner. - A live coordinator or mismatched source remains protected. This change neither destroys the sandbox nor starts provider work in the recovery transaction. - Existing cleanup must still prove termination and preserve files. A terminal run alone does not authorize continuation. ## Model Used OpenAI `gpt-6.1-sol` through Codex in T3 Code, with xhigh reasoning effort, repository inspection, tool use, and code execution. The runtime does not report the context-window size. Earlier implementation used the OpenAI Codex model recorded in the PR history. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 13:06:28 -07:00
Nicky Leach	e2cfdc7e0c	feat(plugins): expose GitHub credential broker to task providers (#18178 )	2026-10-29 10:52:44 -07:00
DottaandPaperclip	f1911a4c53	perf(skills): move repository packages out of Postgres (#18082 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/205/changes/paperclip/205 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive selected company skills before each turn. > - Imported skills can need files from the whole source repository. > - The current path stores those files in Postgres and reads large JSON payloads during preparation. > - This pull request stores verified packages in the configured file or object store and keeps preparation queries small. > - It also converts old rows automatically and restores missing files from the exact saved commit. > - The result reduces database traffic while preserving skill versions and repository dependencies. ## Linked Issues or Issue Description What existing behavior does this improve? Company skill import, version storage, and preparation before agent turns. Current behavior Repository snapshot bodies live in JSONB. Skill preparation can read those bodies again even when the files already exist on disk. Historical packages and native assets also need a bounded cleanup path. Proposed behavior Store package archives and manifests through the existing storage service. Keep version identities, receipts, hashes, and recovery metadata in Postgres. Convert legacy rows after the stored package passes read-back verification. A missing package can use legacy bytes or its exact Git commit under current repository authorization. No separate data-migration command is required. Reason and benefit The warm path sends no repository bodies through SQL and reads no object bytes. Cleanup runs through the existing deadline coordinator. Current versions, active runs, explicit pins, and authored content retain their protection. Breaking changes The normal application schema upgrade is required. Single-version and skill-file responses keep their existing shape. Version lists, current-version summaries, and test-run version summaries now contain file names and kinds without file bodies. Diff and restore fetch selected versions through the existing single-version endpoint. Older server builds must not write after conversion unless they support the new package format. The storage override and rollback limits are documented in `doc/skill-package-storage.md`. Additional context Related work: #15731 preserves repository dependencies; #17923 changes version-cache scheduling; #18080 changes delivery to remote Codex homes. This change owns package persistence, acquisition, and expiry. It does not replace those delivery or scheduler changes. ## What Changed - Add company-scoped archive receipts, exact-commit recovery recipes, migration deadlines, and a durable object-deletion queue. - Add bounded archive upload, read-back verification, extraction, concurrent request coalescing, and verified local caches. - Convert old snapshots and versions automatically. Keep legacy bytes when verification fails. - Recover missing files through current source authorization and the original commit. Validate repository identity, file hashes, modes, and the aggregate hash. - Expire unused recoverable Git packages and native assets after 30 days. Preserve active pins and the durable copy of authored or unproven content. - Keep warm preparation queries metadata-only. Bound file digest caches and avoid repeated body hashing when file identities are unchanged. - Extend the existing storage contract with cancellation. Register package work with the existing coordinator. Optional conversion retries retain the original bytes and may wait for ingress without blocking sleep. - Add regression tests, operating guidance, a deep trace, benchmark receipts, and an optional maintenance command. ## Verification ### Final master integration and review repair — October 29, 2026 Current head: `a6065424249e763dfeac37161f1b52a9f25bd9b7`. Commit `f8bbdd3dbd` resolves the conflicts with master `76a796e964`; the final two commits repair the Apex deletion findings below. - Keep both package-maintenance and version-retention notifications in their write transactions. - Keep the shared run-pin predicate and master's controller-lease deadline metadata. Package expiry reads the updated protected-version result. - Preserve every master migration entry. Generate migration 0415 (`0415_zippy_zemo.sql`) after master's 0414. Its skill SQL is byte-for-byte unchanged from the previously tested idempotent migration. Retain only the five newest schema snapshots. Migration numbering and safety checks pass. - Repair failed file/folder deletion: preserve a complete temporary subtree until transaction commit, restore on upload or SQL failure, and restore callback failures before releasing the mutation lock. Both PostgreSQL regressions reproduced missing files before the fix and now pass. They cover binary data, executable permissions, empty folders, unlisted files, unchanged database/version rows, successful retry after upload-claim expiry, and historical reads. - Reconcile lost commit replies under the same advisory and row locks. Check the saved version and source identity before restoring. Keep committed deletions intact and preserve backups when the database outcome is unknown or a newer edit exists. Four real-PostgreSQL outcome tests pass; the prior code failed three of them. - Full workspace typecheck and production build pass on the final commit. All 204 focused PostgreSQL package/cache/retention/authoring and filesystem-boundary tests pass, with none skipped. - Apex review 18 passes 5/5, with zero new comments on the exact final head. All 34 review conversations are resolved. - [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/40914116508) completes successfully on attempt 2. The head has 55 successful checks, two neutral reports, two skipped checks, and a successful Snyk status. No checks are pending or failing. All eight browser shards pass. - Runner Vitest 2/2 timed out once in the unchanged ACPX controller-replacement fixture. The same job passed on the preceding commit. One unchanged CI retry passes all five ACPX recovery cases, 2,253 Runner package cases and 126 native server integration cases; their standard one and five skips remain. The initial timeout's root cause is not established. Local diagnosis also hit the fixture's eight-second session.open timeout with an authenticated Runner; no local Runner pass is claimed. - Both full local `pnpm test:run` attempts were stopped when Apex identified deletion defects and source repair became necessary; neither is claimed as a passing full local run. The final 204 focused tests, full workspace typecheck, and production build pass. Full-suite acceptance uses the final-head CI jobs. The superseded merge-head CI included runners terminated by shutdown signals; each source repair started a new full run. - Auto-merge is enabled for squash merge. The final head is conflict-free and all checks pass. GitHub's active master ruleset still requires a code-owner approval for the dependency files. The existing review requests are with `devinfoley`, `nickyleach`, and `forgottendev`; no policy bypass was used. - The earlier hosted stress and full native receipts below remain tied to `a2e597d800`; they are not relabeled as a deployment of this final integration. The earlier local PostgreSQL startup failure remains recorded below; PostgreSQL ran successfully for this integration verification. ### Full native skill execution — October 29, 2026 The full native-agent acceptance gap is closed on candidate `a2e597d800eb989d565780e7aa947d5368676444`. Two qualifying native turns used OpenAI API / `gpt-5.4-mini` in real Daytona sandboxes, on the same disposable hosted TEST tenant as the stress test. A follow-up on the same task consumed a newly saved skill version. The task text asked for the assigned skill workflow. It did not contain the proof nonce or expected receipt. The assigned authored skill contained a Python verifier and a random package-local nonce. The agent read the actual installed skill files and executed that verifier against the imported, automatically converted 1,360-file repository package. It checked the repository file count and reference/binary bytes, and directly executed the repository's executable probe. Independent verification matched the emitted JSON to the nonce and expected byte hashes. The agent persisted the same receipt as its task response; native finalization committed, workspace finalization succeeded, and task state became Done. \| Qualifying turn \| End-to-end run time \| Recorded skills.prepare \| Repository files \| Binary hex \| Shell receipt SHA-256 \| \|---\|---:\|---:\|---:\|---\|---\| \| Initial task \| 47.881 s \| 1822 ms \| 1360 \| `00ff07800a` \| `a1deb30dc095f2306c7aa9cc5ec396da12c68961cf5b809f4d7c20f7fc68493b` \| \| Corrected follow-up, new nonce/version \| 37.974 s \| 377 ms \| 1360 \| `00ff07800a` \| `dbcdc33fa1376b4edc136feae1cdf770560a73292467e93619efdda20e4282a7` \| Both successful shell receipts reported `probe: STRESS18082` and `assetSha256: e8ea7a2f15f7bb3b332a46adbef848465884c8996595aa234748befd544d1f3a`. Their proof nonces, selected version IDs and receipt hashes differed, and both ran on the same task. These are observed timings, not latency SLOs; they include real provider work. The service stress-test timings remain a separate measurement boundary. Earlier failures remain in the evidence: - The original five-minute Daytona allocation timeout never reached model execution. After normal owned-stack restore, its durable cleanup record was `expired` / `success`. The cause of that original allocation timeout remains unknown; a later success establishes recovery only. - Initial continuation setup used unsupported generic agent-update fields and returned HTTP 409 before dispatch. Setup was corrected to use the supported skill-assignment and versioned instruction commands. - The intervening completed native turn read repository bytes and ran the probe, but omitted the verifier JSON under the older test agent's overly-specific instruction file. Its strict receipt validator failed; it is not counted as a qualifying turn. Correcting the test agent's instruction file and sending an explicit task-scoped wake produced the qualifying follow-up above. - The pinned application's immutable rate card omits `gpt-5.4-mini`, so completed usage triggered the normal block-on-unpriced budget policy. Supported append-only accounting corrections estimated the three completed runs using the [official OpenAI model rate card](https://developers.openai.com/api/docs/models/gpt-5.4-mini): $0.75 input, $0.075 cached input, $4.50 output per million tokens, assuming standard processing. Estimates totaled 12.482655 cents. Original usage and missing-price records are preserved; the $1 hard stop and unpriced-usage guard stayed active. - Model shell attempts included a missing `xxd` command and incorrect guessed file paths. Those failed calls and subsequent successful actual verifier calls remain visible in complete event streams. No zero-tool-error claim is made. No product source changed during this continuation. The previously measured contention, cold-start and memory limits remain. The later merge resolution is covered by the separate integration checks above; these hosted receipts qualify their recorded candidate. ### Hosted stress test — October 29, 2026 Hosted acceptance: full native-agent execution and a changed-skill follow-up pass on the recorded hosted candidate. The final integration section above records the subsequent conflict resolution and its separate checks. Tested baseline `929e840595c8933ca485ac7a2e99413e1e11ca64` and this exact candidate sequentially on one disposable TEST tenant, with external Neon PostgreSQL and actual S3. The fixture contained 1,360 files / 22.4 MB / ten skills sharing one repository. Measurements cover deployed skill-preparation and native-context services in an adjacent diagnostic process, not a complete agent turn. The container had 24 vCPU / 24 GB. \| Warm workload \| Median before → after \| p95 before → after \| DB returned value bytes/preparation before → after \| \|---\|---:\|---:\|---:\| \| One skill, serial \| 1,060 → 125 ms \| 1,303 → 155 ms \| 22,610,389 → 1,771 \| \| One skill, four concurrent \| 4,415 → 297 ms \| 5,143 → 328 ms \| 22,610,389 → 1,771 \| \| Ten skills, serial \| 5,723 → 535 ms \| 6,057 → 600 ms \| 224,739,030 → 10,671 \| \| Ten skills, four concurrent \| 24,410 → 1,986 ms \| 24,833 → 2,221 ms \| 224,739,030 → 10,671 \| - Candidate warm preparation performed zero S3 GETs. Ordinary-pool queries fell from 16 to 9 for one skill and 61 to 9 for ten, excluding dedicated lock connections. DB numbers measure returned values before Drizzle decoding, not wire bytes. Baseline/candidate sample counts were 30/100, 32/100, 5/50 and 8/48 respectively; the small baseline ten-skill samples are comparative evidence, not SLO estimates. - Application-UID soak: 800 ten-skill preparations over 10.3 minutes, four workers paced by 2.5 seconds; zero failures, median 559 ms, p95 775 ms, p99 910 ms, zero warm S3 reads. All 726 accompanying health/company-list requests returned HTTP 200. Their end-to-end p95 was 569/502 ms, including network latency. The probe overlaps most of the soak. Soak DB-response capture was limited to its first 60 seconds. - Important remaining costs: an unpaced 16-way ten-skill burst had p95 8.32 seconds because company-level preparation serializes. Cold preparation took approximately 1.1–2.5 seconds for one skill and 7.4–10.0 seconds for ten. Ten-skill diagnostic RSS remained about 1.17 GiB (baseline 1.18 GiB), and the soak peaked at 1.35 GiB. A concurrent whole-container sample was 3.45 GB. This does not establish suitability for a small-memory host or eliminate per-entrypoint native asset duplication. - Upgrade without a manual data migration passed: legacy snapshot JSON shrank from 22,465,691 bytes to `[]`, with a stored package and zero pending conversions. - Recovery passed: sixteen concurrent cold requests coalesced into two S3 GETs; full bytes/binary/executable modes matched; same-size corruption repaired; an injected storage failure preserved warm access and failed cold access safely. A disabled public-Git source denied recovery, and after the actual five-minute backoff its exact SHA reconstructed all 229 files. Cross-company descriptor access was denied. Private-Git credential revocation was not tested. - Retention passed: reclaimed 27,754,496 allocated cache bytes, deleted an unreferenced package row and both objects, preserved aged authored history, and expired/reacquired reconstructible Git content. A version pinned by the failed native run was correctly retained. Only owned fixture timestamps were aged. - Browser UI passed: authored skill creation, saved edit surviving reload, version history, and restoration of earlier content. - Initial native attempt (superseded by the qualifying turns above): one real run failed at Daytona allocation after five minutes (`setup_failed`), before any model execution. Provider cleanup also reported failure. That attempt does not count as a passing native turn. Its receipt and all evidence remain preserved; its cleanup later recorded success after normal restoration. - Initial A/B commands ran as SSH root. Diagnostic-owned lock/cache permissions caused one setup HTTP 500 and the first UID-1000 soak to fail; exact fixture paths were repaired. The final soak, retention and recovery ran as application UID 1000. A fresh UID-1000 one-skill batch also passed (30 samples, median 116 ms, p95 321 ms, 1,771 DB value bytes/preparation). Earlier harness-export, SSH, npm provenance, backoff and UI-wait failures remain recorded alongside their successful retries. - Private evidence includes raw timings, assertions, HTTP probes, screenshots and failure/cleanup receipts. Initial failed-campaign cleanup suspended and retained the owned TEST tenant and released the shared QA account lease at 15:22:41 UTC. The tenant was subsequently resumed under its existing ownership for the native execution qualification above. Final cleanup completed at 15:53:09 UTC: the account lease is released and the tenant is suspended/retained under the original October 30, 12:24:52 UTC retention deadline. Its separate stack lease remains held; no active or pending-cleanup environment lease remained. The original retention window is deliberately preserved even after successful continuation. - Earlier hosted-tested head: `a2e597d800eb989d565780e7aa947d5368676444`, integrated with master `929e840595`. The earlier merge conflicts were resolved at this head. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/40884087629) passed; all checks are complete with 55 successes, two neutral reports, two skips, and no failures. [Apex review 15](https://github.com/paperclipai/paperclip/pull/18082#issuecomment-6480257258) passed 5/5 with zero findings on this exact head. All review conversations are resolved. - Ten real-filesystem boundary cases and server typecheck pass locally for the security fix. Coverage includes cache roots, descendants, individual files, symlink aliases, recursive ancestors, a future concurrent cache fill, and allowed sibling directories. [Current-head private-project integration tests](https://github.com/paperclipai/paperclip/actions/runs/40884087629/job/123483306405) passed all 29 cases, including authorized and unauthorized imports of a cached private package. Full typecheck, build, server suites, all eight browser shards, and Runner checks passed in current-head CI. Local database tests could not run because PostgreSQL could not initialize; this limitation is recorded below. - The earlier generated migration preserved the newest five snapshots. The final integration renumbers its unchanged SQL to 0415. Migration safety/numbering checks pass. Its SQL is byte-for-byte identical to the previously verified idempotent skill migration; repeated application on this latest schema was blocked by the same local database startup failure. - The broader verification before the final migration-only rebase passed 1,025 package, lifecycle, remote-launch, native-session, and asset-retention tests, 52 focused UI tests, full workspace typecheck/build, token gates, and map checks. The linked complete CI run passed before the latest focused fixes. - A disposable MinIO instance passed multipart upload, verified read-back, and cold restoration of an 18 MB archive. - Twenty warm samples with a synthetic 1,349-file, 22.8 MB repository retained five metadata queries and zero object reads. Median preparation was 256 ms for one selected skill and 691 ms for ten, with 5 ms emulated network delay each way. These are local measurements, not production latency guarantees. - No production instance was changed. Earlier verification failures and their disposition are retained below. <details> <summary>Earlier failures, review findings, fixes, and rerun evidence</summary> - The first CI pass found a stale inventory-span expectation and a missing repository-read span. Both are fixed; all three observability tests pass. One browser test copied page text into its YAML parser after reload; its first unchanged-head retry passed, but the race recurred after rebasing. The test now waits for the renamed URL and completed reload before copying the editor, retaining its original assertions. The earlier full local suite did not complete cleanly: it reported setup timeouts and assertions, while isolated relevant reruns passed. The complete current-head CI run closes that broad verification gap. No production instance was changed. - The first Apex review scored 2/5 and identified six issues. The fixes protect authored history even when it shares an in-flight Git upload, recheck upload expiry during orphan deletion, bound failed-upload cleanup, retain per-version kind/encoding metadata, add sanitized failure diagnostics, and correct the migration guide. The final focused run passes all 140 tests; the preceding eight-file run passed all 149 tests, including local storage and S3 diagnostics. Server typecheck and migration safety/numbering checks pass after the fixes. The second Apex review scored 1/5 and found unbounded maintenance storage calls and mismatched package/Git batch limits. Those are fixed with a 30-second cancellable cleanup deadline, shared 100 MiB/10,000-file scanner and storage limits, and bounded Git read batches. The proposed manual lockfile edit was rejected with evidence of the enforced CI-owned lockfile policy. All 19 storage/recovery unit tests and server typechecking pass. The new database-backed cleanup/conversion tests were locally skipped because the host exhausted its PostgreSQL semaphore capacity (confirmed directly by initdb). That gap is closed by current-head CI: [package-store tests](https://github.com/paperclipai/paperclip/actions/runs/40771320775/job/123109300889) pass all 18 cases and [package-worker tests](https://github.com/paperclipai/paperclip/actions/runs/40771320775/job/123109301157) pass all 7, with no skips in either suite. The corrected YAML browser shard passes. A separate planning-fixture test failed its cleanup DELETE with an activity-log foreign key violation after its behavior assertions passed; its one unchanged-head retry passed. The third Apex review scored 1/5 and identified a one-slot pool deadlock, projected path limits, cleanup holding the preparation lock during archive reads, and a publication/expiry race. These are fixed with transaction-local reads, prefix-aware bounds, isolated verification, and shared package row locks with a publication grace. The separate Paperclip review also found stale historical inspection, eager history downloads, and conversion retries blocking sleep. Those are fixed. Local PostgreSQL capacity recovered: the new database regressions ran, including the one-slot fork, paused storage during a warm turn, and a locked publication during expiry. The expanded run passed 144/145 tests; its remaining assertion still expected file bodies from the version list and was updated to fetch the selected version. The corrected company-skill suite passes all 81 tests, and the final package/projection run passes all 27. All 52 focused UI tests pass after updating their query-provider fixtures; the first UI run exposed a fixture remount, and the follow-up exposed an editing syntax error, both corrected. Full workspace typecheck and build, token gates, and map checks pass. The sandbox initially blocked the build tool’s local IPC socket; the authorized build rerun passed. Master then claimed migration 0401. The branch is rebased on `f8226b0515`, the unchanged SQL is regenerated as 0402, and repeated application passed twice on the combined schema. Post-rebase workspace typecheck passes. All 1,025 package, lifecycle, remote-launch, native-session, and asset-retention tests pass on the combined head. All CI checks passed for head 9d80a3b52d. The fourth Apex review scored 3/5 and found a save/edit lock-order cycle, a retry retaining an uncommitted source ID in its recovery recipe, and a summary read racing conversion. Commit b7d9237c6a fixes all three. The new concurrency and retry tests pass within all 61 package/source tests, and server typecheck passes. Master then claimed migration 0402. The branch is now rebased on `65fa1cd617` and the unchanged SQL is regenerated as 0403. All 68 package, source, and worker regressions, server typecheck, migration safety/numbering, and repeated migration application pass on current head `62553671a0`. Subsequent verification and review outcomes are recorded below. - The fifth Apex review scored 4/5 and found cached version reads unnecessarily depending on object storage and conversion reads missing the coordinator cancellation signal. Commit `1eb90d80be` fixes both: cached reads verify bytes without the preparation lock; manifest and archive reads carry cancellation through conversion. The first focused run passed 70/71 tests, including all three new cases, but the existing diagnostic test observed no warning. Its fixture now sets an explicitly expired migration deadline; the final worker suite passes all nine cases. Server typecheck passes. - The sixth Apex review scored 4/5 with one finding: concurrent cached inventories were unbounded. Commit `4b7589e14c` gives cached inventories and previews a two-reader/32-waiter limit independent of cold storage. The first expanded pressure test observed a third reader before the preview path had the same bound; both paths now share the local limit. All 73 focused cases pass, followed by all 29 package cases after the final overload adjustment; server typecheck passes. The separate Paperclip review found no must-fix or should-fix issues. Its optional suggestion to persist durability verification was considered: cleanup intentionally verifies current external bytes before discarding the last local authored copy, and the missing-object regression requires that check. Shared-package eviction happens on cold fills with at most two attempts and a ten-second budget. - The seventh Apex review reached 5/5 with one remaining P2: timeout-bearing manifest reads bypassed request coalescing. The security review also found repeated archive acquisition after a cold administrative read. Commit `79a80a39d2` shares manifest loads with per-caller cancellation and cancels the underlying work when the last caller leaves; coalesces identical cold version/file requests; and publishes verified restored packages into the shared cache. Publication uses a short nonblocking lock, while eviction verification runs outside it. The first expanded run passed 73/74 tests and exposed a missing originating DB for cache publication inside read-only inspection; transaction readers now pass that owner explicitly, keeping metadata on the transaction. The corrected run passed 75/75. A final eviction test initially expected the shared package to remain after only its separate runtime projection was reused; it now explicitly accesses the shared package during verification. All 32 final package tests pass, including independent caller cancellation, one acquisition across concurrent/repeated reads, and warm preparation during stalled eviction verification. Server typecheck passes. - The eighth Apex review scored 4/5 with one finding: decoded cold inventories outlived the storage semaphore while waiting for cleanup. Commit `9d7921d1ec` adds a separate two-response/32-waiter lifetime bound held through cleanup, preserving the independent warm-read queue and forwarding cancellation through eviction verification. The new regression pauses cleanup after two responses decode, verifies a third archive stays queued, and completes warm preparation before releasing cleanup. All 77 package/source/worker cases and server typecheck pass. On the preceding head, serialized server shard 6/9 was interrupted with “The operation was canceled” after 57 seconds of tests; the log contains no failing assertion. Fresh latest-head CI reruns that shard along with the other gates. - The ninth Apex review scored 4/5 with one P1: cold responses could own both permits while waiting on a one-slot ordinary database pool held by a queued transaction reader. Commit `a94f0e409f` reads caller-scoped metadata before admission and isolates all acquisition/cleanup SQL on a private connection. The new real-PostgreSQL regression reproduces the deadlock against the previous head and passes with the fix; all 78 focused cases and server typecheck pass. The previous CI run completed with 54 successful checks, including both formerly interrupted/stuck serialized shards. - The tenth Apex review scored 4/5 with one P1: preloading legacy bodies before admission could retain many full repositories in queued requests. Commit `276e353a37` removes that preload and also bounds legacy manifest derivation/fallback, using private connections inside the I/O slot. All 80 focused tests pass; the final synchronized package suite passes all 36 tests and server typecheck passes. - Apex review 11 returned 5/5 with zero comments on `276e353a37`. While it ran, master `8bc051245f` added migration 0403, creating a metadata conflict. Merge commit `e1783764d9` integrates master and generates 0404_clumsy_leo.sql; its skill SQL is byte-for-byte identical to the previously verified migration. The combined schema passes migration safety/numbering, repeated SQL application twice, all 80 focused PostgreSQL tests, full workspace typecheck, and full build. The preceding `a94f0e409f` CI run was superseded and cancelled while its final runner job was active; no old-head success is claimed for that run. - Final acceptance: Apex review 12 returned 5/5 with zero comments on `e1783764d9`. All 28 review threads are resolved. The final head has 55 successful checks, two neutral reports, two skipped checks, and no pending or failing checks; the PR workflow completed successfully. Superseded run 40789121902 was cancelled to free its workflow slot for the final head; normal cancellation did not finish promptly, so force cancellation completed it. - October 29 conflict resolution (`867c00673b`): master `929e840595` added installed-selection reuse and bundled relocation, routine cleanup notifications, a hardened YAML copy helper, and migrations through 0413. The merge retains those behaviors together with package read scoping and the public-selection fast path; restricted selections still reauthorize and the full row read rechecks scope. Generated migration 0414 retains the existing SQL unchanged. Workspace typecheck/build and migration checks pass. Local database verification is unavailable: initdb reports semget / “could not create semaphores: No space left on device”; all 168 cases skipped, and repeated migration application failed at startup. Docker returned HTTP 500 during container creation, so no Docker test pass is claimed. Current-head CI must close this gap. - Apex review 14 scored 3/5 with one P1: a local import could copy cached private package files into a company-wide skill. Commit `a2e597d800` rejects cache trees and ancestors through a canonical filesystem boundary, including aliases and ancestors checked before a concurrent fill. Ten local filesystem tests and server typecheck pass. The existing private-project integration case now rejects package-cache imports for both authorized and unauthorized users. The preceding merge commit’s complete CI run passed, and logs confirm all 168 locally skipped cases executed successfully: package store 36, sources 35, package worker 9, runtime selection 4, observability 3, authoring 81. - Apex review 15 returned 5/5 with zero new findings on `a2e597d800`. The separate Paperclip review found no must-fix or should-fix issues. Its optional suggestions were considered: unexpected filesystem cleanup failures remain visible, the durable marker cannot replace checking that bytes still exist before deleting a last local authored copy, and administrative history batching remains outside the pre-turn optimization. The cache-path literal centralization suggestion is also optional; the current producer and guard use the same directory, covered by the new integration regression. All review conversations are resolved. - Earlier October 29 CI/Apex acceptance: `a2e597d800` was mergeable at that check, Apex review 15 passed 5/5 with zero findings, and all review conversations were resolved. The hosted verification above records the later acceptance limits and renewed conflicts. Workflow 40884087629 completed successfully, with 55 successful checks, two neutral reports, two skips, and no pending or failed checks. The project-skill-sources job log confirms all 29 cases ran and passed, closing the latest integration gap. The preceding merge workflow also confirmed all 168 focused database cases ran and passed. No local PostgreSQL or Docker pass is claimed. </details> ## Risks - First use after eviction can require storage or Git access. Revoked grants, removed sources, and unreachable commits can prevent reconstruction. Recovery never substitutes a newer commit. - Historical packages without a proven recipe keep durable bytes. Cleanup favors retention when a run pin cannot be interpreted safely. - New uploads and conversion use bounded work and persisted retries. Large initial conversions still add temporary storage and database load. - Optional cleanup can wait until the next server wake. Supported notifications assume one owning server process. - Per-entrypoint native tree duplication and cold preparation cost remain. The final ten-skill cold sample took 47.6 seconds on the shared host. - Generic authored edits can still hold an existing edit transaction while a package uploads. - Explicit file/folder deletion uses temporary disk space for rollback. An uncertain commit with a changed skill or unavailable database retains that backup and logs its location for recovery. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and tests. The exact served model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 12:19:37 -05:00
DottaandPaperclip	6a932bb979	fix(daytona): preserve workspace exports under disk pressure (#18172 ) ## Thinking Path > - Paperclip manages AI agents and preserves their work when execution stops. > - Remote workspace recovery must save the retained files before another executor can start. > - Daytona directory export first writes a complete tar on the source sandbox. > - When that disk is full, each preservation retry fails while writing the same temporary archive. > - This change streams an archive after a storage-capacity failure, without another full copy on the sandbox. > - Existing source ownership, archive validation, and task admission gates remain in force. ## Linked Issues or Issue Description Related: #14402 established safe workspace export and retained-source recovery. This PR handles the remaining temporary-archive storage requirement. What happened? A sandbox ran out of disk space. The provider could no longer save its state, and workspace preservation then failed with tar `No space left on device` or `Wrote only ... bytes`. Repeating the export could not resolve that failure because it required free space on the same disk. Expected behavior Export retained files without needing a second full copy on the source disk. Reject incomplete transfers and unsafe archives. Preserve the source and newer host files if export fails. Steps to reproduce Run the capacity regression with real tar and a file-size limit. The original export fails during archive creation. The fixed export transfers the same bytes through a real PTY. The credential-gated live test uses a real Daytona sandbox and `/dev/full` as the archive sink to reproduce ENOSPC without filling a provider disk. Paperclip version or commit Reproduced on `27be859044`; fix rebased onto `6d59f5ef7e`. Deployment mode Source checkout on macOS arm64; live provider verification on a disposable Ubuntu 24.04 Daytona sandbox with SDK 0.203.0. ## What Changed - Fall back only after archive storage-capacity errors. Keep the ordinary bulk download path. - Transfer 40 KiB archive chunks over a raw PTY. Wait for each host write acknowledgement and a final receipt acknowledgement. Send long scripts only after raw-mode readiness. - Require successful tar exit and complete framing before the existing host archive validator can import files. Keep the bounded safe-link retry and exclude rules. - Bound total archive bytes by the provider-reported disk allocation, preserve a 1 GiB host free-space reserve, and bound terminal cleanup to five seconds while closing the host file independently. - Add real-tar/PTY regressions, stream failure tests, a credential-gated provider test, and recovery documentation. ## Verification - Red before production changes: the capacity regression failed on the real tar write with exit 153. Green afterward: unchanged bytes, file mode, confined links, and source files survive. A tar producer that fails after sending bytes cannot overwrite newer host files. - 43 focused tests pass. Missing receipts, nonzero exits, transport loss, invalid base64, oversized frames, stalled transfers, and late connection cleanup fail closed. - Live Daytona test passes: stop/start, actual tar ENOSPC, matching file hashes and modes, unsafe-link filtering, intact source, then another stop/start and export with a later saved file. The disposable sandbox was deleted. Initial default-sandbox creation hit account quota; the successful test used a 1 GiB image. Large output bursts exposed a real socket closure, corrected by bounded chunks and verified in the successful run. - Repository `pnpm -r typecheck` and `pnpm build` pass. `node scripts/check-maps.mjs` passes. Standalone plugin typecheck passes. - Standalone plugin build and final typecheck pass. The final broad plugin run passed 482 tests, skipped seven credential-gated cases, and hit one existing 30-second archive-listing timeout. Earlier runs hit both large-listing cases. Both timeouts reproduce with unchanged `file-sync.ts` from master `6d59f5ef7e`, with 22 other archive-listing cases passing on both base and fixed source. Parallel, serial, and GNU-tar local runs retain those timeouts. The serial local `pnpm test:run` was stopped after 36 minutes once the complete current-head CI test suite passed; it is not counted as a full local pass. GitHub Actions on production-code head `9ffbd9e56e283a01c22f005c022b8f6838cf0143` passed typecheck, all general and serialized test shards, Runner verification, build, E2E, and canary dry run. - Review fixes have passed the live Daytona test again. Sparse-file budget enforcement, rejection before host writes, low host reserve, and stalled provider cleanup are covered by the 43 focused tests. Greptile is 5/5 and security scanning passes on `9ffbd9e56e283a01c22f005c022b8f6838cf0143`; both review threads are resolved. - Test portability: `d2280108f5` verifies GNU tool versions and Python before running the real-PTY regression. All 43 focused tests pass with prerequisites installed; restricting PATH to the macOS system tools cleanly skips the three integration cases. Standalone plugin typecheck passes. Current-head CI passes typecheck, build, all test shards, Runner verification, E2E, and canary dry run: https://github.com/paperclipai/paperclip/actions/runs/40910736373. Greptile and Paperclip Review both assess the final head 5/5, security checks pass, and no review threads remain unresolved. - Paperclip Review assessed the final head 5/5 with no actionable defects after the test-portability fix. Remaining accepted tradeoffs are recorded under Risks. - This is live provider-boundary acceptance. It does not prove a full task Retry journey, repair lost provider session state, or recover the original deployed task. ## Risks - Streaming requires Bash and GNU tar, dd, and base64. Missing tools or transfer failure reject export and retain the recovery obligation. - Acknowledged terminal transfer is slower than bulk download, especially across high-latency links. Very large exports can still reach the configured deadline. The ordinary bulk path is unchanged. - The fallback checks host free space before every 40 KiB write; this deliberately favors a strict reserve check over reducing local syscalls. It carries archive-download error context but does not yet emit a dedicated transfer span. Provider cleanup is bounded; a kernel-stalled local filesystem operation can still delay completion and file close. - This change does not increase sandbox capacity or reconstruct provider state that failed to persist. The plugin must reach the target installation before recovery can use it. No merge, deployment, or original-task replay is included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, shell execution, and automated testing. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 11:41:01 -05:00
Nicky LeachandPaperclip	247de89eae	fix(runner): preserve warm tasks during recovery (#18169 ) ## Thinking Path > - Paperclip manages agent work and the lifetime of its execution resources. > - A warm native task can keep one Runner alive between successful turns. > - Foreground completion selects whether that resource stays running or stops. > - Background recovery must use the same selected policy. > - Recovery currently reads that policy only from workspace copy-back records. > - A task without such a record can be stopped immediately after foreground retention. > - This pull request saves the resource policy with the native runtime and permits the retained task to refresh its control connection. ## Linked Issues or Issue Description What happened? Background native finalization can stop a successful warm Runner when its run has no Core workspace-sync record. Foreground completion retains that same task. The stopped Runner cannot receive the later drain and suspend commands. Cold resume then correctly rejects its unproven checkpoint. Hosted diagnostics confirmed the recovery reconciler attempted environment release immediately after foreground retention. The provider connection returned HTTP 404 within the same second. A database-backed regression reproduced the stop without a workspace-sync record. A separate authorization gap also prevents the current retained task from refreshing its connection endpoint after the run succeeds. Expected behavior Foreground completion and recovery must use the selected resource policy. A successful warm task stays running until its idle shutdown can suspend it. Failed or cancelled work still stops. The current retained task can refresh its existing control route, but cannot prepare or submit new execution. Steps to reproduce 1. Select a warm native runtime on a reusable remote execution environment. 2. Complete a turn without a Core workspace-sync record. 3. Let native finalization recovery settle the still-active run lease before foreground retention finishes. 4. Before this change, recovery defaults to `stop_and_retain` and invokes the provider stop path. 5. Separately, request the retained task's connection endpoint after successful completion. Before this change, Core rejects that request. Paperclip version/commit The hosted failure used Core `31b927e301`. Regressions also reproduced on master at `6d59f5ef7`. Deployment mode A remote native execution environment with reusable warm tasks. The resource policy is generic and applies across harnesses and providers. Related recovery coverage: Refs #18161. This change preserves the suspended-checkpoint admission guard. Existing tasks without the newly saved policy retain the previous conservative fallback; deployments must drain or recover them explicitly. ## What Changed - Save the selected native resource disposition in the existing runtime-selection transaction. Add no startup database round trip or network request. - Read this small policy in background settlement and recovery when no workspace-sync record exists. Preserve valid workspace-sync policies and export-retry behavior. - Keep failed and cancelled runs out of warm retention. - Permit connection refresh for a successful, dispatched warm task on its current retained reusable lease. Keep retirement, termination, expiry, saved-run identity, and plugin capability checks. - Add database-backed retention and selection regressions, failure/cancellation coverage, and bounded projection checks. Document the lifecycle and access contracts. ## Verification - Before the source edits, the no-workspace-sync regression invoked the driver with `stop_and_retain` instead of retaining the lease. Both runtime-selection cases failed because the resource policy was absent. - After the source edits, those regressions and both run-state/runtime-selection files passed. The three-file run had 77 passes and two new fixture expectation failures. Those fixtures were corrected. Final-head GitHub CI passed, including the server suites, Runner checks, typecheck, build, and E2E gates. The local rerun was deferred at the user's request. - Commands: `node_modules/.bin/vitest run server/src/__tests__/native-environment-settlement-work.test.ts server/src/services/heartbeat/runtime-selection.test.ts server/src/services/heartbeat/run-state.test.ts`. - The connection authorization regression failed before its edit. All 34 tests in `server/src/__tests__/environment-task-runtime.test.ts` then passed. - `node scripts/check-maps.mjs` and `git diff --check` passed. - No comprehensive local suite was run. Further local tests and VM reproductions were stopped at the user's request to conserve battery and mobile data. GitHub CI verifies the final head. - Hosted verification of this patch requires deployment after merge. Passing pre-patch retries did not establish repair of the original failure. - Standard Greptile and the subsequent requested Apex review both returned 5/5 on `360bcd7e789d65950c27484af237faec02553666`, with no inline findings. ## Risks - This adds one server-owned field to the existing run profile JSON; no migration is required. Invalid or absent policy keeps the previous conservative fallback. - Valid workspace-sync disposition remains authoritative. Pending export and failed finalization still use stop-and-retain safety behavior. - Retained connection access requires a successful run, matching saved run identity, dispatched warm task, unexpired reusable lease, and no retirement or termination. Superseded leases and new execution remain denied. - The cold-resume checkpoint proof is unchanged. This patch does not make the already-stopped task's unsuspended checkpoint resumable. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The host does not expose a more specific serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run targeted local checks; final verification passed in GitHub CI (local rerun deferred at the user's request) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 09:19:49 -07:00
DottaandPaperclip	6f278dd6d7	fix(runtime): settle recovery lease and connection lifecycles (#18167 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - A remote runner can survive a controller restart. > - Recovery renews its execution lease during slow preparation. > - The native executor then shares that renewal timer. > - Preparation kept its reference until all finalization finished. A tick after lease release treated completion as ownership loss and detached the retained runner. > - A Linux continuity test also found that buffered wire events could arrive before their connection was registered. Cleanup then kept a completed promise under an undefined key and never finished. > - This pull request transfers renewal ownership after execution acquires its reference. It also registers each connection before wire callbacks can run. > - Recovered turns can complete, retain the warm runner, and settle connection cleanup. ## Linked Issues or Issue Description What happened? A recovered remote run could complete, release its coordinator lease, then detach during instruction collection. The next turn could find the old runner still using the ingress port. Recovery then stopped because it could not prove process ownership. Expected behavior Preparation keeps the lease alive until execution owns renewal. Execution stops renewal before releasing the lease. Actual ownership loss still detaches the runner. Buffered wire callbacks must use the registered connection, and cleanup must finish. Steps to reproduce 1. Recover an in-flight remote runner after a controller restart. 2. Let the recovered turn complete. 3. Hold instruction collection open through the next lease renewal tick. 4. Send a follow-up that needs the retained runner. Related: #17439 introduced remote controller recovery. #18011 preserves checkpoint authority through warm preparation. This change fixes the lifetime of preparation's renewal reference. ## What Changed - Pass an internal lease-acquired callback from recovery through heartbeat dispatch. - Release preparation's renewal reference after the executor registers its own. Keep final cleanup for exits before dispatch. - Add a PostgreSQL regression for completion while finalization is still pending. Cover callback forwarding, acquire-before-release ordering, and failure before dispatch. - Register the authority connection before subscribing to wire events. Handle an immediate close before accepting buffered data. - Add regressions for a message or close delivered during subscription. - Extend Linux continuity coverage with preparation renewal and delayed instruction collection after recovery. - Update the continuity guide and session verification recipes. ## Verification - The new recovery regression failed before the fix because renewal detached the runner after lease release. The 76 recovery, renewal, and dispatch tests then passed. The final ordering and early-exit checks passed again after replacing fixed sleeps with controlled interval ticks. - Both buffered-wire regressions failed before the connection fix. All 170 control-plane tests passed afterward. Existing tests cover stale authority, ownership loss, pending operations, and closed ingress. - Linux ARM64 acceptance passed at `a7daefacceb8bd994d0b79d284cdf54f3e3829d1`, with Node 24.19.0, PostgreSQL, the real Rust runner, TLS ingress, and a deterministic Codex protocol fixture. The test interrupted the first turn, recovered it, held finalization open for a renewal interval, and completed two follow-ups. It retained one provider process, three turns in one history, and the same runner identity. Final suspension and connection cleanup completed. The ordinary warm-reuse case also passed three turns with one provider process. - Earlier Linux attempts completed the turns but hung during cleanup. A debugger found a fulfilled promise keyed by an undefined connection. The regression reproduces that defect. One later setup attempt timed out while staging an unstripped 373 MB debug binary. The passing run used the same binary with debug symbols removed. - Reproduce Linux acceptance with the `same-server` case in `doc/testing-native-remote-continuity.md`. - `pnpm -r typecheck`, `pnpm build`, map validation, and `git diff --check` passed at the final revision. The full local `pnpm test:run` command was attempted, but the serial run had not finished after 13 minutes. It was stopped after the equivalent CI lanes passed. It is not counted as a full local pass. - Greptile and the Paperclip review both scored the final revision 5/5 with no findings. All Runner verification lanes passed. The first CI attempt hit timing failures in voice recovery, agent YAML configuration, and the task-title outage browser test. The full YAML and voice-recovery files passed locally (88 and 19 tests). These paths were unchanged and passed on the prior revision. All three passed on the single retry. Final current-head status: 55 successful checks, one neutral check, and two skipped Storybook checks. No pending or failed checks remain. - Live Daytona/model acceptance and recovery of the hosted task remain unverified. This PR does not deploy the change. ## Risks - Releasing preparation before execution starts renewal could leave a gap in authority. The callback runs after the executor registers its reference. - Preparation errors must not leak renewal. The recovery promise retains idempotent final cleanup. - Wire callbacks can run during subscription. Message and close regressions check that registration is complete before either callback runs. - No schema, API, model, permission, lease duration, or retry policy changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser diagnostics. The exact serving model identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 11:18:53 -05:00
DottaandPaperclip	cc6a3296e5	fix: keep agent YAML compact and preserve omitted settings (#18168 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Advanced tab lets operators edit agent settings as YAML. > - Its first format embedded company catalogs, profile definitions, history, and instruction text. > - This made one agent document grow with unrelated company data. > - Removing those fields from the display also needs safe save behavior. > - This pull request renders a compact document and resolves sparse edits against saved state. > - Operators can change settings without copying shared data or deleting hidden values. ## Linked Issues or Issue Description Refs #2219. Follow-up to #18036. Current behavior The Advanced document expands shared profiles, embeds instruction files, and lists read-only state and unused connections. It also emits empty command lists and unset fields. Proposed behavior Show the agent's own settings and profile/instruction references. Omitted settings stay unchanged. Explicit empty values clear supported settings. A named instruction file changes only when supplied; `null` deletes that file. Reason and benefit An agent's YAML should remain readable when its company adds profiles or its instruction files grow. Every setting remains available through YAML or its owning settings tab. ## What Changed - Remove the read-only `observed` export and unused connection inventory. - Render profile-key references and instruction locations. Keep file contents and legacy prompt text out of the default editor. - Omit unset nullable fields, empty maps/lists, zero budget, standard permissions, and empty commands. - Add a sparse input schema and saved-state resolution. Preserve omitted fields, binding definitions, file contents, permissions, and channel settings. - Keep explicit clears and map/list replacement. Patch named instruction files and use `null` for deletion. Preserve hidden legacy prompt text unless explicitly changed. - Check profile references, shared edits, and file restrictions before writes. Keep revision guards, native authorization, and partial-save recovery. - Update documentation, verification maps, and regression coverage. Remove operating-system clipboard dependence from browser tests. ## Verification - All 48 jobs pass in the [final CI run](https://github.com/paperclipai/paperclip/actions/runs/40904124001) on `6dbb11d7a6`. Greptile is 5/5 on this exact commit. Both findings are resolved; there are no failed or pending checks. - Paperclip Review ended neutral because its agent finished without a complete assessment. It did not provide a code finding. Superagent Security Scan passed. - The focused suite passes 58 tests across YAML parsing, planning/saving, compact output, and the editor. - A large unit fixture with 30 profiles, 50 entries per profile, and a 1,000-line instruction file renders under 70 lines and is more than 95% smaller than the prior expanded representation. - All eight browser cases pass against real APIs and a dedicated PostgreSQL database. The compact-output case stays under 60 lines with large profiles and instruction text. It checks sparse saves and profile-key binding without changing shared definitions. The resource-save browser case also checks that unpinned skill shorthand does not trigger another sync during an unrelated save. - Browser coverage also checks resource saves/clears, secret redaction, key receipts through rename, stale drafts, lost responses, instruction relocation, and revision restore. The MCP fixture is a local target; no model run or external provider login is claimed. - Workspace build and typecheck pass. Map, Storybook catalog, and token gates pass. These checks and all eight browser cases passed after rebase onto current master, including its new Secrets regression. The merge conflict consisted of adjacent test additions; both were retained. The final skill no-op correction also passes a focused browser rerun. The mixed-resource channel regression failed before its fix and now passes, including revision restore with unpinned skills. UI typecheck passes after both corrections. - The first browser run hit a five-second test wait during an active multi-resource save. A focused rerun exposed shared clipboard interference. The helper now waits for save completion and uses scoped copy events plus keyboard input. The next full run passed. Earlier logs and screenshots remain outside the repository. Run the focused browser suite with: ```sh pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/agent-configuration.spec.ts ``` ## Risks - Omitted fields now preserve saved values. Removing a line does not clear that setting. Supply its supported empty value instead. - Instruction file maps are patches. Old exports that removed a file by dropping its entry must now set that file to `null`. Entry files and read-only assets remain protected. - Explicit lists and adapter/runtime/metadata maps retain replacement behavior. Hidden legacy prompt text is preserved unless supplied explicitly; an empty string clears it. - Shared definition edits still require explicit consent. Saves still use sequential native APIs and can partially succeed. Existing unversioned resource race windows remain. - No database migration, new dependency, lockfile edit, or workflow change. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed in this session. The work used reasoning, repository editing, code execution, and Playwright browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 11:17:17 -05:00
DottaandPaperclip	c32e1ba6da	fix(onboarding): keep approved child task handoffs runnable (#18160 ) ## Thinking Path > - Paperclip manages work through tasks, agent runs, dependencies, and human decisions. > - The first-task workflow asks for approval before it creates an execution task. > - File-producing work must run on a child task before onboarding can finish. > - A child blocked by its onboarding source cannot start while the source waits for that child. > - This change rejects that dependency during task creation and later edits, and gives the agent a correction. > - The shipped recipe shows the dependency direction and requires the actual approved inputs in the child instructions. ## Linked Issues or Issue Description What happened? After human approval, an onboarding run could create a child with the onboarding source in `blockedByTaskIds`. Creation saved a blocked child with no wake. The parent could then wait for that child. A handoff could also refer to input from the parent conversation without including the input in the child description. Expected behavior The approved child can execute before its waiting parent. Invalid first-task dependency input fails before task creation and leaves the same command key available for a corrected call. The child instructions include the required approved input and constraints. Steps to reproduce 1. Accept a proposal on a server-created first task. 2. Call native `create_task` with `status: "todo"` and the onboarding source in `blockedByTaskIds`. 3. Observe a blocked child with no scheduled wake. Block the parent on that child to expose the stalled handoff. Paperclip version or commit The regression reproduces on `f8484a9f8d1f68e9534fee4f29e00e716dc680f4`. The repair is based on current master. Deployment mode Core native and REST task creation. The same server boundary serves self-hosted and hosted instances. Related: #18080 fixed remote delivery of the selected first-task skill. This change handles the later child handoff boundary. ## What Changed - Reject a task blocked by its authenticated first-task source during creation and later dependency edits. Later agent edits also check the locked target's server-owned onboarding parent, protecting a child running under its own execution. Compare UUID identity in PostgreSQL so alternate supported UUID spellings cannot bypass the check. The error explains the correction. - Add a native child/dependency recipe with self-contained approved input guidance. Clarify the general `create_task` input descriptions and update their generated contracts. - Cover native and REST rejection, corrected retry, preserved valid dependencies and replay, and an executable recipe with exact input bytes read from the child's own context. Thread the authenticated run through native dependency edits. Keep blocker corrections independent of a new proposal decision. - Update execution semantics and the existing onboarding map entries. ## Verification - Red on `f8484a9f`: the new native regression received a created blocked child instead of the expected rejection. - Review regressions were red on the original PR head: uppercase, unhyphenated and braced source UUIDs created blocked children; a later child PATCH returned 200 and saved the reversed blocker. - Review follow-up reproduced both native and REST child-own-run bypasses on `75fa75a6`: the native dependency call applied the reverse edge; the child's own PATCH returned 200. The regression uses the actual authenticated child execution and an already checked-out task. - All 95 tests across first-task admission, assets, and the full native tool-authority suite pass. The shipped recipe is validated against the native schemas and exercised through real task/dependency services with disposable PostgreSQL. Rejected PATCH tests preserve existing authorization and attempt-audit behavior and require no requested task-field, relation or wake change. Compatibility controls retain ordinary manual parent blockers, explicit board edits, other blockers and empty-list corrections. - Runner TypeScript build, generated contracts, capability inventory, protocol manifest, and map checks passed. - Server typecheck and all 95 focused tests pass with the child-run follow-up. All latest checks pass on `f24509ae`: 55 successful checks, one neutral review check, four skipped checks, and a successful legacy security status. Both automated reviews are 5/5 on this exact commit. - Prior-head CI failures are preserved: the Runner close fixture had unconfirmed provider drain while suspension succeeded; its exact fake-provider subprocess row passed locally with unchanged sources. The browser outage test lost its title textarea before any outage write. A bounded local control with a fully loaded composer passed every original outage assertion. The prior failure is an editor-readiness race; its exact asynchronous ordering remains unproved. No UI, browser-test, or timeout change is included. Neither failure is counted as a pass or silently discarded. - The first `f24509ae` CI attempt had two interrupted jobs: Typecheck and General server 9/12. Their logs record runner shutdown signals. The server job also records lost runner communication. The same-head failed-job rerun passed both jobs and the required aggregate. It reused the other successful results. No source, test, or timeout change was made for the rerun. The failed attempt remains preserved. Full repository checks pass in CI; the local full repository typecheck, test, and build commands were not repeated. - No external model or provider calls are used by these checks. Real-provider acceptance at this revision remains separate. ## Risks - The new rejection uses the authenticated run and the server-owned onboarding origin. Ordinary tasks, other blockers, empty-list corrections, and already committed creation replay retain their existing behavior. Blocker edits do not reapply the creation approval gate. - Nonempty blocker edits perform two indexed identity reads; the task lookup covers at most the run source and locked structural parent. Other patches and empty-list corrections add no source lookup. The update guard adds no parent row lock behind a child. Existing REST authorization checkout and attempt auditing remain in place before the domain mutation guard. - The explicit input recipe guides the model. Schema and persistence tests do not prove that a future model follows every instruction. - The tool guidance changes the generated semantic catalog identity. The maintained generated files are synchronized in this change. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with `ultra` reasoning effort, repository tools, code generation, and test execution. The context window is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and stated the available runtime detail - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the bug issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 10:57:31 -05:00
Devin FoleyandPaperclip	fd98075a22	fix(ui): show environment variables on agent Secrets pages (#18121 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators inspect agent configuration to understand which credentials an agent can use. > - Agent environment bindings can contain inline values, company secret references, and user secret references. > - The Secrets page did not mount the environment editor. Inline credentials were absent, and its empty message said no secrets were bound. > - The existing environment editor already handles these binding types and redacted values. > - This PR mounts that editor on the Secrets page in both UI layouts and clarifies the empty reference message. > - Operators can inspect and edit the configured bindings. Values remain masked, and the Runtime editor remains available. ## Linked Issues or Issue Description What happened? An agent can receive an inline credential through its environment while its Secrets page shows no matching entry. That page only rendered the secret-reference overview and API access controls. Expected behavior The agent Secrets page shows existing environment bindings as well as secret references. Saved inline values stay redacted. Saving another binding preserves existing credentials. Steps to reproduce 1. Save an agent with an inline environment credential. 2. Open the agent Secrets page. 3. The credential row is missing, although the runtime configuration contains it. Paperclip version or commit Reproduced on master `27270e9bf8b7a294cf03f8b84f4857f1b1ff05c7` before the fix. Related: [#12517](https://github.com/paperclipai/paperclip/pull/12517) proposes changes to secret assignment controls. This PR covers display of existing inline environment bindings through the current editor. ## What Changed - Show the environment editor on agent Secrets pages in both UI layouts. Keep the Runtime editor available. - Change the empty overview message to describe missing company secret references. Personal references remain visible in the environment editor. - Add native and legacy agent regressions through the actual page components. - Add a browser check for masked values, secret references, save/reload, and exact credential preservation through an opaque configuration revision. - Update the feature and agent access maps for the existing UI entry point. ## Verification - Red before the fix: the native and legacy page regressions could not find the saved environment credential row. - The focused UI suites pass: 207 tests, including all four layout/adapter page cases. - The server redaction and configuration revision suites pass: 42 tests. Native Runner environment assembly suites pass: 40 tests. - An isolated Chromium flow passes against the real local API and database. It checks a masked inline value, a company secret reference, and save/reload of an added variable. It then removes the new variable and checks that the opaque server revision matches its original value. This proves that the hidden credentials survive both saves without reading them. The initial browser attempt exposed a fixture ordering assumption; the test now compares variable-name sets because PostgreSQL JSONB does not retain insertion order. - `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, and `node scripts/check-maps.mjs` pass. - The full local `pnpm test:run` attempt found three failures in unchanged server fixtures. The chat-recovery case passes in isolation and in its 22-test group. The two workspace cases inherit `init.defaultBranch=main`; all 179 workspace tests pass when the child process uses `init.defaultBranch=master`, as the fixtures expect. The long local invocation was then stopped. Later local groups were not reached. All corresponding CI test shards pass. - [CI passes on the final commit](https://github.com/paperclipai/paperclip/actions/runs/40809064907). The first canary dry run found an npm version collision. Its single retry passed without a source change. - Apex passes at 5/5 on the current commit with no unresolved findings. ## Risks Low risk. The change reuses the existing editor and save path. No API, secret delivery, permission, or database contract changes. The tests cover masked values and existing references during save/reload. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — scoped checks pass; the full local command limits are recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 08:32:30 -07:00
DottaandPaperclip	7e27bc5007	fix(tasks): keep anchored opens compact and stabilize saved cards (#18092 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task pages must make the current instruction and recent conversation readable quickly. > - Compact startup avoids reading all earlier messages and run activity. > - Every URL fragment previously selected the full-history path, even for documents. > - Saved document cards also arrived after the conversation was visible. PR statistics could add another row later. > - This pull request keeps anchors compact and reveals the saved conversation structure together. > - The benefit is stable reading position without loading unrelated history. ## Linked Issues or Issue Description Related: #17593 introduced compact startup. #14667 covers earlier reveal and scroll work. This fix retains the bounded history contract. What happened? Opening a task document link loaded all task history. Ordinary cold opens could reveal messages before saved document cards and plans arrived. Later PR metadata added a statistics row and changed card height. Expected behavior Every URL fragment keeps compact startup. Document links open their pane. Older-comment links fetch only the target comment. Saved cards keep their geometry after first reveal. Steps to reproduce 1. Create a task with many comments, a saved plan, documents and saved PR work products. 2. Delay the enriched task response and open the task. Observe cards inserted after messages appear. 3. Open a document fragment. Observe full comment, activity and run reads. 4. Delay PR metadata refresh. Observe each card grow when diff counts arrive. ## What Changed - Remove fragment and tab checks from automatic history expansion. Resolve validated comment UUIDs through the authorized single-comment endpoint, encode their path components, retain fetched targets across document navigation, and preserve cancellation/deletion cache updates. - Include the saved plan, document metadata and local work-product projection in the core task read. Skip recovery, workspace and related-task enrichment. Do not refresh providers. Summary queries no longer select other document bodies. - Seed saved work-product and plan queries from core, including already pending or errored reads, without replacing known lists or an authoritative empty result. Coordinate the bounded recent reads, saved cards, latest transcript and settled initial pane width before first reveal. Keep the thread and composer mounted during refresh and explicit expansion. - Open continuation summaries in the document pane. Reserve PR statistics geometry while provider metadata loads. - Add a reusable synthetic local task, nine browser checks, frame measurements and video. Extend the PostgreSQL core-route fixture with a saved PR row and assert its identity, URL, state and stored metadata survive the response. Update design guidance and capability maps. ## Verification - Red: visible document/plan cards changed from 0 to 1 to 4; thread height grew from 690 to 1,432 pixels. Green: all seven saved cards appear before reveal, with constant thread height and zero per-card movement in the final reviewed browser run. The final 1,492-pixel layout includes the reserved PR statistics lines. - Red: delayed PR diff counts changed all three cards from 62 to 82 pixels. Green: their heights stay at 82 pixels before and after metadata refresh. - Nine browser journeys pass: delayed enrichment, document/comment/unknown/continuation anchors, document clicks and explicit history expansion with a retained draft, missing-comment retry, delayed PR statistics, and retained older-comment position after document navigation. - 583 focused UI tests passed on `c91fa107ba`. After the plan-cache follow-up, all 212 affected page/cache/plan-hook tests and UI typecheck pass, including 14 plan-hook cases. The original 16 focused PostgreSQL tests passed for the server changes. - Merge-review red: holding the duplicate stored work-products read blocked the first reveal past 10 seconds despite core already containing the saved cards. Both empty and populated core projections now reveal while that request remains pending. Malformed-comment tests prove no target request is made; API tests prove path encoding and abort-signal forwarding. - The populated core-route test now inserts a real saved PR work product alongside the plan and documents, then checks the complete saved projection through HTTP. All ten task cold-open PostgreSQL tests pass. - Plan-cache review red: an existing pending or errored plan query prevented the saved core plan from reaching the consumer. Four real-hook cases failed before the data-presence fix and pass afterward, for both present and absent plans. A separately fetched newer plan or known absent plan remains authoritative. - Further layout red: the initial properties pane reduced visible thread width from 775 to 742 pixels after reveal. The reveal now waits for two stable width frames; the browser check includes width, height, card counts and per-card geometry. - `pnpm -r typecheck` and `pnpm build` passed for the original change; UI typecheck and build pass after review fixes. Token, module-boundary and map checks pass. `pnpm test:run` was stopped after about 15 minutes in the serialized server lane. No full local test-suite pass is claimed. On the earlier head, hosted build/typecheck and all Vitest lanes passed; one E2E shard failed because the agent-configuration clipboard helper copied page text into its YAML parser. On `c91fa107ba`, Greptile was 5/5 and every E2E shard passed, including the earlier clipboard case. One unchanged Discord chat-binding test in server shard 8 returned false after 185 ms; its 16-test integration file passes locally without changes. The CI cause remains unestablished. On `be436ce492`, both server shard 8 and E2E shard 7 passed without changing those tests or their product code. On final head `e357b3bced`, all CI jobs pass, including full general and serialized tests, all eight E2E shards, typecheck, build and the aggregate verification gate. Exact-head Greptile is 5/5 with no findings. The overall status rollup contains 55 successes, two intentionally skipped Storybook checks and a neutral Paperclip Review. That advisory review ended without a complete assessment; all previously posted findings are resolved and it added no current findings. - Test startup exceeded an initial 5-second deadline once while development assets and other checks were active. The final browser checks allow 10 seconds. These runs establish geometry and bounded requests; they do not establish a reliable latency improvement. An initial 503 error fixture exercised global reconnect/backoff. The target-comment failure test uses a terminal 404. - Browser data is synthetic. Provider metadata is simulated for the delayed-response check. No staging deployment or live provider execution is claimed. ## Risks - Core task reads now include local saved-card queries and the saved plan body. Other document bodies and all historical run activity remain outside that path. - First reveal waits for the bounded current structure. The existing 15-second incomplete-history fallback and inline retry remain available. - Continuation-summary links open in the document pane in the current task interface. Classic tasks keep their existing surface. - No migration or mutation permission change. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code editing, shell tools and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 09:13:01 -05:00
DottaandPaperclip	b81b870732	Add QA smoke journeys and OpenCode run traces (#18098 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify saved application behavior across deployments. > - Task creation and questions need reusable smoke selections and exact run attribution. > - Operators also need to inspect the OpenCode execution behind a measured question. > - Company observability does not currently support native OpenCode. > - This change adds the journeys and a bounded projection of normalized OpenCode run events. > - The projection follows the configured capture policy and retains the current run identity across warm reuse. ## Linked Issues or Issue Description Related foundation: #17262. Related alternative: #17989 exports metadata from OpenCode's native OTLP stream. This change uses normalized run events to support the existing content switches and per-run linkage. It does not forward the native stream. The two PRs overlap in launch registration and must be reconciled before both land. What existing behavior does this improve? Reusable Product E2E smoke selections and company observability for native OpenCode. Current behavior The journey catalog lacks reusable task creation and attached-agent question checks. Native OpenCode is marked unsupported for company observability, so a successful destination connection test does not prove that agent traces will appear. Proposed behavior Add explicit task-create and ask-question selections. Verify saved state and exact company, issue, agent, model, environment and successful run attribution. Export normalized OpenCode turns, tools and usage to the configured OTLP destination. Honor each capture switch and redact known secrets. Reason and benefit CI can run a small measured smoke selection, retain its screenshots and checks, and link results to traces belonging to the same run. Breaking changes Existing selectors remain unchanged. OpenCode export is opt-in through existing company settings. Legacy OpenCode stays unsupported. ## What Changed - Add reusable task creation and question journeys, configured-agent capability, owned-fixture checkpoints and strict evidence validation. - Add an OpenCode-owned projection of normalized run events. No provider SDK or raw payload schema enters heartbeat coordination. - Extend the private launch union and reserved environment allowlist for native OpenCode. With export disabled, no receiver request runs. - Keep destination headers in the parent wrapper and out of the actual provider child environment and configuration. - Bound each turn to 64 tool and 64 usage entries, 8 KiB per text field and a 1 MiB request. Use a three-second request deadline, four concurrent exports, and no retries or idle worker. - Pass the rotated runner authority to each proxy turn. A retained process no longer attributes later turns to its original launch run. - Update settings copy, observability documentation and capability maps. ## Verification - `pnpm -r typecheck` and `pnpm build` passed after the implementation changes. - Adapter launch/redaction: 9 tests passed. Projection and actual proxy: 12 tests passed before the review fix; the expanded projection suite passed all 12 cases after the fix. The proxy test exercises two runs in one process against a real local HTTP receiver and a fake provider. - Full OpenCode driver/projection suite: 72 passed, one existing response_wake test timed out at five seconds. A focused check then passed both existing omitted-continuation cases and the new trace export case. The timeout cause remains unconfirmed. - Runner package TypeScript and Rust `cargo check` passed. Map and UI token checks passed. - Prior journey validation: 273 tests across 12 files and the journey TypeScript check passed. - Broad local `pnpm test:run` encountered native-runtime integration failures with repeated embedded PostgreSQL initialization failures and was stopped. All 16 failing cases passed in an isolated run outside the filesystem sandbox (961 unrelated cases unselected). The broad local run is not a clean acceptance result. A later focused database-backed chat/heartbeat run passed all six selected observability cases, including native OpenCode launch and successful allocation (1,809 unrelated cases were unselected). - All CI gates passed for `21085f880d8babc9efeb22d0623db99613d33093`, including packaged-runner, clean-install, browser and build checks: https://github.com/paperclipai/paperclip/actions/runs/40884658995 . Greptile reviewed the same commit at 5/5 with no unresolved findings. - Verified OTLP JSON acceptance and indexing with an explicitly synthetic transport marker. No live OpenCode model trace or deployed smoke acceptance is claimed yet. ## Risks Export is best effort. Abrupt termination, recovered in-flight turns without a new submitted event, limits or receiver failure can lose traces. Export failure does not change task success. The projection represents observed normalized events, not raw native model spans or hidden reasoning. Known-secret redaction is not an OS isolation boundary. UI changes can break journey locators. Deployment adapters retain credential, ownership and cleanup responsibility. Both the controller and native runner bundle must be updated before live qualification. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution and browser tools. The exact serving model ID and context size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 08:57:12 -05:00
DottaandPaperclip	7d8d1fbc1d	fix(adapter-utils): provide a default author for local commits (#18094 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/209/changes/paperclip/209 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use managed launchers for Git and GitHub commands. > - These launchers exclude host credentials and host Git identity. > - A checkout without an author cannot save a local commit when managed GitHub identity is absent. > - A local author default belongs in the launcher that owns this isolation. > - This pull request supplies a private Paperclip default and preserves explicit and managed identity selection. > - Agents can save local work without manual author setup or extra GitHub access. ## Linked Issues or Issue Description What happened? An ordinary `git commit` fails with `Author identity unknown` in a fresh checkout without author settings or a usable managed GitHub identity. An agent must set its own author to continue. Expected behavior The managed launcher supplies a default local author. Repository and command settings override that default. A current managed identity retains its existing precedence. Steps to reproduce Run the managed launcher in a fresh Git repository without author settings or broker credentials. Add a file, then run `git commit -m work`. Paperclip version or commit Reproduced at `063cfd43813cabf18417605fb9eefb9aacf95c8d` on Linux with Node 25.9.0 and pnpm 9.15.4. Deployment mode Managed sandbox execution. The shared launcher also serves local and SSH execution that selects managed access. Related: #16662 changes precedence for a connected GitHub account. #8945 assigns identity to managed worktrees. This repair supplies the missing anonymous launcher default and keeps the current managed precedence. It does not require either change. ### Hypothesis and thesis for a fix Hypothesis: The launcher removes inherited author variables and host Git config. Without a broker identity or explicit repository settings, it has no author. Four real-Git cases reproduce the reported error on the base revision. Thesis for a fix: Put `Paperclip <noreply@paperclip.ing>` in a private Git config for each operation. Git treats it as global config, so repository and command settings can override it. The existing broker environment still takes precedence. Do not store a person's identity in shared checkout config, where it could remain after steering or revocation. The default grants no GitHub access. ## What Changed - Write the default author to the existing private operation directory with mode `0600`. Remove it when the operation exits. - Point managed Git at this file while keeping host global/system config, credentials and SSH identity excluded. - Add real-Git regressions for missing, absent, unavailable and rejected access. Preserve explicit overrides, identity changes and revocation controls. - Verify two shell commits through normal launcher staging and a commit after anonymous heartbeat cleanup. Check saved files, commit metadata and temporary config cleanup. - Update identity documentation and the two run map recipes. ## Verification - Red: the four new local-commit cases fail on the base launcher with `Author identity unknown`. - Green: all 80 focused tests pass in one run across `github-launcher.test.ts`, `github-launcher-environment.test.ts`, `heartbeat-github-launchers.test.ts` and `native-github-access.test.ts` (23.31 seconds). - A controlled filesystem failure first made the candidate discard valid managed credentials. After the correction, the same real launcher process retains them. Default-author file failure does not change operation-directory readiness. - Review found a second failure: a partial write can leave invalid Git config. The real-process test failed before the repair with `fatal: bad config line` from `git status`. The launcher now selects the default file only after the write succeeds. Both write-failure cases retain managed credentials, allow status, require identity when no default is available, and save a commit with explicit identity. The test reads the stored commit metadata with real Git. - Real shells, generated launchers and Git create and inspect saved commits. Sandbox transport and broker replies use controlled fixtures. No live cloud rollout or GitHub write acceptance is claimed. - The isolated test process excludes inherited task launcher paths, Git config and broker authentication variables. An initial run exposed those test-environment inputs; the isolated replay passed. The heartbeat assertion now checks the private default file and its cleanup instead of expecting `/dev/null`. - Adapter-utils typecheck and build pass. Map validation and `git diff --check` pass. - Conflict repair on 2026-10-29: merged master `fa5e70faf47ef2ca20bf2daf95b7503c94469b89` into the PR branch. Resolved 19 conflict paths by retaining master for unrelated files. All seven Git identity files remain byte-for-byte equal to the reviewed fix. The final diff is seven files, 145 additions and 15 deletions. - At head `c501e3f28369d404c91082b89188dc150c9362a9`, all 80 focused tests pass again (37.53 seconds). Adapter-utils typecheck and build pass. Map validation and migration checks pass. The PR diff passes `git diff --check`. - Full workspace typecheck and build were attempted again. Both stop because this sandbox has no Rust `cargo`. The duplicate full local Vitest run was stopped after about five minutes without a final result while full CI was running. The earlier full local attempt was stopped without a final result. No local full-suite pass is claimed. - GitHub now reports the PR as mergeable. All current-head checks have completed. [Full CI run 40878973679](https://github.com/paperclipai/paperclip/actions/runs/40878973679) passes, including typecheck, build, Rust, test shards and all eight E2E shards. The exact-head check inventory has 54 successes, one neutral Paperclip Review and two Storybook skips. Snyk passes. Greptile and Paperclip each reviewed `c501e3f28369d404c91082b89188dc150c9362a9` at 5/5 with no actionable findings. All review threads are resolved. GitHub reports the PR as clean and mergeable. - The prior CI run had one failed E2E test: `a lost creation response requires reload and cannot blindly replay commands`. Its YAML parser received page text that starts with `Skip to Main Content`. The new run passes E2E shard 7/8 and the full E2E gate. The prior failure remains part of this record. A passing run does not establish the earlier failure's cause. This change does not alter that test or the agent configuration UI. ## Risks - Commits without explicit or managed identity now use the generic Paperclip author. Operators can still set repository or command identity. - `git config --global` writes to temporary operation config and does not persist. The documentation states this behavior. - If the private config directory cannot be prepared, commits still need explicit identity. Existing redacted diagnostics and credential isolation remain. - One small synchronous file write is added to a directory the launcher already creates. No new subprocess, network request, persistent identity or authority cache is added. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, shell execution and tool use. The runtime does not expose the exact model ID or context window size to this agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 08:56:12 -05:00
scotttongandPaperclip	71f0c0c99e	feat: add opt-in Coordinator workspace for chat, tasks, and decisions (#17983 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/147/changes/paperclip/147 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People coordinate work through conversations, tasks, and requests for input. > - These surfaces must share task state and preserve the context of each decision. > - The existing workspace makes people switch surfaces to follow that context. > - This pull request adds an optional Coordinator workspace with linked conversations, tasks, documents, and decisions. > - Each person can enable it in Settings > Experimental and return to the classic workspace. > - The benefit is a coherent path from a request to an assigned task and reviewed work. ## Linked Issues or Issue Description Subsystem affected UI, server, shared contracts, and database. Problem or motivation People need a consistent way to start tasks from agent conversations, answer questions, review work, and return to the source context. Decisions and task state must agree across these surfaces. Proposed solution Add a personal, default-off workspace experiment. Keep one conversation per agent. Add task proposals, compact task and decision cards, contextual document panels, and a Needs You queue with search, filters, sorting, grouping, and history. Alternatives considered Replacing the classic workspace for everyone would remove the opt-out path. Keeping separate decision state in each surface would create conflicting answers. This change uses the existing canonical task and interaction records instead. Roadmap alignment This supports the existing agent chat, task management, and human review capabilities. It is an experimental presentation choice. The user requested this isolated implementation after design review. No Discord discussion is claimed. ## What Changed - Add per-person workspace preferences, a default-off setting, and an additive migration. - Add Coordinator navigation, pinned destinations, task lists, agent switching, and contextual panels. - Link task suggestions and creation to the source conversation. Default new task ownership to the current agent unless the user specifies another owner. - Share pending questions, decision receipts, exact review targets, and history across tasks, chat, and Needs You. - Preserve task access, assignment authorization, revision binding, and required output review gates. - Add compact document links, relative timestamps, composer controls, and reduced-motion support for task and decision transitions. - Add component stories, regression coverage, and capability-map documentation. ## Verification - Final head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, based on master `8bc051245f77a47bee0f8ba45896b8bfa5922e0a`. Verified October 29, 2026 at 00:10 UTC: MERGEABLE, 56 checks passed, four conditional skips, no pending or failed checks. [Current CI run](https://github.com/paperclipai/paperclip/actions/runs/40790639688) passed without a rerun on this head. - Fresh Greptile 5/5, zero new comments and zero open review threads; Superagent passed with no suspicious changes. [Checks](https://github.com/paperclipai/paperclip/pull/17983/checks). - Local workspace typecheck/build passed after the latest rebase. The final account-read patch passed UI typecheck/build and all 12 related preference/settings tests, including a delayed real-client request across account changes. Earlier scoped checks cover review deduplication/replay, decision deep links, cross-surface state, access boundaries, and migration replay. - Local database/browser rechecks after the latest rebase were limited by exhausted macOS shared-memory IDs: `initdb` failed before database creation. The local run passed 31 UI/snapshot tests and skipped 24 database tests; those skips and the blocked local browser start are not claimed as passes. The independent Linux CI database and browser checks passed. No running dev database or host setting was changed. - Upstream migration 0403 is preserved. Preferences and durable review retry aliases are generated as replay-safe migrations `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`; five newest snapshots are retained. Every detected conflict was rebased and pushed before waiting for CI/review, as requested. - The PR is unmerged. The active master ruleset still requires code-owner approval for the changed Paperclip skill; no approval or rule change was performed. <details> <summary>Earlier verification failures, integration history, and evidence</summary> These entries preserve prior checkpoints and failures; the current acceptance result is above. - Focused UI and server checks from the isolated build are recorded in the implementation plan. - Browser checks covered the opt-in shell, navigation, task/document links, and decision card states. No new live provider trial is claimed for this handoff. - Current acceptance head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, rebased onto master `8bc051245` immediately after its GitHub review-repair migration took 0403. Preferences are generated as idempotent `0404_tense_silver_sable.sql`, preserving upstream SQL/journal/snapshot history and five newest snapshots. GitHub reports MERGEABLE after the immediate lease-protected push. Scott's updated instruction is to rebase and push every conflict resolution first, without waiting for CI or Greptile; verification follows the push. - The final reviewed feature tree was consolidated for repeatable single-commit rebases. The complete previous commit history remains reachable through local recovery ref `codex/coordinator-recovery-before-rebase-20261028` (resolved snapshot `0e1bbdf4e`). The actual rebase completed, and its tree hash exactly matches that resolved snapshot. The force-with-lease required the known remote head `0e6ef7c44`, protecting against concurrent branch changes. No PR merge or permission/runner setting change occurred. Fresh CI/review are in progress for the current head. - `2001132fa` received Greptile 5/5 with one nonblocking account-switch read finding. Current head detaches path-coalesced preference GETs and disables HTTP caching, retaining React Query's account-scoped sharing. A delayed real-client GET regression proves account B can load its saved choice before account A responds, and A's late response only updates A's cache. Preference/settings tests 12/12 and UI typecheck passed; its review thread is resolved. Fresh current-head CI/reviews are pending. The prior CI chat shard 3 ended with “The operation was canceled” and no assertion failure ([job 123171595631](https://github.com/paperclipai/paperclip/actions/runs/40789688333/job/123171595631), `/tmp/pr17983-2001132-chat3.log`); no code/timeout change is attributed to that cancellation. - `843e823b4` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass. All browser tests including the repaired YAML shard passed, and 54 checks passed. General server shard 9 ended with “The operation was canceled” and no failed assertion; its test step was marked cancelled (job [123164350893](https://github.com/paperclipai/paperclip/actions/runs/40787501761/job/123164350893), log `/tmp/pr17983-843e-server9.log`). A single unchanged-head rerun was started after the workflow completed, but master moved while it ran. A local nested-clone check timed out in its 30-second database-startup hook and then failed cleanup before the 20 database cases ran; this is retained as a local verification limit (`/tmp/pr17983-843e-nested-clone-check.log`), not claimed as a pass or proof of the CI cancellation cause. - Following Scott's instruction, the new conflict was rebased and pushed immediately without waiting for that rerun. Upstream's `0403_natural_cerise.sql` is preserved; regenerated preferences and retry-alias migrations are `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`, replay-safe with five newest snapshots retained. The YAML helper conflict combines upstream's click and header behavior with the editor-selection guard and injected regression. Recovery ref `codex/coordinator-recovery-843e823b4` preserves pre-rebase work. Current head is MERGEABLE. Full workspace typecheck/build passed; UI and snapshot checks passed 31 tests, but 24 database tests were skipped by the local support probe, and the local browser server could not initialize PostgreSQL. A direct disposable `initdb` diagnostic established the host limitation: `could not create shared memory segment: No space left on device` (`shmget`, shared-memory IDs exhausted, not disk space). No dev database or host setting was changed. Diagnostics: `/tmp/pr17983-2001132-initdb-probe.log`, `/tmp/pr17983-2001132-db-probe.log`; checks: `/tmp/pr17983-2001132-tests.log`, `/tmp/pr17983-2001132-browser.log`, `/tmp/pr17983-2001132-types.log`, `/tmp/pr17983-2001132-build.log`. These local skipped/blocked checks are not claimed as passes; fresh CI on independent runners is in progress. - `d18f683cd` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass, but [E2E shard 7](https://github.com/paperclipai/paperclip/actions/runs/40785966243/job/123159719925) failed parsing an Advanced YAML clipboard read beginning with “Skip to Main Content.” The log is retained at `/tmp/pr17983-d18f-e2e7.log`. Current head hardens the test helper to verify editor focus and selection, clear stale clipboard content, and require the YAML header at the start of a fresh copy before parsing. Saved-state assertions and timeouts are unchanged. All six Advanced YAML browser journeys passed on a disposable local instance (41.0s); the exact lost-response journey also passed with a one-time injected surrounding-page copy, verifying rejection/reacquisition (17.6s). The precise browser focus/clipboard timing in CI remains unproven; the observed invalid-read path is now guarded and deterministically exercised. Logs: `/tmp/pr17983-yaml-clipboard-scope-browser.log`, `/tmp/pr17983-yaml-clipboard-injected-browser.log`. Fresh current-head CI and reviews are pending. - `19e6dcb0b` passed Superagent and Greptile 5/5, with one nonblocking deep-link finding. Current head preserves attention links using `?decisionId=`: the requested pending decision is rendered even beyond the first 40 cards, focused and scrolled into view, then only that parameter is consumed so later refreshes do not refocus. Other URL parameters remain. Needs You tests 10/10 and UI typecheck passed; its review thread is resolved. The preceding post-rebase regression run passed 45/45 tests. Fresh current-head CI/reviews are pending. - `6d772b7cc` reached Greptile 5/5 with zero comments/open threads and Superagent pass. While 32 checks had passed, master moved and caused a documentation conflict in `maps/feature-map/onboarding.md`. Rebased immediately, preserving both onboarding paragraphs. The complete four-commit branch was pushed as `19e6dcb0b` before waiting on CI/review, per Scott's instruction. A transient Git index lock interrupted the first continue; the command sequence briefly pushed intermediate `34e298c07`, then the completed rebase was immediately pushed. All feature and security/retry fixes are present; local recovery ref `codex/coordinator-recovery-6d772b7cc` preserves the pre-rebase history. The only tree differences from that ref are upstream's two smoke-test files and the combined documentation paragraph. GitHub reports MERGEABLE. Fresh current-head verification is in progress; prior-head review results are not current acceptance. - `21efa3120` passed Superagent but Greptile scored 4/5 for a valid retry-key persistence bug: an acknowledged duplicate key could reopen a resolved review. Current head adds an additive, replay-safe `0404_light_swarm.sql` migration for indexed, task-scoped aliases from acknowledged retry keys to the canonical interaction. Alias registration and retry lookup occur under the existing task lock; payload/actor mismatch still conflicts, and each canonical review permits at most 100 aliases. Original and acknowledged keys replay the terminal result rather than creating a new gate. Five newest migration snapshots are retained. Service/routes tests 165/165 passed; final quality review plus migration replay/snapshot checks 25/25 passed; server typecheck passed. The P1 thread is resolved in code; fresh current-head reviews and CI are pending. - `69c2c7543` passed 55 checks and Greptile 5/5 with no open Greptile findings, but Superagent check [123140675812](https://github.com/paperclipai/paperclip/pull/17983/checks?check_run_id=123140675812) correctly flagged unbounded pending quality-card creation. Current head fixes that under the existing task-row lock: equivalent requests with fresh idempotency keys reuse the pending card; conflicting requests for an unresolved deliverable return the existing interaction ID; pending cards are capped at ten per task across all reviewers. Distinct outputs remain independent, and already-answered items in a partially pending batch can receive a fresh review. Added real-database concurrency, cross-card duplicate, partial-answer, and hard-cap regressions. Quality review 22/22 tests and final server typecheck passed; earlier service/routes 163/163 tests passed before the final partial-answer refinement. One added assertion initially expected a service HTTP error, but duplicate keys are rejected earlier by the existing shared schema; the assertion now verifies that validation. Fresh current-head CI/reviews are pending; the action-required security result is not treated as passed. - `c9c575e5f` completed all 56 checks successfully with four conditional skips, Greptile 5/5, and Superagent pass. Its two P2 review findings are fixed in the current head: independently bound quality approvals now each remain visible in attention; changing/clearing URL-backed assignee/status filters updates the URL and survives task navigation. Both review threads are resolved. Regression evidence: attention service 34/34 tests passed, CoordinatorTasks 20/20 passed, UI and server typechecks and the full workspace build passed on `69c2c7543`. The first new UI test run caught an assertion racing React Router's transition; waiting for the navigation before asserting passes without weakening the navigation/remount checks. These older-head green checks are not claimed for the new fix commit. - Preceding `0e6ef7c44` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, and passed Superagent. A fifth migration-metadata collision arrived before the final Runner job finished. Those green results are preserved as prior-head evidence, not current acceptance. - Previous fully verified migration head: `0e6ef7c44696a377f35e69521b93587576374bd0`, integrating master `f8226b051` after the fourth migration-number collision. Master added terminal GitHub-response migration 0401 while the final native check ran; upstream 0401 SQL/journal/snapshot is preserved unchanged, and preferences are generated as idempotent `0402_yielding_blink.sql` on top. Exactly the five newest snapshots remain. GitHub reports MERGEABLE after the push. All 1,155 relevant migration/review/interaction/native-execution/remote-assets tests, workspace typecheck/build, migration guards and whitespace checks pass. Fresh exact-head CI/Greptile/Superagent are pending. - Immediately preceding head `36c67cf03` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, plus passed Superagent security. No retry was needed for that head's CI. Master advanced and introduced only a migration-metadata conflict while its final native check ran, so that green result does not establish acceptance of the current integration. Four consecutive collisions (0398, 0399, 0400, 0401) during verification are retained as concrete integration churn; a brief pause of competing migration merges was requested, without assuming it happened or changing any repo setting. The PR has not been merged. - Prior fully verified head: `36c67cf03`, on the clean `30059c0ed` integration of master `efd65c112`. The last Greptile P2 is fixed: Needs attention retains the participant-agent scope through authoritative server membership queries for each pending task, including historical participation when another agent currently owns the task. It does not approximate participation with current assignment or enumerate every company task. The actual view-switch regression and all 31 focused task/routing/collection tests, UI typecheck/build, token gates and whitespace checks pass. Greptile completed a fresh 5/5 review on this exact head (review 19) with zero new comments and no open review threads. CI and Superagent remain in progress; no current-head CI failure has been reported. - Head `30059c0ed` received Greptile 5/5 with that one nonblocking participant finding, now fixed. Superagent passed that head. Current-base integration restored the ephemeral RunsOn fleet after a superseded workflow retained queued aggregate jobs: normal cancellation was insufficient; the single force-cancel request was interrupted but readback confirmed that obsolete run cancelled and current CI began on the trusted fleet. No cancellation retry or runner/security/workflow changes were made. Its partial CI was superseded by the necessary participant fix and is not current acceptance. - Previous compatibility/base-sync head: `30059c0ededc0eab4e04e8299a884f36db8b292f`, integrating master `efd65c112` and the Inbox/workspace compatibility fixes. GitHub reports MERGEABLE after the push. Coordinator keeps Inbox standalone under More without rewriting the instance's Combined Inbox preference; opting out restores that preference. Saved merged-Inbox URLs resolve to their original owner, task status presets remain usable, and canonical repeated `workspace` parameters retain scoped, deduplicated pagination (including Needs attention). All 192 broader routing/Inbox/sidebar/privacy/review tests and the final 16 task-page tests pass, as do workspace typecheck/build, token gates and whitespace checks. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The preceding `d5b9be5bd` remained mergeable but its CI routing gate intentionally fell back to GitHub-hosted runners: “stacked base merge commit does not contain the current base branch.” All 45 downstream jobs stayed queued. Current master was merged cleanly to refresh the pinned base; no runner/security settings or workflow logic were changed. Greptile scored that preceding head 4/5 with the two compatibility findings fixed here. Its automatic Superagent scan was inconclusive at checkout; the documented retry was still pending when superseded. Those results do not establish acceptance of this current head. - Previous approval/migration head: `d5b9be5bd3b8d6b0437e42bf48138377f34fa67d`, integrating master `ba5bba86d` and the stale-output approval correction. GitHub reports MERGEABLE after the push. The fresh P1 finding is fixed: every submitted item now validates and locks its bound output inside the resolution transaction, whether or not it carries a required-quality-review binding. Real database regressions verify that an advanced document revision or deleted attachment refuses the entire batch without recording approvals, and current ordinary output reviews still resolve. All 272 combined migration/review/interaction tests pass; workspace typecheck/build passed. Fresh exact-head Greptile and CI are pending. The automatic Superagent scan ended inconclusive at checkout; one documented on-demand retry is pending. These are explicit verification limits, not passed security results. The P1 thread is resolved. - Master consumed the next migration slot three times during this verification pass: confirmation recovery took 0398, continuation recovery took 0399, and GitHub acknowledgement recovery took 0400. Each upstream SQL/journal/snapshot is preserved unchanged. Preferences are now generated as idempotent `0401_white_lenny_balinger.sql` from upstream 0400; exactly five snapshots remain, and replay coverage locates this migration by table content. This is recorded concurrent integration churn, not a claim that older green results apply to the latest base. - Head `ed3b68da9` fixed the two history/URL findings with 26 focused regression tests, server/UI typechecks, workspace build and token gates passing. Greptile then scored that head 4/5 with the stale-output P1 corrected in the current head. Its partial CI is superseded, not counted as current acceptance. - Previous filter-fix head: `ed3b68da9`, based on the `8458d39d9` merge of master `063cfd438`. The two fresh Greptile findings are fixed: history outcome filters now match recorded per-item verdicts in SQL before pagination (mixed reviews remain in both applicable outcomes), and same-page URL assignee changes/back/forward navigation update that filter without clearing independent filters. All 26 focused database/navigation/collection tests, server/UI typechecks, workspace build, token gates and whitespace checks pass. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The `8458d39d9` migration integration also passed all 328 targeted migration/interaction/confirmation tests, workspace typecheck/build and Superagent security scan. Greptile scored that head 4/5 with the two functional gaps fixed in the current head; those prior-head checks are not current acceptance. - Migration integration head: `8458d39d9721f4cb312c2390070a0609f8cddd1a`, integrating master `063cfd43813cabf18417605fb9eefb9aacf95c8d`. GitHub reports MERGEABLE after the push. Master advanced twice during this conflict-resolution pass: first its confirmation-recovery migration took 0398, then its continuation-recovery migration took 0399. Both upstream migrations and journal entries remain unchanged; Coordinator preferences are now generated as idempotent `0400_sad_guardian.sql` from upstream's latest snapshot. Exactly the five newest snapshots remain. The replay regression now locates the table migration by content instead of hardcoding a renumbered filename. - All 328 targeted question/review/confirmation/migration tests passed again after the second merge. Workspace typecheck/build and fresh exact-head CI/reviews are in progress. The preceding `0c46f46d6` passed workspace typecheck/build and Superagent security scan, but was superseded because master consumed migration 0399 while those checks ran. That prior-head result is not acceptance of the current head. No production settings or runner/security permissions were changed, and the PR has not been merged. - Previous integration head: `0c46f46d6f2b23417ab872ce3cb6ff2673647287`, integrating master `287b04d72f1b9fca1412e05435c75b8302ba0710`. GitHub reports MERGEABLE after the push. The renewed conflicts were `0398_snapshot.json`, `_journal.json`, and the top imports in `issue-thread-interactions.ts`. Upstream's `0398` confirmation-recovery index and journal history are preserved unchanged; Coordinator preferences were regenerated as idempotent `0399_bouncy_miek.sql`, with replay coverage updated and exactly the five newest snapshots retained. Both terminal confirmation notifications and Coordinator task assignment/required review behavior are preserved. - All 328 targeted interaction/review/confirmation/migration tests pass, as do workspace typecheck, map validation and whitespace checks. The workspace build and fresh exact-head CI/Greptile/Superagent checks are pending. The first migration-generation attempt hit the sandbox's local IPC restriction before generating files; the unchanged authorized local generation completed successfully. Prior-head green checks below do not establish acceptance of this merge. - Previous acceptance head: `8b7dcf739ed15afc51351fdfa0e0e5fbc75ab788`, with master `1aa3a21ac` integrated and GitHub reporting MERGEABLE. Greptile completed a fresh 5/5 review on this exact head with zero new actionable findings and no open review threads. The [fresh CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40752864319) completed SUCCESS on attempt 2 at 19:40:08 UTC, including all eight E2E shards. Final completed check set: 55 succeeded, four conditional checks skipped, zero pending/queued/running checks, and one neutral/inconclusive Superagent scan. The auxiliary commitperclip review and isolated native Runner compilation/cache-reuse check both passed. This is not a claim that Superagent produced a passed security verdict. GitHub still reports MERGEABLE against current master `60b8089a44b4eaa37e627adbfdf2f130bda13335`. - Current-head E2E: all eight shards and the aggregate passed, including the previously failing YAML browser journey. Serialized server shard 9/9 failed: the protected YAML projection test exceeded its original 15-second timeout; its late update request then reached the next test's reset mock (the captured payload is the prior test's model/label update). All 88 tests in that exact unchanged file passed locally in 38.03 seconds, with no timeout or assertion changes. The original slow-request cause remains unproven. CI failure log: `/tmp/pr17983-8b7-serialized9.log`; local recovery: `/tmp/pr17983-adapter-yaml-repro.log`. The single bounded unchanged shard rerun passed, and the workflow completed SUCCESS. Rerun log: `/tmp/pr17983-8b7-serialized9-rerun.log`. The initial failure is retained; a passing retry does not establish its original slow-request cause. - Greptile scored preceding head `4873704be` 5/5 with one valid P2: opened task decision records missed company-scoped live invalidation. The current head uses the existing company decision prefix for resolution and effect-completion events, and resets private record payloads on access changes. Its thread is resolved. All 107 focused record/live-update tests, UI typecheck/build, token checks and whitespace checks pass. - CI on `4873704be` failed E2E shard 7/8 in the Advanced YAML lost-creation-response journey: its clipboard helper selected page text starting with “Skip to Main Content” instead of YAML. The exact unchanged journey passed in a disposable local browser instance (1 test, 17.1 seconds). No assertion, timeout or production behavior was weakened; the original failure cause remains unproven. Failure log: `/tmp/pr17983-487-e2e7.log`; local check: `/tmp/pr17983-agent-yaml-repro.log`. The full unchanged Advanced YAML browser suite also passed locally (6 tests, 37.8 seconds). Fresh current-head CI subsequently passed this journey and all other E2E shards. - Both the automatic and one supported on-demand Superagent scan on `4873704be` ended neutral/inconclusive: “Pinned PR scan merge-base checkout failed or timed out.” These do not count as a passed security verdict. The current head's automatic scan and one documented on-demand retry also returned the identical neutral checkout failure, with zero annotations or further diagnostics. This is an external scanner blocker, not a passed security verdict. No security settings or permissions were changed. - Recorded decision links now render and focus the full source-task record in both Classic and Coordinator, including recorded choice, submitted inputs, and the existing effect-result resolver. Fresh terminal records take precedence over older open-decision cache entries. Context and account checks keep mismatched or inaccessible records out of the DOM; no decision-history tab was added to tasks or chat. The shared company live-run feed now drives live-only filtering, including ended runs and terminal tasks with stale run entries. All 213 focused tests, UI typecheck/build, token and whitespace checks pass; the final focus follow-up passed all 188 task-detail tests and UI typecheck. The first new navigation test failed because its test host omitted company context; the host was corrected before the passing run. - Previous integration head: `375c5e89516705050d94bab99336212841b2e6e1` merges master `1aa3a21ac` after four additional upstream commits. The sole conflict was competing imports in `server/src/services/issues.ts`; both quality-review policy and inbound wake-notice notifications are preserved. The PR-specific review import now sits beside execution-policy imports instead of the frequently changing first line. GitHub mergeability is rechecked after the push. - On the combined master tree, workspace typecheck/build, all 153 affected UI tests, map validation, and whitespace checks pass. Four server suites passed 77 tests; the fifth suite's module-import hook timed out while broad verification was running. Its unchanged isolated run passed all 41 tests in 7.2 seconds with the original timeout. This records recovery without claiming the original import-timeout cause is proven. CI on `375c5e895` passed every substantive job, including all eight E2E shards and Superagent; Greptile found the two functional gaps fixed in the current head. A short-lived `669d90876` commit added the main fixes before the focused navigation follow-up. - Immediately preceding head `9d9737b49` completed all 56 checks successfully with four conditional skips, Greptile 5/5 and no open findings, plus a successful Superagent scan. The signoff E2E shard passed on its single accepted rerun; its initial cancellation/timing failure is retained below. Master moved again while those checks ran, requiring this final integration rather than treating the prior head's green result as current acceptance. - Previous head `9d9737b496911a6054d0ea22d1b42b28b115e9ab` includes master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`; that head was subsequently conflicted by moving master. The master merge preserved durable account-bound writes and Coordinator UX. Workspace typecheck/build, 416 focused UI tests, and 142 affected comment-route tests passed. A stable full UI run on preceding head `938ad1665` passed all 9,833 tests across 836 files; its final full workspace build also passed. - Greptile found nested proposal project inheritance and Classic bound-revision links on `91be74952`. Both were fixed with 142 interaction-route tests, 113 focused UI tests, and UI/server typechecks. Its next review identified a valid draft-preservation bug in the new Classic loading UI. Head `9d9737b49` keeps loading/error notices outside editors and annotations while preserving dirty drafts; 115 focused review UI tests pass, including pending/failed revision loads and continued draft editing. All related threads are resolved. Greptile completed a fresh 5/5 review of `9d9737b496911a6054d0ea22d1b42b28b115e9ab` with no open findings. - CI on `938ad1665` exposed an integration regression with the newly merged durable startup saves: the personal preference gate blocked the task page before its first session lookup. The guarded title editor now mounts while that lookup is pending, with presentation-specific tools still gated. All 216 affected component tests pass. The unchanged five-test `disconnect-write-recovery.spec.ts` browser suite passes against a newly launched disposable local-trusted PostgreSQL instance on port 33219, including the exact failed startup-save journey. No live provider, hosted authentication or production proof is claimed. The helper-owned test server exited and the port has no listener. UI typecheck, built-browser assets, token and whitespace checks pass. - Initial durable-write merge tests found duplicate imports and unscoped draft fixtures, fixed before 416 focused tests passed. A sandbox typecheck could not create its IPC socket; the supported rerun passed. The first revision-link test used a wrong Return to latest button assertion, corrected before the 113-test pass. The startup E2E reproduction initially still failed because an outer shell gate also blocked the page; the subsequent fix passed all five cases without changing the test or its timeouts. A focused loading-header assertion initially used an incorrect fixture title; correcting it preceded the 216-test pass. These failures remain recorded. - Earlier routing gates fell back to GitHub-hosted runners while master was moving or conflicted. Superseded queued runs were cancelled, then the `938ad1665` routing gate explicitly selected the ephemeral RunsOn fleet. Its CI actively executed and completed most tests; only the diagnosed startup E2E shard and aggregate failed, alongside the valid Greptile finding. Auxiliary GitHub-hosted workflows also completed successfully on earlier heads, so queueing alone is not an administrative blocker. Fresh [CI on `9d9737b49`](https://github.com/paperclipai/paperclip/actions/runs/40744012678) is the acceptance target; the previously failing startup E2E shard now passes. A different signoff E2E shard failed during executor resubmission when its fixture run was cancelled as `issue_continuation_waiting_on_review`. The exact unchanged journey passed locally (1 test, 46 seconds), and the preceding head with identical server/signoff code passed that CI shard. This supports a bounded rerun but does not establish the timing failure's root cause. After the workflow finished, GitHub accepted one rerun of that failed shard and its dependent aggregate; both passed, completing all checks on that head. The failure log is retained at `/tmp/pr17983-9d9737-e2e7.log` and the local reproduction at `/tmp/pr17983-signoff-repro.log`. No assertion or timeout was weakened. - On the first merge, full workspace typecheck/build, all 9,622 UI tests, 267 tests across seven affected server suites, two migration checks, map validation, token gates, and the PR diff whitespace check passed. Greptile then found duplicate local task-search filtering and lost completed item-review detail. Both are fixed, with 42 passing focused UI tests and passing UI typecheck/build. - A subsequent Greptile review found attachment confirmations without working links and mobile history without a visible host. Confirmation layouts now derive the authenticated immutable-content endpoint from the attachment ID, and Needs You uses the existing full-screen mobile sheet. All 119 focused tests pass, including missing/stale display URLs, both review layouts, and both mobile history entry buttons. UI typecheck/build and token checks pass. Full workspace typecheck/build also pass after the second master merge. - Greptile gave head `22f69c304` a 5/5 score with one non-blocking desktop-history reopening finding. Head `6dd52b6d3` fixed it, passed all nine history tests, UI typecheck/build, token gates, and the full 9,679-test UI suite, then received a fresh 5/5 review with zero open findings. These are prior-head results; review on the master-sync commit is pending. - The chat and workspace CI jobs on head `22f69c304` were interrupted by runner shutdown signals and cancellation before completing their tests. Their logs do not report failing assertions. Those interrupted jobs are not claimed as passes. The new head starts fresh runs. - The first merge CI had a Slack session-sync idle assertion fail after an unexpected database read. Its exact isolated reproduction passes, and the chat CI shard passed on head `618c5a2a3`. The original failure's cause remains undetermined. The broader local reproduction, run concurrently with full verification, instead encountered 21 timeout failures across its two test chunks; it is not claimed as a passing full local server run. A concurrent local UI run had eight failures (seven timeouts and one delayed autosave assertion). The complete UI run on the final head passed all 9,679 tests across 830 files with two workers; no test timeout or assertion was weakened. This establishes recovery without claiming that every earlier timeout's cause has been proved. An earlier intermediate UI run also overlapped the second master merge and failed the newly added Advanced-sidebar expectation, so it is not treated as stable-tree evidence. The newly merged UI dependencies initially required refreshing the local install from the upstream package manifest; the generated local lockfile update is not committed, per repository policy. - Superagent returned neutral/inconclusive scans on the preceding heads because its pinned merge-base checkout failed or timed out. The available retry endpoint returned 404. These are not successful security verdicts. The fresh scan on `91be74952` passed (PR scan passed; no suspicious PR changes detected). The `938ad1665` scan again returned neutral/inconclusive with the same pinned merge-base checkout error; the check-suite rerun endpoint returned 404. The fresh `9d9737b49` scan completed successfully at 18:22 UTC with no suspicious changes detected. Earlier neutral scans are not counted as successful security verdicts. - Prior head `57902152ac3218096fca669d33ff18400a2a86ae` passed 9,618 UI tests across 826 files, received Greptile 5/5 with no open findings, and completed CI with 56 passes and four conditional skips. These are historical results, not acceptance of the latest merge commit. - The first full build found generated capability-contract drift. The generated outputs were refreshed. Initial sandbox runs could not open local IPC sockets; verification was rerun with local test access. - Current master was merged. The workspace preference migration was regenerated after the existing migration history. - The UI run first found two outdated expectations: early custom-answer validation and the document access error copy. Both were corrected, and the full UI suite then passed. - The local server run found three failures in unchanged tests. The Muse test passes with Python 3.13 instead of the host's Python 3.9. The two workspace tests pass when their test process uses the expected master default branch. These environment settings do not change application behavior or global machine settings. The unsharded server run was stopped after those reproductions; CI supplies the full sharded server result. No successful full local server run is claimed. - Skill inventory generation needs an external eval checkout, which is absent locally. The existing generator functions refreshed only the live skill inventory and its derived files. The checked-in eval inventory was preserved. The inventory validation gate passes. - Greptile found a dropped MCP review requirement, missing per-item output links, generic item-review summaries, and a history-refresh failure. The fixes have 32 passing database-backed server tests and focused rendered-card tests. A follow-up partial-review finding was corrected by forwarding the canonical mutation status before settling the card. Attachments open through the authenticated immutable-content route in both card layouts. The final focused UI set has 117 passing tests, including these regressions. - Both browser shards that first failed expected a disabled-chat message. The copy was restored, and those shards passed in CI on the next commit. The two journeys also passed locally. One local attempt first timed out before feature disable while waiting for the provider's initial message; its isolated reproduction passed. That earlier timing failure remains recorded and is not presented as an explained infrastructure failure. - Serialized server shard 9 first failed during pnpm setup with ECONNRESET, before dependencies or tests ran. Its single rerun passed setup and all tests. The build and production runtime boot passed on the first attempt, but that run's public-install step timed out during Pi setup. The single build-job rerun passed compilation, runtime boot, and clean public-install verification. Those setup scripts are unchanged by this PR. The successful retry establishes recovery; the earlier Pi timeout's underlying cause remains undetermined. The aggregate CI gate is green: [workflow run](https://github.com/paperclipai/paperclip/actions/runs/40637789695). </details> ## Risks - This touches shared task, interaction, and completion paths. Regression tests must cover classic and experimental behavior, authorization, duplicate answers, and stale review targets. - The additive preference table must migrate before the setting is used. The setting defaults off. - The experiment changes navigation and message presentation. Canonical task and document records remain the source of truth. - The initial task brief and saved plan can repeat information. This change fixes chronology; a broader redundancy policy remains a design decision. ## Model Used OpenAI GPT-6 Codex with reasoning and tool-assisted code editing, tests, and browser verification. Earlier iterations used Codex models in this task; their exact model IDs are not available in the current context. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used - [x] I have checked ROADMAP.md - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run relevant tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open recommendations or follow-ups - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 17:20:01 -07:00
Devin FoleyandPaperclip	c2163b3ecf	test(release): verify paused onboarding setup records (#18102 ) ## Thinking Path > - Paperclip manages agents and controls when their work starts. > - Onboarding saves an unfinished agent before account connection. > - The setup hold keeps that record paused until Connect completes. > - The published canary smoke still required an empty agent list at that point. > - That old assertion rejected the documented setup state and blocked release work. > - These tests now check the setup hold and the absence of early work. > - The nightly smoke also checks that Connect completes the same record. ## Linked Issues or Issue Description What happened? Published canary onboarding failed at the empty-agent-list assertion after reaching Connect a model. The API returned the expected paused setup record. The same failure affected recent canaries and prevented their promotion to a nightly. Expected behavior Require exactly one named setup record before Connect, paused with the `setup` hold and heartbeat disabled. Require no tasks or runs. After successful Connect, complete that same agent record. Steps to reproduce Run the published canary onboarding smoke with `PAPERCLIPAI_VERSION=2026.1028.0-canary.15`. Enter the organization and agent names, then open Connect a model. The old assertion fails on the held setup record. Paperclip version or commit The published artifact is `2026.1028.0-canary.15` at `743698e700638cddc4d3fc4c0ad89e6c132fab40`. The test branch starts at `65fa1cd617b43a45c9d22e10480f4fb6e52dbd21`. Deployment mode The canary check installs the exact public npm package in a fresh local-trusted instance. The authenticated nightly smoke uses the existing Docker harness and its provider mock. Refs #17976 and #12667. The existing agent setup contract is in `doc/SPEC-implementation.md` section 10.3. ## What Changed - Replace the stale empty-agent assertion with a bounded check for one paused setup record, disabled heartbeats and no tasks or runs. - Check the same setup state in the authenticated nightly smoke. Require Connect to complete that record without creating a second agent. - Update the onboarding verification recipe with the scope of both release checks. ## Verification - Red: the unchanged smoke failed locally against the exact published `.15` artifact at the empty-agent assertion. This matches the [release failure](https://github.com/paperclipai/paperclip/actions/runs/40761030399/job/123112508486). - Green: the revised smoke passed against the same `.15` artifact, with a real browser and fresh data. No model credential was required. - Full `pnpm build` and `node scripts/check-maps.mjs` pass. - Full recursive `pnpm -r typecheck` passes. - Full local `pnpm test:run` hit embedded PostgreSQL initialization failures. A standalone probe of the same bundled `initdb` reproduced `could not create shared memory segment: No space left on device`; this Mac had exhausted its shared-memory IDs. The local suite was stopped after those failures. No full local-suite pass is claimed. - The additional old-nightly artifact probe stopped before the browser journey with the same PostgreSQL bootstrap failure. Its failure is retained separately from the canary red/green result. - All 54 current-head checks pass; two optional checks are skipped. [CI run](https://github.com/paperclipai/paperclip/actions/runs/40779358071). - [Apex review is 5/5](https://github.com/paperclipai/paperclip/pull/18102#issuecomment-6482273653) for `40c63db4e9bf66e915843a9f325763d449aee049`, with no unresolved review threads. - The first [authenticated Linux canary smoke](https://github.com/paperclipai/paperclip/actions/runs/40779405110) stopped before the browser because the canary tag moved to `.16` before its matching server dependency was published. That dependency is now available and the failed job has been retried. Both the canary and [nightly smoke](https://github.com/paperclipai/paperclip/actions/runs/40780591068) passed background-service installation. Their authenticated Docker browser jobs are still queued for GitHub-hosted Ubuntu runners. Full authenticated browser acceptance remains unverified pending runner allocation. ## Risks These are test and verification-document changes. Unexpected agent states, extra records, early tasks and early execution still fail the checks. The authenticated smoke uses the existing mocked provider acceptance; it does not qualify real provider authentication. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. The session does not expose a more specific model ID or the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — targeted canary smoke; full local-suite limitation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 16:03:08 -07:00
DottaandPaperclip	c74171d620	fix(recovery): retire superseded native workspace retries (#18077 ) ## Thinking Path > - Paperclip coordinates agents while retaining their work and execution authority. > - A failed native run can schedule an exact-session retry and retain a sandbox. > - An explicit task Retry can already have completed successfully on a successor. > - Recovery correctly refuses the old run after its task execution lock changes. > - It nevertheless leaves that old retry scheduled and its workspace without a preservation owner. > - This change retires that proven superseded retry and registers exact-source preservation together. > - Existing stop receipts and saved-file verification still control workspace switching. ## Linked Issues or Issue Description What happened? A native sandbox run failed before starting a provider turn. An explicit Retry completed on a linked successor. The original coordinator remained in `retryable_failure` with an active sandbox lease. Restart recovery returned `issue_execution_lock_changed`, while the ordinary sweep kept delaying it. A pending workspace selection then made every new task run fail during setup. Expected behavior Stop replaying a failed retry after its linked explicit successor has completed. Preserve the original files through the existing system worker. Keep the current task, ownership, pending choice, and unresolved action outcomes intact. Steps to reproduce 1. Persist a failed, unowned native run with a prepared remote workspace and `retryable_failure` coordinator. 2. Complete a same-company/task/agent explicit Retry linked to that run. 3. Release the task's old execution lock and schedule a workspace selection. 4. Run startup or ordinary retry recovery. Observe the old retry remain scheduled and the selection fail with an unfinished-recovery conflict. Paperclip version or commit Regression reproduced against `644937c931`. The current repair is based on refreshed master `60b8089a44`. Deployment mode Native Runner with remote sandbox workspaces. The reported runtime used ACPX Claude with per-turn lifecycle. The original provider session-resume rejection is not independently reproduced or claimed fixed here; this PR addresses the subsequent stranded recovery. Related: #17839 records preservation when restart classification becomes terminal. This change handles the earlier lock-mismatch return and ordinary retry admission after a completed explicit successor. It uses the same preservation service and does not require that PR. ## What Changed - Share a narrow superseded-retry settlement between startup recovery and the ordinary retry sweep. - Require a failed source with no coordinator owner or process, its exact prepared source lease, and a completed linked successor under the same company, task and agent. - Atomically retire replay and register preservation. Do not alter task state or create another provider run. - Add PostgreSQL lifecycle/safety regressions, a real-file preservation and workspace-selection regression, and execution/map documentation. ## Verification - Red: both startup and ordinary admission regressions left the old coordinator in `retryable_failure` with no preservation obligation. - Green: the same two cases retire the retry and retain the exact lease without another run or task mutation. - The assembled PostgreSQL/filesystem regression passes: workspace selection remains blocked before preservation, the existing service saves `saved.txt` and preserves newer local work, then the normal selection guard applies the pending shared workspace. - All 90 native retention/resumption cases pass. Three neighboring startup-recovery cases pass, including the blocked-owner inventory. Full recursive typecheck and full build pass at runtime head `23ddb213b4`; final `19d7738ae3` only moves the function comment into execution documentation. Map, module-boundary and diff checks pass. The duplicate full local Vitest run was stopped after 20 minutes in its first long general shard once the complete current-head hosted test gates passed; no complete local-suite pass is claimed. - Current-head CI has 53 passing checks and two intended Storybook skips, including full typecheck, all test shards, build, Runner verification, browser shards and both aggregate gates. The separate `commitperclip PR Review` job is still queued on `ubuntu-latest` with no assigned runner; its outcome remains pending. Greptile reviewed exact head `19d7738ae3` at 5/5, with zero unresolved threads. - Live acceptance passes on exact final source `19d7738ae3`: a disposable Daytona Linux sandbox, the current immutable staging fleet snapshot, SDK 0.203.0, a Darwin arm64 source host and Node 26.4.0. The real native workspace preparation/export and provider stop/start paths preserve an unpredictable report and newer host files. The ordinary workspace-selection guard stays blocked until preservation commits, then applies the pending shared workspace. - The old workload's start count and side-effect marker each remain one. Recovery creates no new run or native result. The sandbox finishes stopped and is then deleted by the probe; a final provider read confirms it is absent. - A second live acceptance also passes on the original runtime's older immutable snapshot with the source explicitly archived before recovery. It verifies cold source reopening, the same report/newer-file checks, one workload start and side effect, pending workspace application, and confirmed deletion. Its initial probe attempt used immediate archive/deletion reads and failed during asynchronous cleanup observation; bounded provider-state polling fixed the probe without a product change. The abandoned probe allocation is independently confirmed absent. - Scope: database fixtures represent the already-failed run and completed explicit successor. The old workload is a persistent shell process with observable file/side-effect markers, not a paid Claude turn. This qualifies the repaired recovery handoff and real provider preservation; it does not reproduce the initial ACPX session-resume rejection. - The original hosted task has only been inspected read-only. No rollout or original-task recovery is claimed. ## Risks - Incorrect successor classification could retire legitimate work. Matching requires the server-persisted explicit Retry link, same company/task/agent, later successful committed execution, no current source execution lock, no recorded process, no coordinator owner, and no workspace finalization owner. - Retirement is not a stop receipt. The exact source remains retained, unknown effects remain unknown, and workspace switching still waits for the existing worker's stop and file-persistence proof. - A failed preservation transaction rolls back retirement. No new scan, retry budget, schema or public API is introduced. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. The host does not expose a more specific model ID or the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — focused suites and live acceptance; full-suite status above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — separate template-review job still queued as noted above - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — exact head `19d7738ae3` - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 15:15:19 -05:00
DottaandPaperclip	60b8089a44	fix(runner): trust managed Cloud sandbox images (#18066 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud selects and updates the sandbox image used by its agents. > - Runner admission also required saved qualification for the exact snapshot and configuration. > - An image update left the old record stale and stopped work before runtime startup. > - This pull request trusts the image selected by Cloud and uses the real runtime startup check. > - Agents can continue after an image update without an operator running a separate qualification test. ## Linked Issues or Issue Description What happened? A managed Cloud image update could cause `sandbox_snapshot_qualification_required` before provider work began. Runner discovery also fell back to Legacy when saved platform qualification was absent. The agent Test action could require a separate qualification request even when discovery already offered Paperclip Runner. Expected behavior A managed Cloud image can start without a saved qualification record. Runtime startup and the handshake report actual compatibility failures. Custom images keep their existing qualification rules. Steps to reproduce 1. Use Paperclip Runner on a Paperclip-managed Cloud sandbox. 2. Save qualification for its current snapshot. 3. Update the platform-selected snapshot and leave the old record in place. 4. Start a task. Before this fix, admission rejects the stale record before starting the provider. Related work: Refs #17445, Refs #17724, Refs #17448. These changes cover saved-computer qualification and setup tests. This fix removes the qualification prerequisite only for managed Cloud images. ## What Changed - Use one managed-image policy in runner selection and lease admission. Require the active Cloud configuration to provision the provider. Active captured custom images are excluded. - Use the current Cloud image and ignore missing, stale, revoked or valid saved qualification records for that image. - Reuse the existing provisioned-runner selection path. Keep adapter availability checks and the Legacy choice for managed sandboxes. - Read custom-image metadata directly during discovery without resolving provider credentials. - Use the existing template-matching rule for captured images. Ignore outdated captures, preserve qualification for matching custom captures, and fall back to Legacy discovery for malformed saved capture config. - Skip the separate qualification request when discovery already offers Paperclip Runner as the default. Test checks the actual runtime. - Add regressions for image updates, provider failures, custom images, self-hosted environments and company access. Update the development and verification guides. ## Verification - Before the fix, managed-image discovery regressions returned Legacy, and the rendered Test action called the separate qualification endpoint. - The original runtime source fails four of the five managed-image regression cases. The fixed runtime passes all five, including provider startup error propagation. The baseline check uses the original module source through a temporary Vitest transform and the same database-backed tests. - All 603 focused tests pass: 231 runner selection, 185 environment runtime, 10 snapshot qualification, 168 rendered agent form and nine form source controls. Discovery controls cover unavailable credentials, stale managed markers without an active provisioner, outdated captures, matching custom captures and malformed saved config. The outdated-capture regression fails before the fix and passes afterward. - `node scripts/check-maps.mjs` and `pnpm check:token-gates` pass. - Full `pnpm -r typecheck` and `pnpm build` pass. All five focused files passed together on `8275bbe72bb4d91fb3a02fb0658f0846b2abc966` (`codex/simplify-managed-runner-readiness`, Node 24.19.0, macOS ARM64). - The full local `pnpm test:run` suite is still in progress. Hosted CI is waiting at `ci / gate / Select trusted runner`; the separate `review` workflow is queued. Greptile is 5/5 on the current head, and both review threads are resolved. This PR is not yet verified by full CI. - Live fixed-source Daytona acceptance is not verified. This workspace has no Daytona API key or image configuration. The earlier deployed recovery is not evidence for this source change. ## Risks - Managed Cloud now trusts its selected image. A bad image can fail at real runtime startup instead of a saved qualification gate. - Company access, credentials, runtime integrity and handshake checks still apply. Self-hosted sandboxes, custom environments and active captured custom images keep their qualification and immutable snapshot binding. - No schema change or data migration is required. Existing qualification records can remain in place. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The exact model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 14:39:26 -05:00
DottaandPaperclip	c4b6168e08	fix(server): disclose assigned tool search page bounds (#18056 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/187/changes/paperclip/187 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the control plane for AI agents at work. > - Native agents find connected app tools through the assigned-tool search tool. > - Large catalogs require small pages and an offset for the next page. > - The input schema enforces a maximum of 20 results, but its descriptions omit that number. > - Providers can render the integer field as a plain number and hide its numeric bounds. > - This pull request puts the existing range and default in both descriptions. > - Agents can choose a valid page size before they make a search call. ## Linked Issues or Issue Description What happened? An agent can request `limit: 50` for `paperclip_search_assigned_tools`. The tool rejects the call with `invalid_tool_call` at `/properties/limit/maximum`. Its model-visible description does not state the accepted range. Expected behavior The tool description and the `limit` description must state the range of 1 to 20 and the default of 5. Agents can use `nextOffset` to read more results. Steps to reproduce 1. Inspect the assigned-tool search description in a provider tool list. 2. Render the integer schema as a TypeScript number field. 3. Request `limit: 50` without a visible page-size bound. 4. Observe the validation error. Retry with a valid size and offset. ### Hypothesis and thesis for a fix The numeric schema is correct, but the descriptions omit the page limit. Put the existing range and default in text that all providers can see. Keep the strict validation in the assigned-tool discovery owner. This should reduce avoidable retries. It does not guarantee that every model call will be valid. ## What Changed - State the accepted range and default in the search description and the `limit` property. Add the existing default as schema metadata. - Test the tool listing and calls through the real HTTP MCP bridge for both compact delivery paths. - Document the page size and offset procedure in the Runner README and verification maps. ## Verification - Red: the two new bridge tests fail against the unchanged implementation because the listing omits the numeric guidance. - Green: all 37 tests in `server/src/services/native-runtime/assigned-mcp-tools.test.ts` pass after the change. The tests cover default pages, a 20-result page, offset continuation without duplicates, and rejection of 21 and 50. Existing tests cover grants, revocation, modes, approvals, aliases, and schema chunks. - Current head `7d658d0c4e4d9db3feb53a5c508214440ff058c4`: 37/37 focused tests pass locally. Greptile is 5/5 with no unresolved threads. Current-head CI is green, including typecheck, build, unit tests, Runner checks, and all browser shards ([CI run, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/40733284523/attempts/2)). Browser shard 7 initially failed in a newly merged agent YAML editor test when its clipboard read included the whole page. The single retry passed without a code change; the exact cause is unconfirmed. The original failure remains in [the CI log](https://github.com/paperclipai/paperclip/actions/runs/40733284523/job/122975224906). - Run the focused suite from `server/`: `pnpm exec vitest run src/services/native-runtime/assigned-mcp-tools.test.ts`. - `node scripts/check-maps.mjs` and `git diff --check` pass. - The bridge uses real local HTTP. The app gateway uses synthetic descriptors. No deployed build or live model improvement is claimed. - Full local verification is incomplete. Recursive typecheck and server typecheck require `cargo`, which this workspace lacks. A direct server TypeScript check was killed by the workspace. The full test suite had no final verdict before the resource stop. CI must supply the remaining verification. ## Risks Low risk. This change describes existing limits and adds default metadata. It does not change search execution, page caps, gateway access, revocation, approvals, or audit. Models can still send invalid calls. This change has not measured their future retry rate. ## Model Used OpenAI Codex, GPT-6 family, with tool use and code execution. The exact provider model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, its capabilities, and unavailable model metadata - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the bug report fields - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] The public branch name describes the change and has no internal task ID - [x] I have run the focused tests locally and they pass - [x] I have added tests for the reported failure - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open recommendations or follow-ups - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 13:55:48 -05:00
Devin FoleyandPaperclip	75754ffb7a	fix: offer setup instead of Run for unfinished agents (#17976 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent can have a saved record before its setup is complete. > - This record stays paused with the setup reason and cannot run. > - Its shared action controls still offer Run beside Finish setup. > - Run then produces a generic refusal instead of taking the user to setup. > - This change makes Finish setup replace Run for that state. > - The user returns to the saved agent and can run it after setup completes. ## Linked Issues or Issue Description What happened? The header of a saved agent with unfinished setup offers Run. Clicking it produces "Agent is not invokable in its current state." Finish setup is a separate control that the user must discover. Expected behavior Show one Finish setup action in place of Run and Resume while the setup hold is present. Open the existing setup record with its selected runner. Restore normal controls when setup finishes. Steps to reproduce 1. Start creating an agent and save it before setup completes. 2. Open the agent from the roster to view its header. 3. Click Run and observe the generic refusal. Related: #17965 fixes the managed Runner connection-test result. This change fixes the controls for a saved setup record. No open duplicate was found in the title search. ## What Changed - Replace Run with one Finish setup action for a unfinished setup record. - Use the existing paused-state fallback when lifecycle details are absent. - Keep the saved agent ID and runner selection in the setup route. - Check unsaved changes before leaving the detail page. - Refresh pause reason and time with lifecycle state so completing setup in another tab restores Run without overwriting editable configuration. - Keep setup available when a caller hides Run or disables work actions. - Extend the agent action tests, onboarding browser journey and feature map. ## Verification - Red: three component regressions fail before the fix. The old UI keeps Run, lacks setup without lifecycle details, and skips the navigation guard. - Three more regressions fail before the preparation-state fix. Preparation, verification and pause transitions retain Finish setup; terminal states cannot reopen setup. - Green: 211 focused tests pass across agent actions, duplication, detail pages and saved setup. The final control suite passes 36 tests. - A real QueryClient, lifecycle hook and action-control regression fails before the pause-field refresh fix, then passes after it. The short-URL editable snapshot stays unchanged while the UUID lifecycle query restores Run. - UI typecheck and production build pass. Token and map checks pass. - The real Chromium onboarding journey passes on final source `67c4c59f6fbfc5bc8eca8f70e12b8dc1c3763f4f`: no Run or Resume, one Finish setup after reload, and the same saved agent reopens. The first attempt could not initialize the disposable PostgreSQL instance at the local shared-memory limit; no app assertion ran. After removing only a verified detached segment from this task’s completed browser run, the unchanged journey passes. No staging deployment is claimed. - A separate local integration branch combines this PR with the final write-recovery fixes from #17960. Its 36 action-control tests and the real onboarding browser journey pass too. This integration preview is not pushed to the PR branch. - All hosted CI gates pass on the final commit, including typecheck, tests, build and E2E. Apex is 5/5 with no unresolved findings. ## Risks - The change uses the existing setup pause reason through preparation and verification. Terminating and terminal records cannot reopen setup. Manual pauses retain Resume, and ready agents retain Run. - The API remains authoritative if state changes while the page is open. No admission, permission, account or credential rule changes. - The browser journey stops at setup configuration. It does not claim provider execution or deployment acceptance. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 11:30:24 -07:00
DottaandPaperclip	21137db805	fix: make Check now work for retained monitors on blocked tasks (#18061 ) ## Thinking Path > - Paperclip lets people manage agents and inspect their work. > - Task monitors schedule checks of work that is still pending. > - A task can retain a monitor after it becomes blocked. > - The UI still offered Check now, but the server rejected the task status. > - The UI also claimed that an automatic check would run on the next tick. > - This change admits an explicit human check through the normal execution gates and shows that automatic checks are paused. > - People can request the check without changing task status, and skipped checks no longer report success. ## Linked Issues or Issue Description What happened? A blocked task with a retained monitor showed Check now and an overdue countdown. Clicking the button failed because manual checks required an in-progress or in-review task. The endpoint also returned success when admission skipped a check. Expected behavior A human can request one check through normal admission. Automatic checks stay paused while the task is blocked. A check that cannot start reports a conflict. Steps to reproduce 1. Schedule a monitor on an agent-owned task. 2. Let the task become blocked while it retains the schedule. 3. Open the task and click Check now. Related: #14909 addresses scheduled-retry promotion. This PR addresses retained task monitors. Paperclip version or commit Base: `7f10cc0585`. ## What Changed - Allow a human to check a retained monitor on a blocked task. Keep agent and automatic checks limited to active task statuses. - Keep the existing admission path and status guard. Do not change task status to start a check. - Return 409 when admission skips the check. Preserve the schedule when a dependency or recovery hold prevents dispatch. Keep a successful result when a new schedule replaces the consumed schedule after admission. - Show Automatic checks paused in the banner, composer strip and properties. Hide Check now on backlog and todo tasks. Provider-quota recovery stays paused on blocked tasks because it uses a direct retry path; its monitor is retained until the task resumes. - Add database, process, route and UI regression coverage, a Storybook state, API metadata, and behavior documentation. ## Verification - `pnpm -r typecheck` passed. - `pnpm build` passed. - Server scheduling, process admission and route suites: 94 tests passed. - UI monitor, properties and time-formatting suites: 142 tests passed. - Additional admission checks passed: an explicit check starts a process on a blocked task; an unresolved dependency prevents dispatch and preserves the monitor. - Token gates, map checks, Storybook catalog checks and 30 map/catalog tests passed. - The full local `pnpm test:run` attempt reported 1 timeout among 1,621 chat integration tests: “keeps an ambiguous github text part blocked across restart and retries only that part and its tail” exceeded 15 seconds. The exact case passed in isolation on the final head (1 passed, 1,620 skipped). The remaining full-run groups were stopped to isolate that failure. This does not establish a full local pass or explain the initial timeout. Final-head CI is green, including the full chat lanes and the successful single server-shard retry. - Browser inspection passed for the blocked-task banner, composer strip and properties row. Clicking Check now set both visible buttons to disabled Checking state. The preview uses isolated fixtures. - CI server shard 7/12 failed one decision-expiry deadline assertion. The entire unchanged decision-expiry suite passed locally on the final head (20/20). The single retry of that shard passed after the first workflow attempt completed. The initial failure remains recorded here. Neither that test nor its production path changed before retry; the cause is not established by a passing retry. - Greptile reviewed final head `ed936f7433` at 5/5. Both prior review threads are resolved. - The first CI chat job stopped after its runner received a shutdown signal. The final head completed a fresh CI run. - The process fixture uses a local no-op adapter. It proves admission and execution, not a live provider's response. The Storybook state simulates dispatch. ## Risks - An explicit human check can start work on a blocked task with no unresolved execution gate. This is intentional. Dependencies, recovery, native finalization, ownership, pause and budgets retain their existing checks. - Skipped manual checks now return 409 instead of 200. The UI already displays API errors in context. - No schema change or migration. ## Model Used OpenAI Codex, based on GPT-6. The session does not expose a more specific served model ID or context-window size. Used repository tools, code execution, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 13:01:11 -05:00
Devin Foley devinfoley Claude Opus 5.5 Paperclip Amy Dotta	d30c8f42b8	feat(ui): keep writes durable through temporary disconnects (#17960 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/123/changes/paperclip/123 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People save tasks, write documents, make decisions and send board messages. > - A disconnect can hide a response after the server accepts a write. > - Recovery must keep the original account, task and request key. > - The merged connection foundation in #15754 handles reads and shared outage state. > - This change adds bounded write recovery and explicit actions when recovery stops. ## Linked Issues or Issue Description What happened? Temporary failures could lose edits, report accepted writes as failures, or require people to copy and send input again. A retry could target a newly selected task or account. Restoring attachments could create a second task. Expected behavior Keep the draft, its owner and its target. Retry only operations with a server contract that prevents duplicate writes. Verify uncertain results. Show conflicts and permanent failures, with an explicit resend or cancel action. Steps to reproduce 1. Save a task title during an outage, then open another task before recovery. 2. Restore a task creation that needs files, selecting those files in another order. 3. Lose a document or decision response, then interrupt the verification read. 4. Switch accounts while a create, upload or connected-channel publication is pending. 5. Type and erase a new-task draft before the first account lookup completes, then reopen it. This PR carries the write layer from #15769 and its review fixes. It targets master after #15754 merged. Related test cleanup #16952 already covers the unchanged copy-link toast test; this PR does not duplicate that work. ## What Changed - Add bounded autosave and submit jobs for supported writes, with pending, conflict and manual retry states. Retain admitted autosaves after navigation and preserve newer task revisions. - Keep each job with its original account, task and operation. First account resolution adopts admitted startup work; a later account change cancels old work. Remount inline and document editors when their owner or document subject changes. - Preserve task-creation keys and immutable request snapshots through file reattachment. Restore generated document keys before comparing reselected files. Reject changed requests instead of creating a second task. - Check the original account scope before creates and each upload. Save successful upload receipts before stopping on an account change. Identify attachments by content digest and metadata. - Verify uncertain document and decision results. Replay only plain, keyed comments automatically; combined reassignment, reopen and run-setting changes require receipt reconciliation or cancel. Settle each request once when a receipt races its retry. - Scope drafts and connected-channel publication reads to their owner. Keep unowned older records unread and unreplayed, with an explicit confirmed discard action after Activity inspection. Preserve typing through the first session lookup, keep erased text erased, and report browser persistence only when storage succeeds. - Wait for the first account lookup before sending startup status changes or decisions. Retry failed required lookups with a fixed limit and surface exhaustion to the original action. Preserve uncertain comment keys on account cancellation. Check the account generation before a comment retry calls its sender and again immediately before POST after asynchronous preparation. Retire only that request’s optimistic echo on completion. - Return an ordinary comment’s accepted same-company, same-task, same-user receipt before workspace reopening, task status changes or wakeups. Reused keys with changed text still conflict. Complete reference and activity records through the same finalizer for initial sends and accepted retries. It locks the task before its comment, matching task and company deletion, comment removal and tombstoning, rejects removed or tombstoned comments, repairs references and inserts a missing activity record in one transaction, and publishes after commit. - Remove reader-only record and comment-body size caps so accepted browser writes remain readable, including JSON-escaped comment requests in local and session storage. Update four existing Storybook fixtures to use owned draft keys. - Document the write contracts and add regression and real browser recovery coverage. - Add the exact reviewed-save recovery regression for both composer implementations and reloads. Align receipt completion, both comment deletion methods and company deletion with task deletion by locking tasks first. Cover real task and company deletion races. ## Verification - The exact reviewed-unknown-save regression fails in all four cases on original commit `1ed0af4186`. It passes with this PR for both composers, before and after reload. Cancel keeps the text and uploaded receipt. The retained draft sends, and the next message also sends. Run `pnpm exec vitest run --project @paperclipai/ui ui/src/components/composer-recovery.test.tsx`. - The new real PostgreSQL task-deletion race fails with SQLSTATE `40P01` on parent commit `f3253a00c4`. The company-deletion race also fails with `40P01` on `e8cf0e02dd`. All six receipt tests pass after the final lock-order fixes. They cover company deletion, task deletion, comment removal, tombstoning, concurrent cross-task references and interrupted reference repair. Run `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/issue-comment-receipt-postgres.test.ts`. - The full UI run passed 9,656 of 9,659 tests across 816 files. Three cases failed under that run: two large-tree timeouts and one plugin-panel assertion. All 67 tests in those two files passed in an isolated rerun. These failures remain recorded; this does not claim a clean full UI run. - `pnpm -r typecheck`, `pnpm build`, token gates and map checks pass on the final change. The local full repository test command was started, then stopped when the delivery moved to this existing PR. Fresh PR CI must provide the full-suite gate. - A local Storybook walkthrough used the real rich editor. Cancel kept the restored text, Send cleared it after the simulated save, and a new draft enabled Send again. Storybook simulates submission; it is not staging acceptance. - Earlier verification on `f3253a00c4` recorded five browser journeys for title saves, cached-task offline recovery, startup recovery, accepted comment replay and session lookup failure. Those results remain prior-head evidence. No new staging deployment is claimed. - On `e8cf0e02dd`, all CI checks passed after one browser-shard rerun. Its first failure copied page text instead of YAML in an unrelated agent-configuration test. On final commit `35131d4380`, Greptile is 5/5 with no unresolved review threads. The first CI attempt has two failures outside this diff: runner close-attachment suspension and scheduled-retry teardown. The scheduled-retry file passes all 14 tests in isolation; the isolated runner test still fails locally. The single standard CI rerun passed both jobs, and the aggregate verification gate passed. All CI checks are green on `35131d4380`; Apex is 5/5 on that same commit with no unresolved findings. ## Risks - Automatic replay requires server duplicate prevention or result verification. Uncertain combined mutations require receipt observation or cancel and a fresh intentional send. - Old unowned form drafts remain stored but are not assigned to an account. Unowned channel records can be retired after Activity inspection; this never cancels an accepted remote update. - Browser storage and hashing can be unavailable. Storage failure is reported honestly. Without a digest, replay can duplicate an upload rather than silently drop a selected file. - Detached autosaves remain bounded and check account ownership on each attempt. Conflicts or exhaustion retain the draft for explicit recovery. - No schema, credential or agent permission change is included. This PR does not claim staging deployment or original-task acceptance. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) produced the original implementation with tool use. OpenAI Codex, GPT-6 family, assisted with restoration, integration, verification and review fixes. This Codex session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Amy (Opus) <claude@users.noreply.github.com> Co-authored-by: Dotta <bippadotta@protonmail.com>	2026-10-28 10:52:06 -07:00
Nicky LeachandPaperclip	a2fbce9305	Fix plugin session and directory reuse on follow-up turns (#18045 ) ## Thinking Path > - Paperclip preserves task state when agents receive follow-up work. > - Plugin computers keep their process state outside the controller. > - A local receipt proves that a previous plugin run suspended. > - A cold resume left that receipt in place after the session became live. > - The next warm follow-up rejected the stale receipt as an identity mismatch. > - A separate follow-up path also treated a fresh task directory as a Git worktree. > - This PR consumes the obsolete receipt after admission and preserves the bound directory on follow-up. ## Linked Issues or Issue Description Related: #17820. This change retains the controller restoration added by #18018. What happened? A plugin session resumed from a suspended checkpoint and completed a turn. Its next warm follow-up failed with `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate`. The journal had the completed run's identity. The local suspension receipt still had the preceding run's identity. Two separate task histories showed this sequence. After a fresh retry completed in a repository-free plugin directory, an ordinary follow-up could instead fail with `The selected plugin environment does not support Core Git worktree preparation.` The saved task settings contained only `mode: isolated_workspace`, which introduced the default Git strategy on the next admission. Expected behavior Keep the same admitted provider session during warm follow-ups. Reuse the saved plugin task directory without Core Git preparation. Retain ownership checks and explicit strategy settings. Steps to reproduce 1. Complete a warm plugin task and let its session suspend. 2. Resume the checkpoint and complete another turn. 3. Send a follow-up while that session remains warm. 4. Separately, complete a fresh retry in a repository-free plugin directory and send an ordinary follow-up. Paperclip version or commit Both regressions reproduce against base `820c45846` with the added tests. Deployment mode Hosted plugin execution. Local reproduction uses the real filesystem and a disposable PostgreSQL 18 database, with synthetic provider execution. ## What Changed - Remove the local plugin suspension receipt after provider recovery and durable run admission publish a live session. Admission still checks the receipt first. Normal suspension writes a new receipt. - Recognize a saved fresh plugin directory through its company, task, workspace, owner, attempt and path. Use the existing directory-selection path. Explicit strategy settings still apply. - Add cold-to-warm, database-backed workspace-selection and ownership regressions. Update execution semantics and session verification recipes. ## Verification - Red: the cold-to-warm regression failed on unchanged executor code with `prior_owner_terminal_state_indeterminate` on the follow-up. - Red: the PostgreSQL regression selected a Git strategy on unchanged environment-selection code. The explicit-strategy denial case already passed. - Green: all 95 tests in `workspace-preparation.test.ts` and `provider-fresh-workspace.test.ts` pass against a disposable local PostgreSQL 18 database, selected with the existing `PAPERCLIP_WORKSPACE_PREPARATION_TEST_DATABASE_URL` test seam. - Green: 92 focused executor cases pass, including plugin checkpoint recovery, warm rotation and denial cases. The full `plugin-runner-task.test.ts` suite also passed (38 tests). - Green: direct server check, `node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit`. - Green: `node scripts/check-maps.mjs`, `git diff --check`, and a local diff scan for secrets, personal data and live identifiers. - Full local native suites: 988 of 1,004 cases passed. Sixteen integration cases could not start their bundled PostgreSQL cluster. The failure persisted outside the sandbox. These cases are not claimed as passing. An earlier sandboxed attempt also denied process and loopback operations; those denials disappeared when run outside the sandbox. - The first broad local commands could not locate Cargo on `PATH`. Rust is installed under `~/.rustup`; its Cargo executable was verified. Per the requested scope, subsequent local verification stays targeted to changed TypeScript paths; no Rust source changed. CI Typecheck + Release Registry and Rust verification passed on the initial commit. The complete repository test suite was not run locally. - Review regression: an explicit agent-level Git strategy reproduced the incorrect directory fallback before correction. After including the raw agent strategy in the existing check, all three targeted cases (implicit, task-level explicit, agent-level explicit) pass. The server TypeScript check passes after this correction. - CI on `a81668c4b`: the first workspace-a 2/2 attempt failed in unchanged UI `AgentActionButtons.test.tsx` (`calls clearError and refreshes agent-related queries`, zero mock calls). [Original failed job](https://github.com/paperclipai/paperclip/actions/runs/40717870938/job/122921463517). That shard passed on `294c5dfc3`. The failure is retained here. The UI shard passed on final commit `687000287`; no unrelated UI edits or local UI suite runs were added. - CI also exposed three existing local-directory recovery failures on `a81668c4b`: the review correction applied the raw agent strategy guard to local directories. [Original failed server job](https://github.com/paperclipai/paperclip/actions/runs/40717870938/job/122921463229). Narrowed that guard to adapter-managed directories, preserving local recovery behavior. All 20 targeted cases (three plugin selection cases and 17 local recovery scenarios) pass afterward, and server TypeScript verification passes. - Final commit `687000287d3339b7e15e35d95515b3c75c273d7a`: [hosted CI is green](https://github.com/paperclipai/paperclip/actions/runs/40720169342), including the corrected server shard, all other server/workspace shards, Runner checks, typechecking and build. All 55 non-skipped PR checks pass. Greptile scores this exact commit 5/5, with no unresolved review threads. GitHub reports no merge conflicts. - Pre-merge hosted acceptance was requested. The maintained deployment CLI stopped during artifact discovery: no published private Cloud composition exists for `687000287`. The current private build requires canonical Core master readiness and standard-image provenance. The Core-only branch preview omits the private computer plugin needed for the journey. No campaign or tenant update was started; private-preview deployment support is a separate prerequisite. No live deployment or provider acceptance run is claimed. ## Risks The receipt is removed only after the existing runtime publishes an admitted session. Earlier admission failures keep their proof. A later normal suspension creates a new receipt. The change preserves existing identity, lifecycle and controller checks. Directory selection remains limited to the exact bound, active, repository-free plugin recovery directory. Foreign owners, changed paths, wrong attempts and explicit Git strategies do not gain this fallback. No schema, API, credential, permission or fleet-default change is required. Existing failed runs and recovery holds remain history. This PR does not edit quarantined state or replay an interrupted task. Rollback restores the previous code and can reintroduce the two admission failures. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. A more specific model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 10:26:12 -07:00
DottaandPaperclip	b14e1ecaf8	feat: add a complete agent YAML configuration editor (#18036 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/175/changes/paperclip/175 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators configure each agent through several settings tabs. > - Operators also need one complete, editable configuration document. > - The document must retain each resource service's access and validation rules. > - This pull request defines the `paperclip/v1` Agent YAML format and adds an Advanced editor. > - The editor previews changes, rejects stale core writes, and explains partial saves. > - Operators can edit agent settings in one place and verify the saved state after reload. ## Linked Issues or Issue Description Refs #2219. Related settings work: #10974. Problem or motivation Agent configuration spans profile, runtime, instructions, skills, tools, channels, permissions, secrets, keys, and revision history. Operators cannot inspect and edit those settings as one versioned YAML document. Proposed solution Add an Advanced agent tab with a YAML editor. Define a shared document schema. Read and write resources through their existing APIs. Separate editable settings, explicit commands, and read-only observed state. Alternatives considered A raw agent-row editor would omit related resources. A generic database update would bypass resource ownership. The editor therefore uses the existing resource contracts and records partial results when a later request fails. Roadmap alignment This extends the shipped agent configuration capability. It relates to easy AGENTS.md configuration, multi-harness teams, governed tool access, and secrets management. Stored credentials remain redacted. This format is an editable settings view; it does not add company portability or provider consent. ## What Changed - Add the `paperclip/v1` Agent document schema and format documentation. - Add the Advanced route, sidebar entry, YAML editor, validation, change summary, save progress, and reload recovery. - Cover core settings, instruction files, skill pins, tool installs/profiles/policies, channels, permissions, secret operations, API keys, and revision restore. - Add private-keyed configuration revisions checked under the agent write lock. Read the current key through its owner so file-based rotation does not require restarting the server. Preserve unchanged secret markers at their original paths. - Retain drafts and returned credentials after partial saves. Require reload before retrying commands. Prevent duplicate submissions. Keep new key tokens visible through rename and disable saves with an empty change plan. - Send only changed channel settings. Require explicit intent for shared profile edits. Load destination instruction files when changing their location. - Update Storybook fixtures, verification maps, and regression tests. Add browser acceptance through real APIs and PostgreSQL. ## Verification - `pnpm -r typecheck` passes. - `pnpm build` passes. - The focused UI suite passes all 50 tests. The API/revision helper suites pass all 93 tests. - All 65 lifecycle tests pass with a dedicated PostgreSQL database. They exercise real row locks, redaction, revision history, audit, and company access. - All six browser acceptance cases pass. They check saved settings and clears, secret redaction, key creation/revocation, stale drafts, lost-response recovery, instruction relocation, revision restore, and key-token retention through rename. The tests use real resource APIs and storage. The MCP target uses the existing local provider fixture. No model run or provider login is claimed. - Signing-key regressions cover both environment changes and real in-place key-file rotation. The protected database save/fencing cases pass after removing the cache. A local measurement of 1,000 file-backed revision calculations took 48 ms. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and Storybook catalog checks pass. - The full [PR CI run](https://github.com/paperclipai/paperclip/actions/runs/40725425470) passes on `34ed97bbf9`: all 48 jobs, including all eight browser shards. Greptile is 5/5 on this commit, with no unresolved review threads. This final commit refreshes CI after a runner-routing fallback; its source tree matches `fb4cf488cf`. - Paperclip Review and Superagent Security Scan ended neutral because their external agents did not complete a valid assessment/receipt. Superagent passed on `fb4cf488cf`, which has the identical source tree. One request to retry the final scan returned HTTP 404. Neither check reports a code finding. - The local full-suite attempt was blocked by host System V semaphore exhaustion. Acceptance and lifecycle verification used an isolated PostgreSQL 18 container. Earlier CI runners were interrupted. A local combined run hit hook/test timeouts; the full 88-test route file passed after the builds finished. The final full CI run passes. Earlier failure logs and browser traces remain retained outside the repository. To run the browser regression: ```sh pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/agent-configuration.spec.ts ``` ## Risks - Multi-resource saves are sequential. Earlier operations can remain saved if a later operation fails. The editor reports the completed steps and requires reload. - Core settings and managed instruction files have atomic revision/hash checks. Other unversioned resource writes and external supporting files retain a race window after preflight. - Shared profile definitions affect other bindings. The editor requires `updateSharedDefinition: true` for those edits. - The editor follows native service normalization and experimental settings. Canonical reload can add generated access profiles or mask environment values. - New editor dependencies require the repository's normal automated lockfile update. This pull request does not edit the lockfile or add a database migration. ## Model Used OpenAI Codex, GPT-6 family. The exact current model version was not exposed. The work used reasoning, repository tools, code execution, and browser automation. Earlier work in this chat used the configured Codex model before a model switch; its exact model ID and context window were not exposed. The current context window size is also not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 11:49:24 -05:00
DottaandPaperclip	567c043b17	feat(server): expose bounded hot-upgrade readiness (#18037 ) ## Thinking Path > - Paperclip coordinates agent work and keeps its authority in the server. > - Remote native runners can preserve execution through a server replacement. > - Deployment operators need evidence that current runs support that recovery. > - Process-local drain counters do not establish this fact. > - This PR adds an optional, operator-only readiness observation from existing records. > - A deployment consumer can use that report while the existing runtime retains recovery authority. ## Linked Issues or Issue Description Refs #17439. What happened? A deployment cannot distinguish recoverable remote native executions from runs that require a task drain. The existing status endpoint reports only process-local counts. Expected behavior An instance operator can request a bounded readiness report without stopping admission or exposing checkpoint material. Missing ownership or persistence evidence must prevent readiness. Ordinary status reads must keep their current cost. Steps to reproduce Run a remote native task. Read the task-drain status. It reports active work but supplies no recovery eligibility. Deployment code therefore has no affirmative signal to select an active-run handover. ## What Changed - Add `GET /api/instance/task-drain?upgradeReadiness=1` for instance administrators and existing Cloud control callers. - Return aggregate eligibility from current run identities, controller leases and bound opaque checkpoint headers. Unknown work, pending admission and incomplete reads prevent a positive report. - Bound the explicit read to 1,001 rows, encrypted headers of at most 256 KiB each and 8 MiB total, and a two-second SQL statement timeout in a read-only transaction. Keep ordinary status reads in memory. - Add PostgreSQL, count-transition and route authorization tests. Update the API contract, operator docs and access maps. ## Verification - All seven readiness cases pass against an isolated PostgreSQL 18 fixture. They cover missing checkpoints, legacy adapters, missing process/session identities, foreign owners, expired leases, missing lease tokens, failure state and in-flight work. The positive SQL fixture uses the real Runner checkpoint writer and encryption. Regular sandbox workspace identity and plugin environment-lease identity are tested with independent IDs. Stopped/released/undispatched plugin tasks and every mismatched identity field are rejected. Local/legacy/malformed/oversized or mismatched headers are rejected. A held PostgreSQL table lock proves the two-second cancellation budget, rollback/connection release and unrelated query responsiveness. - All 123 focused readiness, instance-settings and Cloud control assertion tests pass, including operator-only access and no readiness read on ordinary status requests. Runner’s 168 checkpoint tests pass. Runner and server typechecks pass after review fixes. - Workspace typecheck, full build, module-boundary and map checks passed locally before the master rebase. Current-head GitHub CI is now green, including typecheck, build, all grouped/serialized server and Runner tests, browser shards, policy and review-dispatch checks. Greptile completed cleanly at 5/5 for `f15fb0c7e9dfb975090cd5d791a1c8e6701275f8`; all actionable threads are resolved. - The broad local test run was stopped after the macOS embedded PostgreSQL fixture failed to bootstrap. The SQL test was then run on the isolated PostgreSQL fixture. No assertion or timeout was weakened. One unrelated existing route test exceeded its 15-second limit under concurrent local builds; it passed on one isolated retry and the subsequent whole route suite passed. - Hosted active-run upgrade acceptance has not run. This PR supplies an observation; it does not change rollout policy or establish live continuity by itself. ## Risks This is an instance-wide observation, so company members and agent keys must not receive it. A readiness snapshot can age as work changes; shutdown and startup retain their existing execution authority and fencing. The report does not validate a replacement artifact or authorize a schema migration. It decrypts only bounded headers through the checkpoint owner and delegates format/identity inspection to Runner; full journal restoration and physical sandbox authentication remain with recovery. No migration, recurring query, new checkpoint write or wire protocol change is included. The additive Runner header inspection contract reuses its existing parser. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, source inspection, tool use and code execution. The host does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the affected contracts; the broad-run limitation is recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 11:28:15 -05:00
DottaandPaperclip	820c458465	fix(runner): restore per-turn controller checkpoints before resume (#18018 ) ## Thinking Path > - Paperclip coordinates agents and preserves their work between turns. > - Remote native sessions save controller checkpoints in the database. > - External persistence leaves a small identity marker on local disk. > - A later `per_turn` continuation must restore that checkpoint before it reads the controller identity. > - The restore path handled warm sessions only, so valid per-turn continuations failed before provider execution. > - This PR restores the referenced checkpoint and retains the existing ownership and lifecycle checks. ## Linked Issues or Issue Description Related: #17439 added external controller persistence and the warm restore path. #18011 repairs warm preparation and fresh bootstrap checks; it does not cover ordinary per-turn continuation. What happened? A remote OpenCode run completed with `per_turn` lifecycle and saved its controller checkpoint in the database. Its next turn reused the normalized session. Admission read the remaining external-persistence marker as a full controller file and failed with `runner_state_identity_mismatch: durable_identity_unreadable`. The prior run's complete checkpoint and harness backup still existed. The failure occurred before provider work started. Expected behavior Restore the exact prior controller checkpoint before identity inspection. Then apply the existing company, task, session, prior-owner and suspended-state checks before a provider can start. Steps to reproduce 1. Complete a remote native OpenCode task with `per_turn` lifecycle and external controller persistence. 2. Send a follow-up that resumes the same saved session without a session reset. 3. Observe admission reject the valid external controller marker before provider execution. Paperclip version or commit Observed at `40c1ab36cdac70c85dc8fe5e912431834fdb691b`. The same warm-only restore remains on current master. Deployment mode Hosted Linux with Daytona and native OpenCode. The original model was `openrouter/deepseek/deepseek-v4.1-flash-1017`. ## What Changed - Restore external controller state for ordinary remote per-turn continuations through the existing checkpoint loader. - Bind that read to the current company and task, the marker's run, and the expected normalized session. - Keep local, warm and restart-recovery paths, prior-owner checks and harness backup checks intact. - Refresh a prior-run projection from the latest checkpoint after a rejected active-owner attempt. Keep current-run full state and the existing legacy file-backed fallback intact. - Add restoration and denial regressions, execution documentation and session verification recipes. ## Verification - Red: the real controller generated an external marker; a valid per-turn OpenCode continuation failed with the exact `durable_identity_unreadable` error before tool-authority construction. The repair passed the same case and its ownership guards. - Review regression red/green: a real controller advanced its checkpoint after an active-owner rejection. Before the follow-up fix, a later valid continuation retained a null attach seed. After the fix, it restored the newer attach template and prepare command. Legacy full-file roots with no database checkpoint retain their existing behavior. - At current head `1f6589233e6121d082a0c0c4444ea895b6e8cd7c`, 86 focused executor cases and all 14 execution-driver cases pass. Coverage includes checkpoint restoration, direct/backup suspension, Codex, legacy files, warm admission and canonical bootstrap, missing or malformed evidence, wrong bindings, active/foreign owners and unsuspended runners. Tests stop before provider launch. - The real PostgreSQL encrypted-checkpoint integration passes, including normalized-session refusal before write and stale-writer fencing. Independent source review found no actionable safety defect. - `node scripts/check-maps.mjs`, `git diff --check`, full `pnpm -r typecheck` and `pnpm build` pass on current head. - The local full suite did not complete: native PostgreSQL startup failed under the host sandbox, an initial Docker rerun lacked migrations, and the migrated retry crossed the review-fix revision change. Logs are retained; that mixed-revision run was stopped and its owned database removed. Full test evidence comes from current-head CI. - Current-head CI passes: 53 successful checks, two intended Storybook skips, none pending or failing. Greptile is 5/5 on that exact head with no open review threads. The only manual merge conflict with #18011 was in the verification recipes; both recipes and the unchanged per-turn restore helper are preserved. Live continuation acceptance remains unverified, so this PR stays draft. Two disposable OpenCode/Daytona attempts never reached turns two and three: - At `72b49208b1`, the first turn executed tools and produced the exact 33-byte workspace note, but exceeded its four-minute settlement deadline. The first completion lacked a registered attachment; the second decorated a typed attachment reference with filename/size/hash text. The strict UUID parser rejected that reference before checking a valid criterion reference. The scoped attachment, asset and publication receipt exist. Replaying the actual guard rejects the original input before a database lookup; correcting only that reference passes against the recovered row and receipt. This did not show a publication-persistence or controller defect. The task remained unsettled. - At `1f6589233e`, a separately approved replacement used a temporary prompt-only fixture change requiring exact `deliverable:<attachmentId>` references, with metadata in `verification.detail`. Production code, grading and deadlines were unchanged; the fixture patch was removed afterward. This attempt had no recorded provider authentication, harness-ready, model, tool or result event. Ingress acquisition took 84.072 seconds; transport connection took 205.835 seconds within the 239.832-second failed native execution. A local settlement GET also timed out after 30 seconds. The underlying startup stall is unresolved; these observations do not establish a remote provider outage. Final recovery was blocked with `ambiguous_remote_recovery_evidence`; no successful settlement or continuation is claimed. Both attempts used a Darwin arm64 source host with Node 26.4.0, isolated PostgreSQL 18, a Linux x64 runner and OpenCode 1.18.34. The representative qualified model was `openrouter/deepseek/deepseek-v4-flash-0731`; the original incident used `openrouter/deepseek/deepseek-v4.1-flash-1017`. The controller defect occurs before provider work. Runtime artifacts came from immutable build `e7ea2e2961`, whose Runner package source is identical to the candidate. The current-source manifest, artifact, distribution, asset and all 110 archive-hash checks passed. Each attempt had $10 company/agent model budgets and zero automatic workflow/task retries. Accounting was incomplete; zero recorded spend is not proof of zero cost. Earlier setup failures are also retained: host PostgreSQL bootstrap denial, cached-pack compatibility admission before provider work, and an [official artifact build](https://github.com/paperclipai/paperclip/actions/runs/40696838122) that stopped during an unrelated Pi npm install at its 300-second bound. No manifest or integrity guard was weakened. Owned local processes and databases are stopped, and the temporary fixture patch is removed. Standard fixture cleanup had left one replacement sandbox started; it was stopped and deleted through the SDK. Every owned sandbox was then verified absent with a final 404 readback. Resource cleanup does not establish successful task finalization. An earlier head retained one [voice-recovery timing failure](https://github.com/paperclipai/paperclip/actions/runs/40695892339/job/122843005098): a deadline test observed two failed-provider inspections instead of one. A single isolated shard rerun passed all 1,579 tests without a source change. Its timing explanation remains an inference. The current reconciled head passed without a retry. ## Risks Restoration writes a local projection before the existing admission checks. The encrypted checkpoint loader validates company/task/run and normalized session before writing. A later ordinary continuation refreshes a prior-run projection so rejected admission cannot freeze an active owner's older checkpoint. Active, foreign or unsuspended prior owners still cannot start a successor. Missing or malformed evidence remains blocked. The change adds no retry, credential, provider binary or lifecycle-policy change; it preserves the existing legacy fallback. This PR does not deploy the repair or recover the original hosted tasks. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. The session exposes GPT-6 as its model version; a more specific model ID and context-window size are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 09:49:32 -05:00
DottaandPaperclip	25f1070fbd	fix(runner): preserve checkpoint proof across warm preparation (#18011 ) ## Thinking Path > - Paperclip coordinates agent work and preserves its execution state. > - Native Runner checkpoints preserve authority across controller changes. > - Warm sessions also need checkpoint writes while their successor prepares. > - External persistence replaces the local state file with a marker. > - Two existing checks treated those valid states as lost ownership or prior activity. > - This PR uses the existing preparation lease and canonical controller state to keep those checks accurate. > - The result is safe warm-session retirement and fresh-session startup on reused sandboxes. ## Linked Issues or Issue Description Related: #17439 and #17320. The open #17401 changes the recovery welcome window and does not cover these paths. What happened? A configuration change could fail before the next provider turn with `runner did not durably suspend before checkpoint`. The task already pointed to its preparing successor, but that successor did not yet have a native coordinator. The old completed runner's maintenance checkpoint was rejected. A separate fresh session on a reused sandbox could fail with `resumed_unavailable`. Checkpoint registration had replaced the full local state file with an external-persistence marker. The bootstrap check still required the old full-file format. Process supervision then reported unconfirmed termination before launch. Expected behavior Retire the completed warm runner under the same process's live preparation lease. Admit a fresh remote session only when the registered controller proves exact identity, no prior activity and pending bootstrap commands, and the remote session directory can be claimed atomically. Steps to reproduce 1. Complete a native warm turn with external checkpoint persistence. 2. Change configuration so instruction preparation must retire that session, then submit a follow-up. 3. Separately, admit a fresh native session on a reused sandbox lease after external checkpoint registration. Paperclip version or commit Reproduced in the checkpoint implementation introduced by #17439. Fixed against current master. Deployment mode Source checkout tests and a hosted Docker/Daytona incident. No deployment or original-task retry is part of this PR. ## What Changed - Accept completed-run maintenance under the exact successor's live, same-process preparation lease only before a native coordinator exists. - Preserve original controller generation, accepted-result, no-new-work, company and task checks. - Pass the registered controller through the existing connection callback and read its current canonical state for untouched-session proof. - Keep marker-only, identity, activity, backup and atomic remote-directory guards. - Add regressions and update execution semantics and session verification recipes. ## Verification - Warm retirement red: the real Postgres writer rejected the preparing successor's `runner.suspend` checkpoint with `native_runner_checkpoint_owner_lost`. - Warm retirement green: 14 Postgres ownership cases pass, including stale/foreign/expired owners, conflicting native authority and new provider work. - Bootstrap red: real controller registration and external persistence caused the resumed and replacement positives to fail. Six external negative cases passed. - Bootstrap green: all 34 bootstrap cases, the bootstrap-state predicate and 13 connection-driver cases pass (48 total). - `node scripts/check-maps.mjs` and `git diff --check` pass. - Full `pnpm -r typecheck` and `pnpm build` pass. - Full local `pnpm test:run` remains partial/interrupted. The default attempt could not initialize embedded Postgres on this Mac. A dedicated Postgres 18 attempt ran for 17 minutes without a completed file result before interruption; the generic helper separately confirmed unsupported embedded Postgres (`initdb` failed after five attempts) and ignores the external database URL. The generic probe skipped one test. Setup and interrupted-run logs are retained; neither is claimed as a behavioral red or a full local pass. - Real source-hosted Codex/Daytona acceptance was attempted at `6ce7fcacc4` with `gpt-6.1-sol`, `xhigh`. The actual task stopped before provider work at `runner_ingress_auth_failed`, with zero authenticated connections and zero source events. A separate environment-qualification handshake reached challenge/response; that does not prove task authentication. The three-turn warm reuse, configuration retirement, useful output and reload workflow remains unverified, so this PR stays draft. - The live attempt used the cached Linux amd64 Runner binary `7437a4e32a099180889be534fcfb2a086046514490c1448aba46c55848983ed8` from the immutable Daytona image. The current staging binary could not be copied because SSH timed out, and the configured current EXE image returned registry 403. No compatible current local or CI-downloadable artifact was available. Earlier setup failures and cleanup receipts are retained; no provider success is claimed. Disposable test resources were cleaned up. - Exact-head Greptile completed at 5/5 with no open feedback: https://github.com/paperclipai/paperclip/pull/18011#issuecomment-6469515952. All current-head CI checks pass: 53 successful checks and two intended Storybook skips. ## Risks The preparation fallback affects maintenance checkpoints only after an accepted result. An existing native coordinator prevents that fallback. Stale controller generations, expired or foreign preparation leases, wrong tasks and new provider work remain denied. The bootstrap check retains the live controller object and reads current state each time. It does not trust a marker or restore local state beneath a live controller. Pending turns, prior events/connections, backups and existing remote directories retain recovery gates. No schema, provider binary, credential, permission or lifecycle-policy change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution and collaborating agents. This session does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 09:09:09 -05:00
DottaandPaperclip	83709f75eb	fix(ui): hide task row retry status labels (#18017 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task lists and the inbox help people scan their work. > - Shared task rows show system interruption messages beside task titles. > - These messages add detail that the person cannot act on from the row. > - This pull request removes the three system interruption labels from those rows. > - People can still open the task to inspect progress and use Retry. ## Linked Issues or Issue Description Refs #16965, which introduced these labels. Related recovery report: #17386. This change only removes row text and does not resolve that report's retry behavior. What existing behavior does this improve? The shared task rows in task lists and the inbox. Current behavior Rows show “Retry available,” “Thinking,” or “Trying again soon” beside the title for unresolved system interruptions. Proposed behavior Omit all three labels from shared task rows. Keep the existing system-interruption guard so the row does not show a legacy recovery chip in their place. Keep task status, other badges, and retry controls inside the task. Reason and benefit People can scan task titles with less incidental execution text. Breaking changes The three row labels disappear. APIs, recovery behavior, and task conversation controls stay the same. ## What Changed - Remove the system interruption text from every shared task-row presentation. - Cover working, scheduled-retry, and failed interruptions in both row presentations, including a legacy recovery action that must stay hidden. - Record the expected row behavior and verification states in the task feature map. ## Verification - Passed: token gates, feature-map contract and reference checks, and Git diff whitespace checks. - Passed: 83 component tests in `IssueRow.test.tsx` and `Inbox.test.tsx`, including the new regression cases; full `pnpm -r typecheck`; full `pnpm build`. - The full local `pnpm test:run` was attempted and stopped after 16 database-dependent failures. An independent disposable `initdb` reproduced `FATAL: could not create semaphores: No space left on device`. The host has exhausted its macOS System V semaphore capacity. All required CI test lanes passed on the final commit and provide full database-suite coverage. An earlier attempt also found an unbuilt plugin SDK; building that existing dependency repaired collection before this database failure. - Review the working, scheduled-retry, and failed interruption branches. Each returns no system label. The existing guard still suppresses legacy recovery chips. - Independent source review found no actionable code issues. Regression cases check title navigation, removal of the three labels, and suppression of replacement recovery chips. - All required CI gates passed on commit `dd1d7b08dbba8a10a95c5d24c4aea64479befe95`: typecheck, general and serialized tests, browser suites, build, canary, and Runner checks. - Greptile reviewed that commit at 5/5 with no open findings. The optional Paperclip review also returned 5/5 with no findings; its check status is neutral. ## Risks - Low risk. This removes row text only. Task status, recovery records, other badges, and task-level retry actions keep their current behavior. ## Model Used OpenAI Codex, GPT-6 family. This session does not expose an exact model ID or context-window size. The assistant used reasoning, shell tools, and subagents for source review and validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 08:22:13 -05:00
Devin Foley devinfoley Claude Opus 5.5 Paperclip Amy	39279aa34f	feat(ui): shared error, connectivity and query foundation for disconnects (#15754 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board reads API data and keeps the current task on screen. > - A restart or temporary network failure can interrupt those reads. > - Separate page error handlers can hide cached data or expose raw server codes. > - The app needs one connection state, readable error copy and a consistent read policy. > - This change provides that foundation and moves affected read surfaces onto it. > - The dependent write layer in #17960 preserves edits and submissions across the same failures. ## Linked Issues or Issue Description What happened? Temporary API failures could replace loaded content with errors. Different surfaces handled the same outage differently. Some showed raw server codes. A successful query to an unrelated service could also falsely report that Paperclip had recovered. Expected behavior Keep cached content visible. Show one connection banner during sustained trouble. Confirm recovery through the Paperclip health probe or its WebSocket. Retry transient reads and show readable errors for permanent failures. Steps to reproduce 1. Open a populated task or workspace page. 2. Interrupt API requests with a 503 or a dropped connection. 3. Keep an unrelated external request in flight. 4. Confirm that cached content remains and that the external result cannot clear the outage. 5. Restore the API and check that reads refresh. The write layer first merged into this feature branch in #15769 now has its own dependent PR, #17960. This PR contains the shared foundation and read changes. Merge this PR first. ## What Changed - Classify API errors and provide readable error copy, including non-JSON proxy responses and Retry-After handling. - Use one connection store and health probe with bounded backoff. Confirm recovery from the server health probe or live-updates WebSocket. - Pause reads while disconnected, retry transient errors, and refresh active reads after recovery. - Keep cached data visible through shared query views and affected page, panel and plugin read surfaces. - Add one app-level connection banner and support the pending-write count consumed by the dependent write layer. - Add a query-error rendering guard. Call it directly from the existing CI step. Update development documentation. ## Verification - Historical-chat lookup now follows the same ready/stale roster state as rendering. Three regressions fail before the fix; the six new cases pass after it, including working Retry, fresh data, no-cache error, permanent 403 and company isolation. All six focused query/recovery suites pass 200 tests. Final-commit CI passes and Apex is 5/5 with no unresolved findings. Canary Dry Run initially refused a preview version that was already on npm. It passed after one rerun with fresh registry metadata; no check was disabled. - Integrated current master and retained both transcript interest registration and connectivity reporting on socket open. The seven focused recovery/query suites pass 288 tests, and UI typecheck and production build pass. All 9,451 UI tests across 799 files pass on the merge commit. Final-commit CI passes and Apex is 5/5 with no unresolved findings. - Current head `70d9335d732219352ef0f98414439766bbcb33df`: all hosted CI checks pass. Apex is 5/5 on that head with no unresolved findings. The branch has no merge conflict. GitHub still requires code-owner approval. - The foundation passes 9,430 UI tests in 804 files. After integrating current master page paths, 190 focused page tests pass. - Two unrelated-success regressions fail before the latest fix. With the fix, 62 focused UI tests, all six guard tests and UI TypeScript pass. - The composed foundation and write layer pass 9,505 UI tests in 808 files, UI TypeScript, a production UI build, and query-error, token and map guards. - A real Chromium journey on the composed source injects a 503 during title save. The loaded task stays visible; one reconnect banner appears; the edit saves after recovery; another task stays unchanged; the saved title survives reload. - Full recursive local checks were not repeated because Runner compilation exhausted local disk during another repair. Hosted checks must pass before merge. No new staging deployment is claimed. - Latest review follow-up: a cached CaseDetail now survives a failed HTTP 429 refresh. Its page regression fails before the fix and passes after it. Eighty focused foundation tests, UI TypeScript, build and guards pass after that change. - Earlier CI evidence is retained: several foundation jobs received a runner shutdown signal together. The write-layer head also failed an unchanged Runner descriptor close test with `provider_transport_failed`. The same Runner source passed that suite in the foundation run. New CI runs cover the updated heads; no check or assertion was weakened. ## Risks - Shared retry defaults change the time before transient read errors appear. Permanent errors still fail promptly. - ApiError.message now contains readable copy. Callers that compare raw codes must use ApiError.code. - Cache success can come from another service or local data. It must not confirm Paperclip recovery. - Durable writes are reviewed separately in #17960. This PR does not change database or mutation contracts. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) produced the original work with tool use. OpenAI Codex, GPT-6 family, assisted with restoration, integration, tests and review fixes. The Codex session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Amy (Opus) <claude@users.noreply.github.com>	2026-10-27 21:22:07 -07:00
Devin FoleyandPaperclip	c9213d1be4	fix: send saved queued messages after Stop (#17959 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People can save follow-up messages while an agent runs. > - Stop preserves those messages and stops the active execution. > - The queue removed its delivery controls when that execution ended. > - Native admission also refused an explicit resend of older queued input. > - This change adds Send for a stopped queue and records fresh board intent. > - The saved message can continue through normal cleanup and admission checks. ## Linked Issues or Issue Description What happened? A person queued a message with a screenshot while an agent was starting. Steering was not yet available. They stopped the run. The message remained visible, but no control could send it. Native queue delivery also rejected this transition. Expected behavior A stopped queue offers Send. The click preserves message IDs, content, attachments and original authors. It starts at most one eligible continuation. Stop remains effective until the person gives new direction. Steps to reproduce 1. Start a native task and queue a second message. 2. Stop the active run before it consumes that message. 3. Open the task after cleanup. Observe the saved message with no send control. Refs #16965 for the queue-control change and #15015 for stopped conversation recovery. No open repair for this exact transition was found. ## What Changed - Show Send for a saved deferred queue with no active run. - Accept the existing board queue action for a stopped native target. Keep live native interruption restricted. - Use the server-recorded click time to authorize older native messages. Reject stale, malformed and wrong-actor intents. - Reuse normal queue dispatch, revision checks, cleanup proofs and execution admission, including a failed native predecessor after a verified Send. - Ignore a fully committed, successful recovered export when admitting a fresh message, while retaining the wait for results still committing. - Honor an applied `/new` conversation boundary while retaining native execution holds until the existing stop and cleanup checks verify them. Workspace holds also survive reset. Check the newest failed or cancelled native execution even when older recovery actions exist. - Keep native automatic delivery separate from the explicit Send action. Update execution and access documentation. ## Verification - Meaningful regressions failed before the fixes: stopped queues lacked Send, native admission refused fresh intent, cleanup and dispatch receipts were missing, deferred rows disappeared, live plan Interrupt rejected its own receipt, and Agent Chat `/new` inherited an old non-workspace execution hold. - Six assembled-wakeup cases fail before the reset cleanup fix. All 11 reset/history cases pass after it: live processes, missing identity, quarantined cleanup, remote leases and live controllers remain held; verified cleanup proceeds and retires the hold. Two more cases fail before the fallback fix and pass after it: Send verifies a stopped or live native predecessor despite a later reset receipt and no recovery record. - 525 server tests pass across seven queue admission, route, dispatch, envelope, provenance, blocker and conversation files. 343 queue/thread UI tests also pass. Server/UI TypeScript, token and map guards pass. The final scoped server typecheck also builds native Runner successfully. - The latest regressions prove that completed recovered exports permit fresh input, and clearing conversation history still requires old native stop and cleanup verification; an unsafe workspace still blocks after `/new`; failed native Send reaches actual continuation dispatch with the original queued comment. A result still being committed remains deferred. History plus a newer Stop cannot hide a missing process/controller/lease receipt. - The real Chromium Stop/Send journey passes on current head `75fa5e7222c59f43ff7ac326e81093bc9da28330`: queue a follow-up, Stop, reload, explicitly Send, then verify exactly one useful response and no live run. Earlier browser journeys cover legacy Interrupt, interruption during a write and Stop followed by a new direction. These use an isolated app/database and controlled ACP fixture, not an external AI account or a staging deployment. - Negative cases retain actor, company, issue, assignee, queue and cancellation-receipt bindings; cleanup/process/workspace/controller holds; approvals; pause and budget gates. Typed responses keep their original resolver and immutable result. - Full local recursive typecheck stopped in unchanged Rust Runner compilation with ENOSPC, so the chained full tests/build did not run. All hosted CI gates pass on final commit `75fa5e7222c59f43ff7ac326e81093bc9da28330`, including typecheck, tests, build and E2E. Apex review on that commit is 5/5, with no unresolved findings. Earlier unrelated CI failures involved PostgreSQL fixture-cleanup deadlock and an unchanged decision-expiry deadline assertion; the latter's 20 cases pass locally without source changes. No assertions or limits were weakened. Local resource cleanup was limited to this task's verified completed test processes and rebuildable npm caches. ## Risks - Send can authorize a new turn for a saved message after Stop. A Send click must be newer than the stopped execution; a live plan Interrupt is bound to the cancellation receipt produced by that same click. It does not imply that cleanup finished or that an external action is safe to replay. - The existing endpoint and intent format are reused. No schema or credential change is required. Live native steering and legacy interruption keep their existing paths. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose the exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 20:34:14 -07:00
DottaandPaperclip	f52da40f08	fix: resume queued turns after maintenance (#17967 ) Keep an accepted turn's starting input in task history, show its admission wait, and resume existing queued runs promptly when maintenance ends or expires. Verified with behavioral red/green regressions, real PostgreSQL and process execution, browser/native Runner acceptance with a deterministic provider, workspace typecheck, production build, and current-head CI/review. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-27 22:20:03 -05:00
DottaandPaperclip	09ad9adce2	feat(server): adopt running remote agents through durable controller checkpoints (#17439 ) ## Thinking Path > - Paperclip must keep admitted remote work alive while its main server is replaced. > - Runner owns recovery mechanics and exposes opaque checkpoints through #17097. > - The server owns company scope, encryption, Stop, tool authority and execution ownership. > - A successor must prove current ownership before it adopts the original execution. > - Checkpoint bursts must also leave capacity for normal requests and lease renewal. > - This PR persists and adopts that authority through the existing runtime boundaries. ## Linked Issues or Issue Description What existing behavior does this improve? Remote agents retain the original execution during a bounded main-server outage. Current behavior A server restart can lose the controller's local state and connection. An unsafe replacement could repeat a turn or an uncertain mutation. Checkpoint writes in the request pool can also delay ordinary requests during an aligned burst. Proposed behavior Restore encrypted control records from Postgres and adopt the original runner under a fenced lease. Recover known tool receipts through current tool authority. Give control checkpoints a shared pool capped at two connections per database owner. Depends on #17097. Successors are #17455 and #17456. Qualification and presentation remain separate review slices. ## What Changed - Add company/run-bound encrypted checkpoint records and migration 0394. Keep the SQL idempotent after renumbering against current master. - Recover startup and expired ownership through native runtime and heartbeat. Preserve the sandbox lease, admitted runner artifact, harness configuration, native session and active turn. - Keep Stop, finalization, preparation ownership and stale-writer fencing transactional. Preserve graceful detach and natural crash takeover. - Recover committed tool receipts through the existing authority. Reauthorize reads and hold uncertain mutations. - Bound startup retention with the existing coordinator and preserve zero idle scans. - Share at most two checkpoint connections per database client. Keep application and liveness capacity separate. Close the checkpoint pool during shutdown; late writes cannot reopen it. - Count SQL across both application and checkpoint clients. Prove request/liveness progress, command recovery and failure after pool shutdown with real Postgres. ## Verification Project creation recovery now uses the existing project activity receipt owner. It retains authenticated input normalization, fingerprint checks and current run authority after the receipt lock. An absent receipt returns no result before mutation. The new lost-HTTP-response and no-effect regression uses real API routes and Postgres. The seven transport cases pass locally. This Mac skips all 20 database cases. On current source `5de803c2cd`, Linux [job 122593415890](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415890) executes all 20 project-tool cases successfully (9.480 seconds), including the committed-project lost-response regression. Its shard passes 1,627 cases with three unrelated skips. Server typecheck passes. The transport regression initially has two failures; a subsequent incorrect expectation about schema defaults was corrected before the final pass. Ordinary execution adds no checkpoint writes or recurring work. Current candidate `5de803c2cd` includes the project-creation receipt recovery fix and master `2cd747d1b6`. Migration 0394 retains the prior idempotent checkpoint DDL. Every master SQL file and journal entry is preserved. Snapshots 0390–0394, migration checks and server typecheck pass. The ordinary warm-turn regression fails on its second turn before `a219ab51ad`. After correction, three completed warm turns select the correct prior checkpoint and preserve the original session. All 288 provider-wiring cases pass. The control plane, external persistence callback, identity rotation and filesystem are real; the backend and database-load seam are scripted. The 13 warm admission/configuration variants also pass after final master integration. This does not qualify an underlying harness. At current source `5de803c2cd`, Linux [job 122593415907](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415907) passes all 940 native-session-executor cases (61.431 seconds), including the warm external-checkpoint behavior; the shard passes 2,358 cases with one skip. Current [Runner job 122593415475](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415475) passes the native recovery lane: 115 passes and five container-only skips, including the encrypted restore guard, fresh preparation (63.189 seconds), blocked claim lock (1.226 seconds) and recovery beyond 101 blocked owners (3.897 seconds). The first lane passes 2,215 cases with one skip. The SQL cost test reports zero idle and streaming SQL for both output volumes; this is component evidence, not a production capacity claim. At prior candidate `1ff7bb44f4`, Linux job 122549307215 passes all 938 native-executor cases, including the three-turn external-checkpoint regression. Job 122549306469 passes the native recovery lane with 115 passes and five container-only skips, including the encrypted-restore session guard, fresh preparation (63.664 seconds), claim-lock contention (3.961 seconds), and 101-owner recovery (3.824 seconds). This Mac skips the real-Postgres suite. The earlier full local executor attempt has 921 passes and 16 failures during embedded-Postgres initialization before assertions. The original fixture collisions and their corrections remain recorded. At earlier candidate `8fbe62b41d`, server shard job 122535673841 has 3,876 passes, one failure and one skip in an unchanged expiry-worker retry/drain timing test. The same server/database tree on qualification source `b64cac6951` passes all 20 expiry cases and all 3,877 shard cases. The single diagnostic retry, job 122541891915, passes all 20 expiry cases and all 3,877 shard cases (one skip). No assertion was weakened. The original failure remains recorded and its exact timing race is not proven. All Actions checks and legacy status passed on `8fbe62b41d`, with completed Greptile 5/5 and no open feedback. The next candidate `1ff7bb44f4` also passed all Actions checks and Greptile 5/5. Its Paperclip Review remained queued before the new master conflict required integration. The new candidate needs fresh exact-head CI and reviews; old checks do not qualify it. Migration generation is checkpoint-only, all master SQL and journal entries remain unchanged, and migration checks, server typecheck and all 13 warm-owner variants pass after integration. The first migration check was blocked by TSX sandbox IPC; its permitted repeat passes. The first warm-test command used an incorrect path and ran no tests; the corrected command supplies the 13-case receipt. All 16 database pool-lifetime/options/teardown cases pass after granting loopback permission; the first batch had two EPERM failures and 14 passes. Earlier semantic-tool/HTTP recovery, adoption, encryption-batch, zero-streaming-write and idle-SQL receipts remain source-attributed in the [qualification record](https://github.com/paperclipai/paperclip/blob/2ff66f57da0da35bac1c19a2366ea4b84d87a986/doc/plans/2026-10-22-hot-upgrade-qualification.md). That record retains earlier assertion, setup and cleanup failures and their corrections. No failed gate is waived. The historical 300-session component campaign passes four after phases: steady read p99 is 107–108 ms at 500 reads/s and 100 writes/s; aligned-burst read p99 is 103–105 ms after the bounded pool, versus 584–689 ms before. This comparison isolates the persistence corrections. It does not measure the final stack's hosted active-agent capacity. The hosted zero-agent API pair leaves SQL and CPU essentially unchanged, but both arms fail absolute latency gates and candidate peak cgroup memory fails its gate. The active-agent/viewer matrix, different-artifact upgrade and rollback remain open. Run `PAPERCLIP_TEST_DATABASE_URL=<owned-disposable-postgres> pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-runner-restart-recovery.integration.test.ts`. Previous head `f2db8a6568` had one failed workspace setup gate: [job 122555672441](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122555672441) passes all 4,169 UI tests and 285 CLI cases (one skip), but the company import/export suite reaches its 30-second health deadline during doctor startup. Its output does not reach the server-start step. The CLI, server, database, shared, adapter and plugin trees are identical on qualification source `a866619d7b`, whose [job 122555886663](https://github.com/paperclipai/paperclip/actions/runs/40609057937/job/122555886663) passes that fixture and all 286 CLI cases. The one diagnostic retry, [job 122561057225](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122561057225), passes the formerly failing fixture and all 286 CLI cases. All Actions checks and legacy status on `f2db8a6568` pass. Its exact-head Greptile completed at 5/5 with no open feedback. The current integrated head requires fresh checks and a completed Paperclip Review before merge. The original failure remains recorded; its exact timing cause is unproven. A repeated failure requires investigation and repair rather than another retry. All current-head CI, legacy status, exact-head completed Greptile 5/5, resolved feedback and ordinary GitHub merge eligibility remain required before merge. The latest integration preserves master’s published Copilot v70 profiles and every published migration. Checkpoint migration `0394_conscious_whizzer.sql` was generated with `pnpm db:generate` and retains the prior idempotent DDL. The Runner vendor rebuild and replay parity checks pass. Server typecheck and all 13 warm-owner variants pass after that rebuild. The imported report-lock patch is byte-identical to master; its intentional patch-context whitespace is retained, and the feature delta passes whitespace checks. No hosted experiment was run. ## Risks The pool adds up to two active connections per instance, shared across agents. Checkpoint delivery can wait behind that bound; it still requires its commit. Steady component-test Postgres CPU rises by about 1.7 percentage points of a two-CPU allocation. These corrections do not remove the stack's control and renewal writes. The migration and key must survive replacement. Fresh-database tests and local load do not establish different-artifact hosted upgrade or rollback continuity. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 21:39:23 -05:00

1 2 3 4 5