mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 10:13:11 +02:00
3bd709df8b83aa855dbf300dcc57383b03ec1c88
264
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
3bd709df8b |
Operationalize Product E2E accounting, tracing, and scheduling (#17009)
## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E tests exercise real agents, tools, setup and cleanup. > - Repeated runs need retained costs and traces, including failed attempts. > - A passing retry must not hide earlier spending or feedback. > - Recurring work also needs admission limits, private reports and failure detection. > - This change supplies those controls while keeping paid activation behind existing gates. ## Linked Issues or Issue Description Refs #16996 and #17158. This is the consolidated landing of the reviewed eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033, #17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096, #17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127, #17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository permits squash merges only. A local rehearsal showed that squashing each slice into master causes repeated conflicts in dependent slices. This PR preserves their commits and lands their combined implementation in one reviewed integration. Superseded slices will close after this merge. ## What Changed - Retain every attempt's cost receipt, including setup, failures, interruptions and retries. Separate reported amounts, estimates, missing costs and reserved capacity in reports and dashboards. - Add bounded request admission, exact model pricecards, gateway error accounting, cache policy and reconciliation evidence. Refuse unsupported routes and requests that cannot fit the allowance. - Integrate Braintrust gateway traces and OTLP correlation, with secret filtering and experiment imports. Record phase timing, including setup, feedback capture and cleanup. - Collect complaint and suggestion evidence after each attempt, including recovery after failures. Retain explicit review dispositions independently from the product grade. - Add gated daily and weekly selections, measured forecasts, setup/readiness checks, wall-time limits and retained monitor notifications. Include declared non-model overhead in admission while reporting the existing subscription separately. - Keep recurring history private, bound uploads and retention, and use isolated archive environments. Keep public publication an explicit operation. Report and publication jobs install only the committed lockfile, with scripts disabled and isolated stores; stale pins fail closed. - Integrate current master, preserve its Product E2E journey billing and privacy checks, and test that shared setup receipts do not double-count estimated agent usage. Preserve upstream fixture repairs. ## Verification - Complete integration baseline `0649a5ce7544ab20825c2183d2f421157db49f24` includes master `a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head as ancestors. - On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012 Vitest tests passed, two skipped, across 193 files; all 137 Node source-contract tests passed. - Focused billing and Product E2E journey contract checks: 24 tests passed. They cover estimates, unknown receipts, single-owner setup costs, browser-only execution and private publication guards. - Monitor and workflow wiring: all 96 Node tests passed. `node scripts/check-token-gates.mjs`: all gates clean. - `pnpm test:e2e:runner:typecheck` passes after the standard Runner TypeScript build refreshed local declarations. The first attempt rejected the new master `runner_prepared_launch` union member because this checkout retained an older generated declaration; the source union already contained that member. - Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes dependency re-resolution from archive-authorized reporting jobs after the security scan flagged it. All 15 workflow-security tests pass after updating the old two-install assertion. - Hosted [CI run 39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180) passes on the current repaired head, including typecheck, build, general tests, Runner checks, browser shards and canary dry run. Greptile is 5/5 on this exact head with no unresolved threads. The security scan passes after the fixed lockfile-resolution thread was resolved. - Paperclip Review returned a terminal neutral check but an incomplete 0-file assessment on the repaired head: its platform blocked both diff responses as prompt-injection content. This is not a successful code review. No bypass or fabricated pass was attempted; current-head Greptile, security and hosted CI are independent gates. - No paid test, reservation release or recurring-run activation accompanies this landing. Live route qualification and invoice reconciliation remain separate evidence. <details> <summary>Retained earlier validation failures</summary> Earlier broad local runs had two 15-second test timeouts (CLI poll cadence and Retry validation), one embedded-database setup timeout, and later PostgreSQL semaphore exhaustion. The exact timed-out tests passed separately without larger timeouts; the broad-run causes were not established. Those local failures remain distinct from subsequent hosted CI results. The independent #17158 fix records its Muse fixture collision, its UUID correction and hosted verification. Local Muse execution was also blocked by macOS embedded-PostgreSQL startup. No historical failure is treated as a successful paid qualification. </details> ## Risks - Admission is a local control, not a provider invoice cap. Missing provider receipts or resource charges retain unknown amounts and holds; estimates do not become billed dollars. - The bounded path admits only documented local suites and provider routes. Ordinary legacy or separate Runner protocol campaigns have different controls. Merging this code does not qualify additional routes. - Recurrence still needs the existing enable gates and current qualification/allowance evidence. Archive permissions and resources must match the documented private-environment contract. - Legacy reports may lack complete attempt or phase evidence. Reports preserve that limitation and do not infer zero usage from missing data. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks; full-suite host limitation disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4173059081 |
fix(inbox): keep advisory GitHub review waits quiet (#17318)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Mine shows a person's participation and requests that need their response. > - Automatic GitHub reviews should remain quiet until the assigned work needs human input. > - Review bots used completion requests to keep tasks in review while CI or a PR was still open. > - Those requests correctly entered Mine, despite the automatic-work filter from #16356. > - This pull request gives published reviews an existing provider-event wait in the in-review state and withdraws superseded completion requests. > - Real input requests and separate approval gates keep their current behavior. ## Linked Issues or Issue Description Refs #16356 Refs #13314 An advisory score below 5/5, pending or failed CI, or an unmerged PR does not by itself require a person's approval in Paperclip. The bot can track those conditions without adding a completion card to Mine. ## What Changed - Explain the quiet review lifecycle in the GitHub review instructions. Use a yielded provider-event wait, or an existing saved monitor, after publication. Report done when the configured completion conditions are met. - Keep a confirmed, published review in `in_review` during that wait. Reuse existing authorization and monitor delivery. Add no schema, timer, or polling loop. - Recognize the committed external wait in review attention and recovery. Shared execution receives a verified publication flag; GitHub receipt lookup stays with the GitHub module. Other chat waits keep their current status. - Withdraw an older system-created completion request only when the same bot has published both reviews and its new report has no attention request. Keep explicit cards, separate approval gates, and requests without the prior review evidence. Finish validation stays read-only; withdrawal commits atomically with the new status after checking current governance, the admitted result, and both low-risk contracts. - Update the product rule, capability maps, and regression coverage. ## Verification - Final commit `7c502118eec36028d1b4537bb8797ef893940bf0`: 54 GitHub checks passed, two intentionally skipped, none pending or failed. Greptile is 5/5, the security scan passed, and all review threads are resolved. - Passed: `pnpm -r typecheck` and `pnpm build`. Rechecked server typecheck and build after the final authorization guard. - Passed: 195 tests across `external-chat-wait.integration.test.ts`, `status-arbiter.test.ts`, `review-instructions.test.ts`, and `issue-liveness.test.ts`. The database suite used an isolated PostgreSQL 18 container through `PAPERCLIP_TEST_DATABASE_URL`. - Acceptance coverage: 3/5 stays in review outside Mine; 5/5 with pending completion conditions stays in review; 5/5 with satisfied conditions finishes quietly; a real repository-access request enters Mine until resolved. Also covers obsolete-card withdrawal, transaction rollback, rejected reports, separate approval gates, multiple reviewed heads, saved monitors, and revoked endpoints, repositories, or membership. - Passed: `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check`. - The initial acceptance run exposed a missing authenticated user in the test fixture. Added the user required by the existing addressee authorization check. All 154 tests in that database suite then passed, including the final rerun. - Review found premature withdrawal during validation, stale repository access, and a missed receipt when a run reviewed multiple heads. The revision fixes all three and includes acceptance regressions. The saved-monitor commit guard now accepts the verified published-review state. - `pnpm test:run` was attempted but could not complete on this Mac. Embedded PostgreSQL failed to allocate semaphores (`semget: No space left on device`); a direct initialization probe confirmed the host resource limit. The isolated database run proves the affected suite, not the full repository suite. - These tests exercise persisted review receipts, finalization, and Mine queries. They do not prove a live model followed the new instructions or submit a live GitHub review. ## Risks - The bot must follow the new lifecycle instructions. A real explicit attention request still enters Mine. - Existing obsolete cards are withdrawn when the same bot publishes a later review and reports no human dependency. This change does not run a bulk cleanup of historical inbox entries. - Review attention now recognizes an external response path. It requires an applied current status decision, a successful bound run, and current conversation, repository, and principal authorization. - No migration or UI change. The full local test suite is unverified because of the host database limit; CI passes on the final commit. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code editing, and test execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (195 focused tests; full-suite limitation above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
583ec2443a | Read deliverable files before locking mutation rows (#17309) | ||
|
Nicky Leach
|
412914cfa7 | Read native plugin task output files on the execution host (#17285) | ||
|
Nicky Leach
|
2cd6eb27c7 | fix(runner): record the plugin-prepared working directory (#17243) | ||
|
Dotta
|
46760cc588 |
fix(chat): honor dismissal for all composer requests (#17218)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations show requests in the feed and above the message composer. > - Earlier fixes kept dismissed questions in history, but excluded completion reviews and other request kinds. > - Closing those forms left a pending badge, and the server could still demand a response. > - This pull request saves human dismissal and removes the request from composer reminders and ordinary completion gates. > - People can continue the conversation and reopen the request from history. ## Linked Issues or Issue Description Related work: #15229 and #15392. **What happened?** Dismissing a completion review closed its form but left it counted as pending. It could return after reload, block a retry, or cause the agent to demand another response. Raw runtime requests could also lose their only usable control when dismissed. **Expected behavior** Dismissal leaves an accessible request in the feed and clears the composer form and pending count. It persists after reload. It does not submit an answer, approve an action, or wake the agent. **Steps to reproduce** 1. Open a task with a pending completion review or another composer request. 2. Dismiss the request, then reload the task. 3. Check that the composer stays clear and the feed entry can reopen the request. ## What Changed - Add nullable dismissal fields and a human-only dismissal endpoint with company, private-task access, resolver, and locked review-policy checks. - Persist saved-request dismissal on the server. Keep browser persistence for raw runtime requests and existing question dismissals. - Keep confirmations, choices, item verdicts, suggested tasks, and runtime requests accessible in the feed. - Exclude dismissed ordinary requests from completion and retry gates. Tell agents not to repeat deferred requests. - Preserve tool, secret, connection, plan, company-policy, and configured review authorization gates. Expire deferred non-question requests when their task completes. - Preserve the selected form when another dismissal finishes saving. Keep reopened reviews visible through delayed comment refreshes. - Add a completion-review Storybook test drive and update design, API, database, and capability documentation. ## Verification - `pnpm -r typecheck`: passed. The final UI changes also passed the UI typecheck. - `pnpm build`: passed. The final UI changes also passed the UI production build. - Focused UI and transcript tests: 550 passed across nine files after merging the current base branch. - Focused database, completion, and prompt tests: 105 passed across three files, including two review-policy regressions that failed before the fix. The tests reject a review requester excluded by current policy, preserve the pending gate on denial, and permit an independent reviewer to defer the request. - Route regression suite: 81 passed, including denial before reading or mutating a private-task request. - `pnpm check:token-gates`: passed. - Map and Storybook structural checks: passed; 30 map/catalog script tests passed. - Browser: used the production task-thread components in the completion-review fixture. Dismissed, reloaded, reopened from the feed, and dismissed again. No pending badge or composer takeover remained. Story callbacks simulate API saves; separate server tests cover persistence and permissions. - `pnpm test:run`: stopped after the general-server group reported one failure in `workspace-runtime.test.ts`: "returns a bounded structured conflict and exposes only same-company workspace references". The same test passed in isolation (1 passed, 178 skipped). The cause of the full-suite failure is not established. The full PR CI suite subsequently passed on e243cbb001; its two authorization review findings are fixed in |
||
|
DottaandPaperclip
|
734c265339 |
Default new Pi agents to Paperclip Runner (#17142)
## Thinking Path > - Paperclip manages AI agents and their work. > - Users choose a harness during agent setup. > - Claude, Codex, and OpenCode already receive their qualified Runner default. > - Pi has a qualified native integration but new Pi agents still use the direct adapter. > - This change gives new Pi agents the native default on supported targets. > - Saved agents keep their runner and compatible settings. Legacy remains available in Advanced. ## Linked Issues or Issue Description Refs #17120, #14921, #14922, #14923, #14924, and #14956. Related generic default work: #15422. This PR is the separate Pi follow-up to the merged OpenCode change. **Current behavior** Choosing Pi for a new agent saves the direct Pi adapter. The native Pi integration remains behind the generic experimental flag. **Proposed behavior** Default new Pi agents to ACPX Pi on macOS ARM64/x64 and GNU/Linux x64 with supported glibc. Keep unsupported targets and explicit Legacy selections on the direct adapter. Require an explicit provider/model before test or hire. Permit model-free inactive setup drafts. **Reason and benefit** Users choose Pi once and receive its qualified runner. Saved agents retain their identity and settings. Conversions reject unsupported legacy thinking and launch settings instead of dropping them. ## What Changed - Add Pi to the shared platform/default contract and server creation selection. - Admit native Pi without the generic experimental flag. - Preserve provider models, credentials, workspace settings, exact thinking, and saved native policies. - Add Pi runner controls, branding, setup resume, onboarding, built-in configuration, and import handling. - Add managed Pi OpenRouter routing and matching native provider API-key accounts. Keep subscription and custom gateway exclusions. Clear alternate credentials and reject custom Pi provider overrides. - Keep Pi draft models empty instead of applying Claude's default model. - Verify setup against the pinned Pi runtime. For native Pi on the official OpenRouter route, reverify the API key live without requiring a global Legacy Pi CLI. - Match Pi’s bounded controller close window to its native snapshot cleanup and lifetime verification, preserving explicit caller deadlines and the mandatory drain/suspension proof. - Bind Pi sidecars to the package that owns the pinned native assets, including source and vendored server layouts. Other providers retain their existing package authority. - Keep protocol-owned finish/block tools out of the ACPX forwarded catalog. The trusted bridge installs these tools itself; the complete Rust authorization catalog and terminal receipt ledger remain intact. This fixes actual Pi session startup and applies to other ACPX consumers. - Advance Copilot’s shared-source attestation to v58 because it pins the shared files touched by Pi. Preserve immutable v57 bytes and historical decoding; Pi’s own release identity is unchanged. - Add regression coverage and update the paired maps and developer documentation. ## Verification - Passed: full repository typecheck and full production build on final head |
||
|
DottaandPaperclip
|
39e44a9d9b |
Default new OpenCode agents to Paperclip Runner (#17120)
Default new OpenCode agents to Paperclip Runner on qualified macOS and Linux x64 targets. Preserve saved runner choices, explicit Legacy selection, and unsupported-target fallback. Pi remains a separate change. Verified the complete browser path with a real OpenRouter connection: create the new agent, assign a task, observe its response and Done status, and reload to confirm persistence. CI passes and Greptile is 5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0a5f1546ee |
feat(cases): add reusable prompt workflows (#17114)
## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
6dda8cc2b7 |
fix: preserve Stop authority and qualify complete recovery journeys (#17105)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final head |
||
|
|
eece07a747 |
fix: repair the complete projectless repository setup chain (#17088)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
00331e0254 |
Connect native Runner admission to plugin-owned workspaces (#16978)
## Thinking Path > - Paperclip manages work for AI agents. > - A plugin can own an execution environment and its files. > - #17029 supplies the task driver and bounded setup verification. > - Heartbeat still prepares workspace files on Core and expects a shell transport. > - This PR gives the selected plugin ownership of workspace preparation and native task execution. > - Core keeps admission, tool authority, and recovery checks. ## Linked Issues or Issue Description **Subsystem affected** Heartbeat workspace admission and native Runner execution. **Problem or motivation** A task-only plugin cannot run through Core filesystem staging or shell execution. Native admission and recovery must use the same lease and workspace ownership. **Proposed solution** Use the task driver from #17029. Resolve workspace and project identities without Core file operations. Connect native dispatch and restart recovery to that driver. **Alternatives considered** Keep workspace admission and dispatch in one change. A workspace change alone would leave dispatch on the previous transport. An execution change alone would leave Core staging files for the wrong host. ## What Changed - Enable plugin environment selection for native workspace sessions. - Resolve workspace metadata and authorized project references without Core checkout, copy, or provision operations. - Preserve agent-default and primary workspace policies. Use the pinned workspace provider for plugin-backed project storage. Reject unsupported Core service commands and cross-owner reuse. - Connect native execution to the prepared task driver. Keep assigned MCP authority on Core. - Save exact checkpoint identity, transfer selected account refreshes, and require termination proof before replacement or cleanup. - Complete managed plugin workspaces through their storage provider. Use the same check after a restart. Keep branch validation and leave task-owned files on the plugin. - Include default instruction content in runtime synchronization when no instruction directory exists. - Defer new-agent Copilot verification until the saved-agent lifecycle can acquire the agent-owned computer. ## Verification - Rebased onto `origin/master` at `898144f4b3`. Master advanced once during verification. Both manual conflicts joined import blocks: one in `native-session-executor.ts`, and one in `heartbeat/runtime-execution.ts`. The seven patches retain their logic. The first rebased head passed repository typecheck, build, all 862 focused execution/finalizer cases, and all CI gates. The latest head `93e5a72892` passes server typecheck and all 880 focused execution, finalization, and execution-control notification tests. All 54 CI/review checks pass, with two expected skips. Greptile ran automatically and scored this head 5/5 with no unresolved findings. No new review or Apex request was made for these import-only resolutions. - The successful CI workflow initially left three stale job-check reports. Targeted policy and summary reruns cleared the reports. No source change or test fix was needed. - Repository typecheck and build pass: `pnpm -r typecheck` and `pnpm build`. - Focused native execution and restart tests pass: 795 cases. The account and harness suites pass: 159 cases. Workspace admission passes: 48 cases. The SDK task lifecycle, workspace API, runtime-hook, and instruction checks also pass. - The heartbeat integration case uses the merged workspace provider contract and reaches native dispatch with no project files on Core. The new-agent case confirms that creation leaves the agent in `preparing` until lifecycle verification. - Map validation and the Copilot source-identity check pass. No Runner implementation or source-identity bump is in this PR. - The first rebased workspace run found two stale path assertions and a recovery guard that did not recognize task-owned targets. These are fixed. The storage integration fixture also needed its persisted source binding; the production authorization check stays in place. - Two prior review findings required rejection of Core service startup and cross-owner workspace reuse. The existing admission fixtures now check both failures. The installed-account authentication finding was fixed in merged #17029. - Tests reuse the current fixtures. No new provider test matrix or live Runner process is added for these fixes. - The prior head `2db273ec12` passed CI on its first attempt and reached standard Greptile 5/5. Apex scored 4/5 and found local-folder checks during plugin completion and recovery. - The completion fix reproduces both failures. It passes all 110 cases in the runtime execution, workspace finalizer, and workspace provider integration suites. Server typecheck and map checks pass. CI passed on `62d8335b4e` on its first attempt, and standard Greptile scored 5/5. The next Apex review scored 4/5. It found that a failed plugin inspection could abort restart recovery. - The restart fix records a failed workspace receipt and uses the current run retry policy. A regression first reproduced the aborted pass. It now checks retry state, retained results, and completion of another workspace in the same pass. All 68 finalizer cases and server typecheck pass. CI passed on `7d4016323b` on its first attempt. Standard Greptile and Apex both scored 5/5. Apex left one P2 about lost failure codes. - The diagnostic fix keeps the known branch and owner mismatch codes in receipts and run recovery state. It gives repair guidance for those codes and keeps unknown plugin errors generic. All 96 affected finalizer cases and server typecheck pass. Before the rebase, required CI passed on `c2fa61c6c8` on its first attempt. Standard Greptile and Apex both scored 5/5 on that head, with no unresolved findings. The new head retains these reviewed patches. - The earlier Superagent scan on `c2fa61c6c8` was inconclusive after a pinned merge-base checkout failed or timed out. The current scan on `93e5a72892` passes. - The exact rebased head passes all 48 workspace cases and server typecheck. The first local test launch could not bind PostgreSQL inside the sandbox. The authorized local-server run passed. - The prior head reached standard Greptile 5/5. Its first CI run hit an unchanged Runner cleanup timeout and an external build-runner shutdown. Both jobs passed on the single retry. The timeout cause is not established. - A local regression check found that a retained Core workspace could also enter a plugin task without file transfer. Core now rejects this move before provider work. The existing ledger test covers both directions and confirms that the source files stay intact. - The first active standard review scored 4/5. It found that a fresh plugin retry could enter Core Git preparation. This mode now uses the existing unsupported-storage error before workspace resolution or acquisition. Both unsupported-mode cases pass, and server typecheck passes. - No live Cloud or provider validation was run. ## Risks - The prerequisite #17029 is merged. This PR targets master. - Native plugin tasks support per-turn workspace sessions. Core service commands and fresh retries that import retained files remain unsupported. A workspace storage plugin can use the modes and strategies it declares. - Core-managed credentials support Codex and direct Claude accounts. Other managed credential routes fail closed. Other harnesses can use an installed account. - A provider must preserve its prepared configuration and report process termination correctly. Uncertain termination keeps the task pending cleanup. - Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
898144f4b3 |
fix: stop idle execution-control recovery polling (#17094)
Replace idle execution-control polling with workers driven by committed writes and persisted deadlines. Preserve startup recovery, bounded retries, and shutdown safety. Stabilize the Slack setup browser regression by waiting for the restored setup screen before simulating its webhook. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
30b753f614 |
perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path > - Paperclip manages AI agents and their paid work. > - Budgets must prevent new work and stop reported overruns. > - Cost settlement also ran serial lifecycle reconciliation for every eligible company agent. > - That made request latency grow with unrelated idle agents and delayed results on stop delivery. > - This change separates atomic admission and accounting from durable asynchronous enforcement. > - Scoped live usage detects overruns without writing provisional ledger charges. > - The result is constant query scope for ordinary settlement and recoverable stop delivery. ## Linked Issues or Issue Description **What happened?** Settlement and budget checks entered company-wide lifecycle reconciliation. A company with 1,000 unrelated idle agents performed 1,001 accounting transactions for one ordinary cost evaluation. User requests could also wait for provider termination. Running usage checkpoints did not enforce reported monetary exposure. **Expected behavior** Keep the final reservation gate authoritative. Commit charges, scope blocks, and enforcement intent before returning. Deliver stop requests after commit. Read only applicable policies and execution scopes for ordinary charges. Stop long-running work when supported reported or priced usage establishes an overrun. **Steps to reproduce** 1. Create a disposable company with one company budget and idle agents. 2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated idle agents. 3. Compare query counts, company accounting transactions, and duration. 4. Use the controlled partial-usage and delayed-stop fixtures to check enforcement and request responsiveness. **Paperclip version or commit** The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from `master`. Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853, #13075. These cover the affected service charge, cancellation, and accounting paths. This change removes their shared reconciliation overhead. ## What Changed - Batch applicable policy exposure under the existing company accounting lock. Preserve exact fractional money and zero-reservation dispatch fences. - Commit a durable final budget dispatch claim before provider I/O. Retain a separate final issue/run ownership transaction through synchronous handoff, without holding the company accounting lock or awaiting provider work. Preserve native recovery lease checks and duplicate handoff fencing. Browser handoff rechecks current policy; it reuses a consumed rate slot only for the identical policy generation. - Separate ledger spend, live observed spend, and effective held capacity. Use the maximum of reservation and live observation for each unfinished operation. - Deliver budget cancellation through the existing coordinator. Revalidate generations and persist queued/running stop fences before external termination. Bound stop concurrency to four. - Apply agent holds only to the affected agent. Project budgets leave other projects running. Project real company transitions in bounded lifecycle batches. - Replace periodic broad reconciliation with durable pending work, startup repair, receipt replay, and concrete retry/window deadlines. Retain drain and idle-safety checks. - Enforce accepted cumulative usage checkpoints without provisional charges. Final accounting atomically replaces live exposure and settles reservations. - Add operator-gated budget spans, regression coverage, map updates, and a real-PostgreSQL comparison harness. No schema or first-party telemetry change. ## Verification - Full workspace `pnpm -r typecheck` passes on the final ownership fix, rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head `4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks pass. - Focused real-PostgreSQL coverage includes the 292-case rebased accounting/dispatch run, 63 browser cases, 52 run-control/cancellation cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and the exact Muse confirmation regression. Native executor unit coverage passes all 768 cases. Final ownership coverage passes: 74 adapter/use-case cases, nine public regression cases, and the additional deleted-project race. These exercise competing ownership, budget changes and charges between phases, zero/nonzero duplicate claims, rollback, recovery contention, read-only capacity rechecks, and controlled HTTP startup after commit. Another 74 pure/startup/tracing cases pass. The complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its external security scan identified a stale-owner handoff race. The final fix restores synchronous issue/run ownership fencing after the accounting transaction commits. Final head `8e84f55ce8` has fresh Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing security scan. The complete latest-head [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808) passes, including budget, browser, Runner, build, and canary jobs. The unchanged Runner close fixture passed on its sole same-head job retry. - Broad local `pnpm test:run` invocations were stopped before required source updates (rebase and review fixes). They are not claimed as full passes. Final full-suite coverage comes from the complete latest-head CI matrix; local focused coverage and the incomplete broad invocations are reported separately. - Reproduce performance with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-enforcement.ts --baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with `--reverse`. The harness uses disposable real PostgreSQL, 20 warm samples per variant/count, and records raw timings, SQL/lock counts, agent reads, unchanged agent state, and query plans. Final production-source measurement revision: `eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is committed in `4c5f4de8b1`. | Unrelated idle agents | Baseline median / p95 | New median / p95 | Baseline SQL / locks | New SQL / locks | New agent reads | | --- | ---: | ---: | ---: | ---: | ---: | | 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 | | 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 | | 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1 | 0 | - These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64. They isolate the cost-evaluation core and complete admission path, without hosted-region or provider latency. Prior-revision reversed-order controls also preserved the constant query counts; the final measurements above use the final production source. - Complete reserve → final ownership gate → immediate offline provider result is measured separately, with one responsible agent and one active operation at each unrelated-agent count. Admission p95 baseline/new: 0 agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents **24.51 / 74.05 ms**. Baseline admission was already scoped; the stronger final fence adds bounded overhead. New counts stay constant at 53 Drizzle statements, three transactions, two company-accounting locks, and five agent-reading statements bound to the responsible agent. Zero unrelated-agent reads, exactly one provider callback per sample, and no leftover runs or holds. All admission scenarios stay below 250 ms. No external provider calls are included. - The partial-usage/delayed-termination fixture passed 20/20: receipt persistence p95 13 ms, receipt return to observed durable Stop fence p95 28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database request p95 1 ms while termination was blocked. After releasing the controlled termination fixture, acknowledgement p95 was 14 ms. These are separately measured phases; fixture acknowledgement is not live-provider termination latency. - Models and provider deadlines are unchanged. No deployment, live inference, or staging acceptance is claimed. A quiet 30-minute follow-up waits for the merged revision on staging, then repeats classifier and controlled budget-stop acceptance in staging, recording hosted bookkeeping, queue delay, and actual provider acknowledgement separately. - Development failures retained: finalizer notification regressions (fixed; 27/27 affected cases passed), invalid browser admission fixture assumptions (fixed; browser coverage now passes 63/63), and first-head lifecycle/dispatch CI regressions (reproduced and repaired). A prior-head Cases E2E test hit its unchanged announcement-dismiss handler race while opening a filter; the same journey passed on `1ac9b58`. The external security review then found a stale-owner race between accounting commit and provider entry; final verification covers a competing run in that gap while retaining provider I/O outside the accounting lock. Early PostgreSQL startup attempts hit local semaphore pressure; serial reruns succeeded. The final-head Runner lane had one failure among 1,823 tests: the unchanged `timeout-attachment` fixture failed at its first process close after 6.28 seconds. The sole same-head job retry passed all 1,819 active tests (four skipped), including that fixture in 1.85 seconds. Three isolated local probes could not run correctly because the Rust fake-provider fixture binary was absent; cleanup masked the original startup failure. CI builds that fixture through its normal lane. These probes are inconclusive and are not counted as passes. ## Risks - This changes accounting, lifecycle, and cancellation ordering. Durable generations, source/sequence checks, ownership leases, and company locking remain required for correctness. - Provider stop acknowledgement can be slow or fail. Stop intent remains pending until existing termination proof succeeds; cancelled row status alone does not acknowledge delivery. - Mid-run limits depend on provider usage reports or supported price calculations. Providers that report only at completion cannot provide exact mid-call detection. - Signals are process-local wake hints for the supported single-active-writer model. Durable state and startup recovery remain authoritative. - There are no migrations, new queue tables, or cached balances. Repeat the classifier and controlled budget-stop acceptance after normal rollout. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and parallel coding agents. The host does not expose the exact backend model revision or context-window size, so those values are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dff63551fe |
fix: resume unfinished work after repository preparation (#17047)
## Thinking Path
> - Paperclip manages agents that perform work for a company.
> - Task repositories are prepared at normal run admission.
> - A running native task can stage a repository while its work remains
unfinished.
> - The controller did not offer that saved setup as a continuation.
> - The agent could add a timer or wait for a user restart.
> - This change uses the existing same-agent continuation and status
ledger.
> - The accepted explanation and proven conversation remain available
after setup.
## Linked Issues or Issue Description
Refs #17045. That prerequisite has merged. This PR targets master.
**What happened?**
A projectless native task can request a repository for the next
admission. An unfinished task has no server-selected repository
continuation. An explicit same-agent continuation also loses chat
provenance.
**Expected behavior**
An unfinished standard task can yield and resume through the existing
admission queue. Its explanation and final result retain the authorized
conversation. A task that only stages a repository can complete without
another run.
**Steps to reproduce**
1. Start a native standard task in a task directory.
2. Request an authorized repository through prepare_repository.
3. Yield with unfinished work and no explicit continuation.
4. Observe that the former driver rejects the yield, although the
pending repository is saved.
## What Changed
- Offer a saved same-agent continuation for the current task's pending
repository.
- Accept that current server continuation in the shared provider driver.
- Resolve chat provenance through applied status decisions, delivered
effects, and admitted wakes.
- Preserve accepted setup explanations without repeating an owned
question card.
- Document optional reproduction details and configured repository
discovery.
- Extend database regressions and workspace access maps.
- Preserve Copilot profile v52 and bind the shared continuation source
in v53. Include the active identity in the QA image fingerprint.
## Verification
- Five focused server suites: 100 checks pass.
- Shared continuation driver: 12 checks pass.
- Server typecheck passes.
- Map contract and whitespace checks pass.
- Existing API and driver assertions fail on the former implementation.
- Real dispatcher and queue cohort: 43 checks pass. The original
reply-loss regression fails before the dispatch receipt fix.
- Continuation driver and installation identity cohort: 62 checks pass.
- Regenerated capability inventory and contract checks pass.
- Final response precedence cohort: 109 checks pass. The stale-comment
case fails on the former implementation.
- All required CI checks, build, and both source reviews pass on
|
||
|
DottaandPaperclip
|
c2daf906d9 |
fix: preserve task continuation and question delivery (#17045)
## Thinking Path
> - Paperclip manages agent work and delivers its results.
> - One task can need several runs and a saved workspace.
> - A run can succeed while the task waits for its next run.
> - Chat fallback treated that success as completion and dropped useful
status.
> - Monitor runs also lost their conversation origin, and text-question
cards lost their prompt.
> - This change preserves the task outcome and exact delivery authority
across those boundaries.
## Linked Issues or Issue Description
**What happened?**
A Slack task requested a repository, yielded to a scheduled
continuation, and showed only a generic completion message. The next run
could not publish through the original conversation. A later text
question showed its title and Respond button without the question
context. Workspace recovery could commit the saved monitor result
without restoring its explanation.
**Expected behavior**
The original conversation shows the accepted continuation explanation,
receives the resumed result, and displays the full question when input
is needed. A successful provider run does not claim that unfinished work
is complete. Recovery delivers saved output once without another
provider result.
**Steps to reproduce**
Start an authorized chat task, prepare a repository for a later
admission, save a monitor wait, and let that monitor resume the task.
Repeat with workspace finalization interrupted and with a free-text
question. Inspect the originating conversation and the persisted output.
Related merged work: #16806 and #16975. The open monitor and recovery
changes found during duplicate search address different status and
liveness policies.
## What Changed
- Keep committed monitor summaries visible through the shared board and
chat response resolver. Preserve interaction ownership and comment
reuse.
- Recover a saved monitor explanation after finalization, with no
repeated provider result.
- Prove monitor publication origin from the admitted wake and matching
scheduling receipt. Match company, task, agent, requester, and schedule;
reject ambiguous sources.
- Include monitor continuations in the existing publication scan and use
the canonical task binding. Keep current transport authorization.
- Suppress completion fallback for yielded results. Explain missing
final output without claiming completion.
- Include question summaries, prompts, and help text in modal-backed
cards. Keep answered text private.
- Update execution semantics, capability maps, and the bounded
reliability plan.
## Verification
- Original source fails four composed PostgreSQL regressions: monitor
origin, yielded completion, question body, and saved-summary recovery.
The shared response resolver also fails before the repair.
- Repaired focused cohort: 148 tests pass across publication, question
privacy, scheduler, and native monitor suites. These use real disposable
PostgreSQL; they do not qualify live Slack.
- Full workspace typecheck and production build pass locally. Map checks
and whitespace checks pass.
- All CI gates pass at
|
||
|
Nicky LeachandPaperclip
|
e5351b240a |
Verify native Runner setup through plugin tasks (#17029)
## Thinking Path > - Paperclip manages work for AI agents. > - An environment plugin can own a Runner task. > - The merged launch-preparation contract lets the execution owner build the provider configuration. > - Core still needs a task process owner and a setup test that uses that configuration. > - This PR connects the task API to the Runner process-owner contract. > - Operators can test a plugin environment before Core sends it task work. ## Linked Issues or Issue Description **Subsystem affected** Runner, plugin SDK, and agent environment tests. **Problem or motivation** A task-only environment cannot use the shell-based setup path. Core must verify the Runner hello exchange and process cleanup through dedicated task operations. **Proposed solution** Add a lease-bound task driver and use it for a bounded setup probe. Keep heartbeat admission and task execution in #16978. This PR follows the merged launch-preparation contract in #16997. **Alternatives considered** A provider declaration alone cannot prove that Runner and the provider connect. A shell bridge would give the plugin an unnecessary command interface. ## What Changed - Pass structured Runner launch data to process owners. Support an external connection and explicit detach. - Add typed launch authentication checks and checkpoint operations. Check lease, run, project, and retained-task authority. - Connect launch preparation, PRP, termination proof, and credential handoff to the task driver. Reject local environment overrides during installed-account setup. Pass saved-account identity and credential home directly from Core. - Run a short-lived setup probe through the existing agent environment test API. Require a verified provider hello and cleanup. Keep saved Copilot account checks metadata-only through a dedicated task operation. - Update the Copilot source identity because the shared transport source changed. Keep provider behavior and the earlier identity fixture. ## Verification - CI passes on `f987adec7`: 55 successful checks and 5 expected skips. Standard Greptile and Apex are both 5/5 with no open findings. The branch merges cleanly with current master. - Repository typecheck and build pass after the rebase onto the workspace-provider contract. Server typecheck and build also pass after the final setup fix. - The 110 focused lease, SDK, setup, Copilot, and route cases pass after the rebase. All 14 setup cases pass after the final change. These include a real local Runner with a test provider. The saved-agent model-discovery case also passes. - Apex found that Core's relative workspace marker failed the driver path check, and non-Codex sessions omitted the observed model. Both failures were reproduced and fixed. The probe leaves physical workspace selection to plugin preparation. The transport reads each provider's observed model and exposes it to the driver. All 14 setup cases, 6 transport peer cases, and both existing real Runner/OpenCode cases pass. Runner build and server typecheck pass. - The coverage audit removed two redundant cases, repeated fixture code, and two extra real Runner launches. It kept authorization, credential selection, cancellation, lost replies, termination proof, checkpoint identity, and provider setup checks. - The Copilot source-identity check passes. The added-line secret and PII scan passes. The unmerged v52 attestation was regenerated after the shared transport fix. - CI on `13faeff02` caught one stale Copilot digest assertion after attestation regeneration. The literal now matches the candidate attestation, and all 49 installation cases pass. That CI lane otherwise passed 1,818 cases. - An earlier CI run hit an unchanged UI toast timing test. The separate fix is tracked in #16952. Current CI passes that shard. CI also caught a stale model-discovery assertion in this change; the assertion is fixed. - The local full-suite run was stopped when master changed. The final CI run supplies exact-head full-suite validation. No live Cloud or provider test was run. ## Risks - Native hello tests require a saved Core account or the installed account. Local environment overrides are unsupported for installed-account probes and fail before the provider runs. - A native task provider must declare launch preparation and authentication-check version 1. It must keep the checked configuration fixed through submission. Core rejects providers that cannot make this guarantee. - A lost task reply does not prove process termination. Core retains grants until the provider proves termination and Core saves required state. - This PR does not enable native heartbeat dispatch through plugin environments. Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d33b40a5c3 |
fix: recover saved work and provide reliable task Retry (#16965)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description **What happened?** A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. **Expected behavior** Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. **Steps to reproduce** 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
572bef4ba7 |
fix: omit re-downloadable Codex remote plugins from failover backups (#17026)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native Codex sessions in remote sandboxes copy the sandbox's Codex
home into a host-side failover backup after every turn
> - Ephemeral sandboxes restore that backup at the start of every turn,
so its size affects both disk use and turn latency
> - Most of each backup is remote plugin data that Codex itself treats
as a cache and re-downloads on startup
> - This pull request leaves that data out of Codex checkpoints, keeping
everything a resumed session needs
> - The benefit is backups roughly 80–90% smaller, less data moved per
turn, and much slower disk growth
## Linked Issues or Issue Description
Bug report described in-PR (bug report template). Related: #16866
(expires failover backups after their issues close).
**What happened?**
Failover backups grow quickly because each Codex checkpoint includes
`$CODEX_HOME` remote plugin data. On one instance, 204 backups held 22.1
GB:
- `codex-home/plugins/cache/openai-curated-remote`: 8.3 GB (about 29 MB
per backup);
- `codex-home/cache/remote_plugin_catalog`: 6.1 GB (one 38 MB catalog
file per backup);
- `codex-home/sessions` (the conversation history): 0.31 GB.
About 80 new backups (9.8 GB) were written per day. Ephemeral sandboxes
also copy all of this into a fresh sandbox before each turn.
**Expected behavior**
Checkpoints keep what a resumed session needs and omit data that Codex
re-downloads itself.
**Steps to reproduce**
1. Run a Codex agent with ChatGPT auth in a remote sandbox environment.
2. Inspect
`<instance>/runtime/paperclip-runner/durable-sessions/*/failover-backups/current/codex-home`.
3. Most bytes are under `plugins/cache/openai-curated-remote` and
`cache/remote_plugin_catalog`.
**Paperclip version or commit:** `master` at
|
||
|
DottaandPaperclip
|
505e1a5582 |
feat(cases): add persistent workflows with same-task agent handoffs (#16959)
## Thinking Path > - Paperclip manages AI agents and their work. > - An agent workflow needs persistent facts and a safe way to transfer work. > - Creating a task for each phase splits the objective, discussion, workspace, and review policy. > - A case can retain the facts while one primary task moves between agents. > - Transfers must survive process failure and preserve private access, budgets, and completion gates. > - This pull request adds durable case handoffs, native case tools, and read-only case pages. > - Prompts describe the workflow. Existing task controls decide when work can run and finish. ## Linked Issues or Issue Description **Problem or motivation** Prompt-driven agent teams need a persistent record that can move through triage, implementation, verification, and correction. Ordinary reassignment does not provide a durable instruction, safe release barrier, or recoverable delivery receipt for an agent handing off its own running task. **Proposed solution** Extend standalone cases with one primary task, a descriptive phase, retained access sources, and a handoff ledger. Apply assignments after source execution and workspace settlement. Deliver each transfer through the existing durable wake queue. Provide `create_case`, `read_case`, `update_case`, and `handoff_case` in the native runner. Creation requires kickoff and explicit workflow text. An explicit handoff to `done` closes the case when normal task completion is accepted. **Alternatives considered** Do not add a workflow engine, filesystem queue, configured state graph, or one task per phase. The workflow document combines transition prose with an optional Mermaid `stateDiagram-v2` map. The server does not execute the diagram. The Pipelines experiment removed on master stays removed. **Roadmap alignment** This extends Agent Reviews and Approvals, Enforced Outcomes, Bounded Run Recovery, and Activity Log attribution. Existing approval, review, budget, privacy, workspace, and responsible-user rules remain authoritative. Related work: [ordinary handoff wake recovery](https://github.com/paperclipai/paperclip/pull/15019). This feature adds a separate persisted case-handoff receipt and release barrier. ## What Changed - Add case routing fields, retained access sources, durable handoff history, delivery leases, and replay-safe migrations. - Keep the same primary task across agent phases. Preserve each returning agent's own session and the task's workspace. - Fence outgoing mutations, wait for provider and workspace release, and recover delivery without duplicate task comments or runs. Recheck current assignment permissions as the original caller before applying a delayed transfer. Scope a run Stop to that run’s own handoff intents, so repeated old Stop requests preserve newer work. Cancelling terminal intent without a source run also requires fenced current task ownership, including native retries and preparation with an old checkout. For Cloud callers, revalidate current stack membership with an uncached Cloud RPC and derive owner elevation from the current managed setting. - Keep case reads, documents, assets, Audit, and live events inside retained task/project permissions. - Add native case creation/read/update/handoff tools. Store required workflow text and initial documents on the first task. Page oversized case facts and referenced document bodies through the dedicated tool with bounded UTF-8 JSON, current routing versions, snapshot conflicts, and current source authorization on every page. - Add read-only case lists and detail pages, task-style list controls, compact dotted field tables, linked handoff tasks, and case filters in Audit. - Add native/legacy integration tests, migration/restart tests, browser coverage, and bounded live receipt fixtures. Extend the trusted sandbox condition to cover the new subscription fixture profile. - Hide obsolete paused-agent warnings on completed or cancelled task threads. - Synchronize the company-switch settings test with React effects after a CI failure. Preserve its security assertions and production remount behavior. ## Verification The branch directly includes master `9ef434fe79`. The required `ci / verify` and `ci / e2e` gates pass on head `765fccd67c`. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/39464348614) passes typecheck, build, tests, and all eight browser shards. Security checks pass. [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16959#issuecomment-6291156549), with no unresolved review threads. The optional Paperclip Review check remains queued because its assigned agent is paused. It is not a required branch check. No check result was overridden. Local workspace typecheck and build pass after the master merge, along with database migration checks, token gates, map checks, Storybook catalog checks, and the Copilot profile identity check. Standard builds refresh Runner vendor output. Focused checks pass 2,008 API/paging/context tests and 7 strict case protocol tests. The final commit changes one test fixture only; its ten tests, UI typecheck, and token gates pass. No production source changed in that commit. On current head `765fccd67c`, all **194 critical workflow tests** executed and passed with no skipped tests in those files: - 38 Stop/ownership tests, including board completion intent, native retries, stale Stop, and preparation with an old checkout. - 2 actual legacy process/HTTP handoff tests, including scratch failure, a closed-stdio child that continues writing, and recovery through a fresh worker after the process group stops. - 5 migration tests, including SQL column metadata, preserved null authority context, and all six prior-numbering receipts. - 3 process-settlement and 5 scratch tests. - 88 handoff-service, 18 heartbeat handoff, and 21 native case-tool integration tests. - 7 bounded case/document-read projection tests and 7 strict protocol-contract tests. The current CI checkout was `fed65d49`, merging `765fccd67c` into master `9ef434fe79`. Its new document regressions reconstruct exact Unicode/escaped bodies and revisions through real native authority and authenticated HTTP with generic API tools disabled. They cover case-owned, primary-task, and linked-task documents; routing on every page; stale document snapshots; access loss and unlinking; unrelated/foreign/private sources; system documents; and quarantined content. Active-task document tools retain their existing behavior. These are current-head test results from the first CI attempt; the browser-only retry did not rerun these suites. The full `64fdd3403f` CI failed one separate settings test: after a company switch, it observed an unsettled API-key draft. The production component and test were unchanged from prior green source. Production switches remount both keyed settings layers; the test directly reused an unkeyed view and used a timer tick instead of waiting for React effects. The final commit replaces that fixture synchronization with React `act`. The exact secret-clearing assertions and all timeouts remain unchanged. The original assertion failure remains recorded; no unchanged retry was used. The narrow unchanged baseline and fixed fixture each pass 10 tests. The upstream result-paging merge preserves case capture namespaces and current asset privacy. Its API integration suite executed 39 of 41 tests successfully, with two skipped; the optional assigned-result PRP integration test was skipped. Neither skipped suite is counted in the 194 critical tests or as live provider proof. All 361 upstream migration entries and SQL bodies remain unchanged. Six case migrations follow master as 0363–0368 with identical SQL hashes. Official Drizzle generators produced the combined metadata and five retained snapshots. Generation reports no schema changes, and the snapshot drift test passes. The earlier `ce361ae5d0` CI passed on attempt 1, including its then-current 179 critical tests. Fresh review later identified unusable document references with generic API tools disabled; the bounded document reads and ten additional authenticated HTTP tests address that finding. The full local `pnpm test:run` attempt was cancelled after this Mac exhausted PostgreSQL semaphore capacity; it is **not** counted as a pass. Later database attempts also stopped before assertions or skipped, including one sandbox socket denial. Two original stale-Stop regressions did reproduce the bug against the old source. A REST serialization assertion passed after two preserved database-startup failures. No test timeout was increased. All 194 assertions listed above executed and passed in current-head CI. The complete 385-file, 3,963-test UI shard also passed after the fixture repair. A configured-provider receipt workflow passed 13 backend checks across triage → implementation → verification → correction → done, using one task and four work runs. The saved receipt is revision 2 with total 6 USD; case and task both closed. The original canonical attempt remains failed because its browser assertion confused the terminal destination with completed-task tooltips. The corrected full browser verifier passed on the same retained database and original runs, with no additional provider calls. The last clean-build browser readback is on `6a86c54a53`; later changes affect recovery, authorization, bounded reads, and settlement. Accounted work cost is a $1.010574 rate-card estimate; provider billing and setup-probe usage are unavailable. Copilot profile 49 passed the unchanged seven-fixture private protocol pack (46/46 checks, no retries) and the product file-edit canary after master's completion-replay change. All 60 bound implementation hashes remain unchanged on the current source. Historical profile-47/48 identities and results remain intact. See the [profile-49 compatibility record](https://github.com/paperclipai/paperclip/blob/codex/persistent-case-handoffs/doc/plans/2026-10-20-copilot-completion-recovery-compatibility-v49.json). This is bounded compatibility evidence, not a fresh full matrix, installed-package product test, or Daytona rollout. Preserved CI failures and retries: - On `20662930ac`, Slack managed setup failed a five-second success assertion after reload. Its test/application path was unchanged from a prior green head. One unchanged-source retry passed all 25 tests in that shard; no source or timeout changed. - On `9402b4f17a`, two CI workers received shutdown signals at 15:24:19 UTC. Neither log had an assertion failure before interruption. One unchanged-source retry of each passed all Runner static checks and all 32 tests in browser shard 6; both aggregates passed. The next head, `7b4374edaf`, passed all CI without retries. - On `295f289b5a`, serialized shard 9 timed out during its first device-login test, then the next test found an active login lease. Direct login/test paths were unchanged from a prior green head. Cold import plus a timed-out async body using the next test's mutable actor/store is the supported fixture-race explanation; no request trace proves that sequence. The original failure remains recorded. One unchanged-source retry passed all 294 tests in 19 files, including all 41 device-login tests. No login source or timeout changed. The subsequent `ce361ae5d0` CI passed without retries. - On `64fdd3403f`, one company-switch settings assertion observed an unsettled draft. The final test-only commit uses React `act` without changing assertions, timeouts, or production code. On `765fccd67c`, all ten settings tests and the complete 385-file, 3,963-test UI shard passed. - On `765fccd67c`, browser shard 3 completed the split-handoff assertions, then cleanup received HTTP 500 from run cancellation after a one-second PostgreSQL row-lock timeout. The extra chat run was reporting completed work while teardown began. Cleanup stopped before restoring a global experiment flag, causing a later assertion to fail. Relevant production and test paths are unchanged from an earlier green commit. Stop's lock order is consistent, and no nested self-lock was found; the exact PostgreSQL blocker was not recorded. One unchanged-shard retry passed. The original failure, trace, and downstream flag failure remain recorded. No Stop implementation, cleanup assertion, or timeout changed. The retry supports intermittent cleanup contention; it does not identify the original blocking transaction. ## Risks - Local process transfers require host-owned PID/group absence. Missing or uncertain termination evidence keeps delivery pending, independently of scratch setup. - Handoff admission and task cancellation share execution ownership. New regression tests cover temporary holds, late source writes, and uncertain provider termination. - Retained private sources use an intersection of existing permissions. This intentionally fails closed when a source is missing or an existing task cannot preserve that boundary. - Migrations add tables and indexes. They preserve master history and can replay over prior feature installs. Migration 0368 adds nullable internal authorization provenance with no default or authority backfill. Historical transfers still awaiting release without this context fail closed; an operator must cancel them and request a new handoff. - Delayed Cloud handoffs depend on the current Cloud membership RPC. An unavailable, ambiguous, expired, or mismatched response holds the handoff for retry; it does not reuse cached authority or grant access. - Bounded reads reload and reauthorize the selected projection per page. Underlying work grows with the data size and number of pages. Changed data or routing requires restarting from offset zero without the old snapshot. - Phase names and workflow diagrams remain advisory. They do not satisfy review or approval gates. - This is a large, cross-cutting PR. The maintainer requested one PR for the whole feature; the commits separate data, runtime, UI, and qualification work. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and delegated code review. The exact deployment model ID and context-window size were not exposed in the session. Live fixture agents used the explicitly configured provider model recorded in their test evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9225796a4e |
fix(runner): page oversized assigned tool results (#16980)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents receive assigned app-tool results through the durable Runner protocol. > - The assigned-tool bridge returned complete provider responses without a size bound. > - A large response could exceed the encrypted command limit after the upstream call had completed. > - The provider then waited for a result that the controller could not deliver. > - This pull request saves large governed responses and returns a small reference with a scoped page reader. > - Agents can retrieve the full result without repeating the original action. ## Linked Issues or Issue Description **What happened?** Large app-tool responses could leave native tool calls pending even though the upstream service completed. The durable command limit is about 510 KiB. The assigned-tool bridge did not account for that limit. A 720,048-byte regression response failed the delivery-size assertion before this change. **Expected behavior** Deliver a bounded response and a way to retrieve all saved content. Keep the original action at one execution. Preserve task, agent, company, responsible-user, and current-run access checks. **Steps to reproduce** 1. Run a native agent with an assigned tool that returns more than 510 KiB of JSON. 2. Let the upstream tool complete. 3. Observe that the original bridge returns the oversized object directly, so the durable controller cannot queue its result. Related: #14734 preserves uncertain tool outcomes across shutdown and restart. This change keeps that protection. #16517 handles native MCP content projection on a different transport; it does not bound native assigned-tool results. ## What Changed - Save governed assigned-tool responses above 500,000 encoded bytes using the existing response asset storage, quotas, and audit path. - Add `read_tool_result` for UTF-8 JSON pages from the same task and agent. Each encoded page is capped at 500,000 bytes, including JSON escaping and metadata, with remaining content available through the next byte offset. It works when generic API tools are disabled. - Reuse existing authenticated asset reads and responsible-user checks. Retrieval does not invoke the upstream service. - Preserve uncertain delivery if storage fails after an action completes. - Add regression, fault, authorization, and opt-in live Codex checks. Document the transport and access contract. ## Verification - Red: the original integration regression returned 720,048 bytes against unchanged production code and failed its bounded-response assertion. The follow-up regression also showed that a 480 KB result was unnecessarily saved under the original 24 KiB policy. - Green: results through 500,000 encoded bytes stay inline. Larger responses are saved in full. Real HTTP integration tests reconstruct the complete Unicode result after controller recreation in exactly two pages, in both standard and low-trust configurations, with one upstream execution. - The encoded-size boundary and escape-heavy Unicode tests verify that every page fits 500,000 bytes, cursors advance on valid UTF-8 boundaries, and concatenating pages exactly reconstructs the original JSON. Storage-failure tests preserve an uncertain outcome without repeating the action. - The focused unit/API run passes 1,957 tests; the changed real HTTP tests pass. `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` pass. - Real Codex acceptance passes with `gpt-6.1-sol` through Codex CLI 0.160.0 and native Runner, on macOS ARM64 with `low_trust_review` and generic API tools disabled. Codex requests every page in order, receives all 660,098 original bytes, and returns the final verification field. The two encoded pages are 500,000 and 160,543 bytes; the upstream operation executes once. Runner closes successfully in a 34.1-second probe. Only the disposable database's connection/worker counts were reduced to fit the host's remaining semaphore capacity. - The native-PRP regression also passes with escape-heavy Unicode pages over 1 MB in total and a separate inline result containing 200,000 backslashes. A scripted provider verifies both complete results by SHA-256. It proves the intermediate text wrapper can exceed 700 KB because the native transport decodes it before queuing the bounded PRP result; every result object stays within 500,000 encoded bytes. Native binaries are required for the local run, preventing a silent skip. - A second real-Codex probe with escaped Unicode, quotes, backslashes, newlines, and NULs also passes: every original byte of a 720,098-byte JSON response is retrieved in two encoded pages of 500,000 and 460,543 bytes. The model returns the final field, the upstream call executes once, and Runner closes in 32.6 seconds. This uses the same production source with only the synthetic fixture content varied. - The live probe exercises real PRP, persistence, and authenticated asset reads. It does not claim the original staging task resumed or full heartbeat checkpoint acceptance. - Earlier broader local testing passed 103 tests but could not start the second database for the existing `rejects a stale source-run question over public HTTP and accepts it from the next run` test. Extending its timeout exposed PostgreSQL initialization failure on this semaphore-exhausted host. The earlier local `pnpm test:run` attempt was interrupted and is not claimed as passing. Full CI passes on current head `cd32237ee5429765b8f349402ac3c79cceec3e6b`. GitHub reports 55 successful checks, two skipped Storybook checks, and one neutral Paperclip agent review. Greptile is 5/5 on this head with no review threads. - Additional Paperclip agent review is incomplete: it withdrew the earlier double-escaping finding after inspecting the native regression, found no new actionable defects, and reported stalled large-source reads prevented independent inspection of the transport decoder. Its neutral assessment is [recorded here](https://github.com/paperclipai/paperclip/pull/16980#issuecomment-6295579705). This limitation remains separate from the passing local native transport, real Codex acceptance, full CI, and Greptile results. ## Risks - Large responses now consume existing response-storage quota and require page reads. Larger pages use more provider context. Results are preserved in full; each encoded page leaves about 22 KiB below the PRP command limit for the command envelope. - A storage failure after execution retains an uncertain delivery outcome. The bridge must not report that the upstream action failed or invite a replay. - The dedicated reader grants access only to saved responses for the current task and agent. The asset route also checks the original responsible user. - Runner command limits and receipt rules remain unchanged. No migration or separate Runner protocol rollout is required. - This PR does not deploy the repair or resume an existing blocked task. ## Model Used OpenAI Codex (GPT-6), with reasoning, tool use, and code execution. The exact serving model ID and context-window size are not exposed in this session. The opt-in acceptance probe requests `gpt-6.1-sol` through Codex CLI 0.160.0. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the changed behavior (focused unit/API, real HTTP, and real Codex acceptance); the broader local database limitation is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
98d17da1e2 |
Fix native monitor dispatch and finalization recovery (#16975)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs persist their results and workspace before they release execution. > - A task monitor must wait for that settlement before it starts the next run. > - The scheduler checked provider status but did not check unfinished finalization. > - A timer could dispatch too early, then its old finalizer could no longer find the consumed schedule. > - This pull request delays dispatch and reconciles prior delivery from server records. > - The task can continue without repeating the provider turn or removing a newer run's ownership. ## Linked Issues or Issue Description **What happened?** A native run saved a yielded monitor result while workspace finalization needed recovery. Its timer dispatched after the provider stopped. The follow-up failed because the earlier run still held workspace recovery. The old finalizer then retried `monitor_wait_authority_lost` indefinitely. **Expected behavior** Timer dispatch waits for native finalization. If an older timer already dispatched, the finalizer uses its saved scheduling receipt and admitted successor to settle the old result. It preserves current task state and ownership. **Steps to reproduce** 1. Schedule a task monitor from a native run and save a yielded result. 2. Leave native finalization retryable after the provider reaches a terminal state. 3. Let the monitor become due. Before this change, the scheduler consumes it. 4. Restore workspace persistence and retry the old finalizer. Before this change, the missing schedule leaves it retryable. **Paperclip version or commit** Reproduced against master `34be1b831bcaba070264ae8d214ceffe6a8f9eab`. **Deployment mode** Self-hosted server built from source, with PostgreSQL and the native runner. Related recent recovery work: #16962 and #16963. No duplicate repair found for this failure. ## What Changed - Scheduled and manual monitor dispatch wait for unfinished native finalization. If the monitor expires during workspace recovery, the accepted result can settle into the persisted schedule, and the scheduler then applies its existing expiry policy. New monitor-wait results still require an unexpired, unexhausted schedule. - A narrow reader joins the run's server-issued monitor receipt to its durable wake and admitted successor. - The finalizer and locked status commit verify that evidence and preserve current task ownership. They do not create another wake. - Monitor delivery participates in assessment identity, so recovery can replace a prior error decision without violating the decision ledger's uniqueness constraints. - PostgreSQL regressions cover saved-result replay, active successors, wrong task or agent, missing receipts, unadmitted wakes, ownership races, and failed workspace persistence. - The execution contract and verification maps describe the recovery boundary. ## Verification - Red: the saved monitor regression remained `retryable_failure` instead of `committed` on unpatched code. - Red: the scheduler admitted a wake while terminal-run finalization remained retryable. - Red/green: two PostgreSQL cases reproduce timeout during workspace recovery for scheduled and manual checks. Before the review repair, both remain `retryable_failure` with `monitor_wait_authority_lost`. Afterward, both commit, apply board escalation once, preserve the saved result, and enqueue zero provider wakes. New completion still rejects the expired schedule. - Typecheck: `pnpm -r typecheck` passed. - Build: `pnpm build` passed. - Maps: `node scripts/check-maps.mjs` passed. - Review: Greptile 5/5 and Ponytail Paperclip Review 5/5 on the current head, with no unresolved findings. Both reviews are static; execution evidence is recorded separately above and below. - Linux staging: the original repair passed 170 tests across monitor integration, scheduling, status arbitration, and workspace finalization. The expanded suite passes all 172 tests, including both new timeout regressions. - Live acceptance: deployed the original repair to the authorized Linux ARM64 self-hosted instance. The stranded saved result finalized automatically; the original task then completed through normal admission, with its final answer persisted after browser reload. The other original task was already complete and its external delivery was verified without repeating it. - The initial Mac regression run passed 68 tests, but three suites failed during embedded PostgreSQL startup. A subsequent focused run confirmed the host had exhausted its PostgreSQL semaphore allowance. Linux verification uses isolated databases; the local failure logs remain preserved. - The local full test suite is not claimed as passed. It is blocked by the same PostgreSQL startup limitation; full-suite CI passed on head `8bc29d1b7783446f397b858e558083570a96bb66` ([run](https://github.com/paperclipai/paperclip/actions/runs/39447913466)). ## Risks - A delivery match must not grant new execution authority. The lookup requires the original run's server receipt and a matching company, agent, task, wake, and admitted successor. The locked commit rejects old-run checkout ownership and permits no status effects. - Workspace persistence must succeed before settlement. Its existing failure path remains effective. - The scheduler waits while a native finalizer is unfinished. Workspace recovery must settle before dispatch; elapsed monitor limits transfer to the existing scheduler recovery policy instead of stranding the result. Neither settlement path renews a deadline or changes budget, approval, pause, or provider authority. - No schema, migration, credential, or provider configuration change. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
24758b9da1 |
Avoid loading tool schemas for effective assignments (#16969)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tool profiles define which tools an agent can use. > - Run preparation and connection checks resolve these assignments from the database. > - These callers need tool IDs and names, but the existing lookup also reads complete schemas and provider metadata. > - Repeated lookups transfer data that these callers discard. > - This pull request adds a small assignment projection through the same permission resolver. > - The benefit is less database transfer and JSON work with current permission checks and the same public response. ## Linked Issues or Issue Description **What existing behavior does this improve?** Effective tool assignment reads during run preparation, native runtime setup, and connection policy checks. **Subsystem affected** server/ — tool access and orchestration services. **Current behavior** Internal callers use the public effective-profile lookup. Its catalog query reads complete tool rows, including input schemas, output schemas, annotations, and provider metadata. These callers only use catalog IDs and names. **Proposed behavior** Internal callers use an assignment lookup that selects catalog identity and selector fields. Both lookups share one resolver. The public effective-profile endpoint keeps its complete definitions. **Reason and benefit** Large catalogs increase database transfer and JSON work for every assignment lookup. The smaller query avoids that payload without caching permissions or changing query frequency. **Breaking changes** None. No route, schema, migration, permission rule, or public response changes. Related: Refs #16761, Refs #16723. The former narrows profile summary reads. The latter changes how the gateway presents tool definitions. Neither narrows this internal effective-assignment lookup. ## What Changed - Share effective-profile resolution between full definitions and assignment records. - Select the existing catalog identity and selector projection for internal assignments. - Use the assignment lookup in run preparation, native runtime setup, and connection policy checks. - Add real PostgreSQL tests for selectors, exclusions, current profile and binding state, company boundaries, and returned payload size. - Document the database read contract. ## Verification - Five focused test files passed: 521 tests. They cover tool access, connection intents, runtime preparation, native runtime context, and the new assignment lookup. - `pnpm -r typecheck` passed. - `pnpm check:module-boundaries` passed. - The PostgreSQL fixture compares the full authorization result with the assignment result. It checks all selector types, excludes removed tools, and observes catalog, profile, and binding changes on later calls. - The payload test measures rows returned by PostgreSQL before application mapping. With large schema and annotation fields, the assignment result is less than 1% of the full catalog payload. This is a synthetic fixture result, not a fleet savings estimate. - `pnpm build` passed. - The full local suite is in progress. It found two Git fixture failures that also reproduce on unchanged master with `init.defaultBranch=main`. Both pass with a process-only `init.defaultBranch=master` setting. The global Git configuration was not changed. This earlier failure remains recorded; it is not counted as a clean first pass. - Apex review: 5/5, no actionable comments. All 54 CI checks passed; two conditional jobs were skipped. - The broader local run also reached two UI test timeouts and one adapter test timeout. All three passed when rerun in isolation with unchanged source. The exact cause of these timeouts is not established. No UI or adapter implementation is changed by this PR. The CLI import/export journey passes its assertions but its local teardown fails with `ENOTEMPTY` while a runtime context asset is written. Its CI check passed. The complete local suite is not reported as a clean pass. ## Risks - Both lookups share permission resolution. Existing tool access tests and result parity checks cover that extraction. - Future assignment callers must use the fields in the smaller result. Typechecking rejects attempts to read full schemas from it. - The lookup still scans the active company catalog and resolves current connections on every call. This change reduces returned bytes; it does not reduce query count or introduce a cache. - No database migration or operational configuration change is required. ## Model Used - OpenAI GPT-6 through Codex, with tool-assisted source inspection, implementation, and test execution. The exact model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4b05795712 |
fix(github): reuse review instructions across native turns (#16971)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub events continue an ordinary task and can reuse its provider conversation. > - Each automatic event appended the full review policy and PR description. > - Native GitHub follow-ups also repeated task history because only Slack used the compact path. > - This pull request separates admitted review configuration from commit updates. > - A verified resume sends the current messages and changed configuration. A fresh session gets full context. > - This reduces repeated input while retaining review rules and current tool checks. ## Linked Issues or Issue Description Refs #13959, #16645, #16901. **What happened?** Opening a PR and pushing twice stored three full review prompts. A later native turn repeated all three in a retained provider conversation. **Expected behavior** A new event supplies its commit and event data. The harness delivers stable review instructions once per retained conversation and configuration. Fresh sessions receive full instructions. **Steps to reproduce** 1. Enable automatic GitHub reviews for a repository. 2. Open a PR, let the review finish, and push two new commits. 3. Inspect task messages and the native submitted envelopes. Before this fix, the policy and old requests repeat. ## What Changed - Store compact automatic event messages. Fetch the PR description through the existing metadata tool when needed. - Load review instructions from processed deliveries bound to the company, task, agent, and source comments. Preserve each admitted policy snapshot. - Use the existing verified native continuation path for complete GitHub text batches. Deliver configuration changes using a key over endpoint, PR conversation, revision, and instructions. - Keep full bootstrap input for fresh sessions and failed resume, including board follow-ups with no new GitHub event. Retain existing framing for incomplete batches, attachments, interactions, and recovery. - Keep the same configuration identity across automatic review, discussion, and automatic review. Coalesced requests use the last accepted request's admitted policy. - Add regression and database coverage. Update the execution contract and verification recipe. ## Verification - Red: the new event-size regression failed because policy and PR body repeated. The GitHub continuation regression failed because it returned no compact prompt. - Green: focused policy, continuation, native input, and adapter prompt tests passed (231 tests). Native resume/fresh-bootstrap cases passed (9 tests). - The signed-webhook/database case passed at `6296168253`. It checks configuration edits, queued older snapshots, coalesced policy selection, discussion continuity, board follow-ups, and company/task/agent/comment boundaries. The private PostgreSQL template used 12 connections because the host reached its semaphore limit with the default settings. - The earlier broad channel/session integration run passed 1,432 tests. Later review fixes invalidated that run. The latest focused database case passed after those fixes. - Reviewer regressions failed against the previous lookup, then passed with the fix: a mixed-revision batch delivered multiple policies, and a discussion lost the configuration identity. The board wake-to-rendered-prompt regression also failed before repair. - `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` passed on the final source. - Full local `pnpm test:run` exposed follow-up/fixture failures during development. Those cases are fixed and pass separately. The stale run was stopped after source edits. Full-suite validation for the final commit passed in CI. - All required CI checks passed for `6296168253`, including tests, typechecks, build, and clean public npm install checks. The two unrelated Storybook checks were skipped. - A live two-turn Codex smoke could not start: the local macOS sandbox helper rejected execution of the installed Codex 0.153.4 binary. No live GitHub/provider acceptance or deployment is claimed. ## Risks Skipping instructions is safe only after a verified checkpoint resume with the correct previous run. The full bootstrap remains available if resume fails. Changed or previously unknown configuration is sent again. Tests cover these cases. Tool authorization and publication checks still run under the current server policy. No database migration is required. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact deployment model identifier and context window are not exposed in this session. No delegated agents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
1f3695ac67 |
fix: expire native session failover backups after their issues close (#16866)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native agent sessions that run in a remote sandbox keep a host-side
failover copy of the sandbox's harness state, so a replacement sandbox
can resume the conversation
> - Each session keeps that copy forever, and one copy can reach several
gigabytes
> - On a busy instance these copies fill the volume, and no existing
cleanup touches them
> - Once a session's work is closed, its copy matters only if someone
reopens that work later
> - This pull request expires the copy after the session's issues have
been closed for a configurable window, and makes a reopened issue start
a fresh session
> - The benefit is bounded disk use without breaking active,
recoverable, or leased sessions
## Linked Issues or Issue Description
Bug report described in-PR (bug report template).
**What happened?**
After every remote turn, `checkpointRemoteRunner` writes the sandbox's
runner state and provider session home (for example `codex-home`) to
`durable-sessions/<scope>/failover-backups/current`. Only the same
session's next checkpoint replaces it. No code path deletes a session's
copy when the session ends, its issue closes, or its agent goes away, so
the instance volume fills. On one instance:
- 246 sessions held 17 GB of copies, 84 of them older than a week;
- 24 new sessions added 5 GB in a single day, one copy alone 3.2 GB.
**Expected behavior**
Disk used by failover copies stays bounded. A copy is kept while its
session can still resume (open issue, live run, pending recovery, or
live lease), and expires once the session's issues have been closed for
a while. A reopened issue then starts a fresh native session, instead of
failing with `runner_harness_backup_unavailable` on a replacement
sandbox.
**Steps to reproduce**
1. Run agents with a native runtime in a remote sandbox environment.
2. Let them complete many issues.
3. Watch
`<instance>/runtime/paperclip-runner/durable-sessions/*/failover-backups/current`
grow without bound after the issues close.
**Paperclip version or commit:** `master` at
|
||
|
DottaandPaperclip
|
813e808af4 |
Remove filesystem history authorization from workspace preparation (#16963)
Remove physical-root source/reader authorization and sticky directory classifications from workspace preparation. Preserve current task/run authority, app-record privacy, credential authorization, containment, and lifecycle/publication integrity.
Validate replacement selections against persisted new inputs and refresh verified checkout ownership so bounded reuse survives earlier workspace relocation. Add real PostgreSQL/Git regressions, a reproducible history-scaling benchmark, and documentation.
Full CI passed all 48 jobs and Greptile scored 5/5 at
|
||
|
DottaandPaperclip
|
92b238137b |
Connect and configure GitHub Copilot through Apps (#15705)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps and AI Connections store credentials and govern access to agent runtimes. > - Users need a supported saved-token connection and config values that survive save and reopen. > - The release must preserve saved-token access and task lifecycle guarantees. > - This pull request will connect Copilot through the normal Apps and AI Connections path. > - The benefit is a reviewable part of the Copilot production release. ## Linked Issues or Issue Description Refs #15701 and #15703. Supersedes setup work in #15562. Base: #15703. Next: #15706. ## What Changed - Require configuration update authority before Copilot credential-bearing model discovery; default to the saved environment and preserve explicit selection for editors. - Keep Copilot discovery passive during reconnect reconciliation; refresh on explicit opening, account/environment changes and reconnect. - Add GitHub AI token connections with encrypted storage, ownership, grants and audit logging. - Keep repository access separate from Copilot runtime authentication. - Probe the selected environment without sending a model prompt and preserve authenticated model selection through save/reopen. - Keep fresh execution admission consistent with normal Copilot setup; persist task model choices without changing provider or permission policy. - Refresh authenticated models after reconnecting the same saved Copilot account, including recovery from a previous authentication error. - Apply the additive provider-default constraint migration through the normal generated migration workflow. ## Verification Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`. History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`. Remaining source [`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d) contains the four-file authorization repair. The remaining PR trees preserve six nonoverlapping files from the newer public mainline managed-settings fix and the CI-owned lockfile. No unrelated provider is enabled. The security finding is repaired: existing-agent discovery requires `agent_config:update` before connection selection or token resolution. The saved environment is the default. Authorized editors can select a prospective environment. Both denied board-viewer and agent-reader regressions fail on the original source. All49 route tests and server typecheck pass with the fix. Denied requests perform no connection selection, credential resolution or probe. Current-head security scans pass on both remaining PRs; the repaired thread is resolved. Copilot CLI1.0.88 and profile47 retain digest `sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`. The [qualification record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json) reports20/20 Product cells,154 assertions and7/7 protocol cases,46 assertions against frozen normal installed source `c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification model is `gpt-5.6-luna`. Every passing Product case confirms cleanup. Original failures remain recorded; automatic behavior retries are disabled. The route-only authorization repair changes no bound runtime/profile bytes and does not relabel the frozen campaign. The ordinary installed remote journey passed saved-token selection, model discovery, normal UI save/reopen, metadata-only Test, completed task, accepted receipt and exact19-byte browser download. An open model catalog caused no additional discovery requests during117seconds. Actual installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta and native Linux x64 on Daytona. Raw token browser entry was not exercised; encrypted connection creation used the normal API. The first Daytona file attempt timed out during delayed semantic dispatch; its precise stalled layer remains unresolved. Independent ordinary remote delivery and the other19 Product cells passed before one instrumented recheck passed8/8 with unchanged source,prompt,assertions and deadlines. No causal latency repair is claimed. Before the authorization repair, all four final PR heads passed required verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads. Current-head CI/review is running for the two repaired heads. Local focused checks retain129 connection,59 runner,72 shared-eval and88 recovery/workspace passes, plus migration safety and token gates. The additive Copilot migration is361; released master history is preserved. [Cloud normal-package run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704) builds the pre-security source4f3f. All four native producers and Linux package assembly pass; the final clean installed consumers are finishing. All18 retained archives match their hashes;37,863 regular files contain zero selected credentials. That package does not contain the later route repair. The actual final published package must include it and pass the local/Daytona live canaries. The frozen Daytona image remains published and independently verified. No separate full-image scan is claimed. Docker builds run in the cloud. **Remaining:** current-head CI/review, setup and qualification merges, final publication, and ordinary local/Daytona canaries against the actual shipped package. The user explicitly authorizes the maintainer review override; CI remains a merge gate. No production deployment is claimed. Provider USD remains unknown and GitHub-attributed. ## Risks Bad credentials, missing entitlement, runtime installation and unavailable models must remain distinct failures. A connection must not adopt ambient credentials. The constraint migration changes accepted provider values, with no credential-storage migration. Current-master NewAgent fixture extraction is retained. Copilot cases use that shared support and test both passive discovery and explicit reconnect under the same setup lifecycle. The onboarding retry regression shares the component suite’s saved-connection, authenticated-model, draft, agent-list, hire and render lifecycle fixtures. Keeping it beside the existing onboarding failures makes adoption versus duplicate hire observable under the same lifecycle; this is the concrete cohesion reason for the existing large test file. ## Model Used OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this task. Used for code changes, review and tool-assisted verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b448710986 |
fix: derive startup warmth from per-layer reuse evidence (#16908)
## Thinking Path > - Paperclip helps people manage AI agents and inspect their work. > - Agent startup can reuse a lease, workspace, Runner, process, or session. > - A session ID can remain the same when other resources are new. > - The old cold/warm label did not show these separate decisions. > - This pull request records closed reuse receipts and derives labels from them. > - Reports now show partial reuse and missing evidence. ## Linked Issues or Issue Description Refs #11685. That issue tracks a specific ACPX fingerprint repair. This change records reuse evidence across startup resources. It does not repeat that repair. **What existing behavior does this improve?** Startup labels in the operator trace and Product E2E reports. **Current behavior** A cached runtime or a continuous session can suggest a warm startup even when workspace staging or provider launch work occurs. **Proposed behavior** Record each layer's reuse decision and a closed reason. Record observed provider launch counts. Use `warm`, `cold`, `partial_reuse`, or `unknown` from those receipts. Keep old runs unknown when they have no receipts. ## What Changed - Add local run-log receipts for the lease, workspace, Runner, provider process, and provider session. - Count physical ACPX launches, including respawns. Preserve the reason for a new process. - Reuse native workspace adoption and session lifecycle evidence. Keep native and bridged remote provider-process reuse and launch counts unknown where physical evidence is absent. - Show the per-run summary in retained results and the dashboard. Keep missing legacy values explicit. - Set the legacy trace boolean only for fully evidenced cold or warm startup. Keep the operator OTLP gate. Add no first-party Telemetry. - Test full reuse, workspace invalidation, two direct provider launches with session continuity, and report generation. - Start the final diagnostic write without waiting for persistence. Keep lease cleanup independent of a stalled event sink. - Keep remote provider evidence unknown after runtime setup fails or the host relay starts twice. - Align a stale Pi support-test expectation with current master profile 28 and its suite fingerprint. Keep the qualification rules unchanged. ## Verification - Passed: all 319 runtime, startup-timing, reuse-summary, and native workspace-history tests. Disposable local Git tests used the native Git binary. - Passed on the current commit: all 2,768 Runner E2E support tests across 128 files. This includes report and history tests for generated HTML and normalized JSON. - Passed: `pnpm --filter @paperclipai/adapter-utils typecheck`. - Passed on `df60f8dbdbfcb6d80c84cd3bf563f8d58b2f9f03`: all PR CI gates, including repository typecheck and build, all test shards, and browser tests. Both automated reviews are 5/5. GitHub reports 55 successful checks, one neutral review, and two skipped Storybook checks. - The first server shard run failed because an unchanged workspace-runtime fixture did not accept health connections within 10 seconds. Its isolated local test and the single failed-jobs rerun both pass. This establishes recovery; the cause of the first timeout remains unknown. - Passed in the hosted source run on this exact commit: recursive typecheck, token gates, the large Runner E2E typecheck, Runner E2E unit tests, and build. The retained `source.txt` matches this PR head. - The extra unsharded `pnpm test:run` reached its 60-minute limit (exit 124). It has no complete verdict. The broad workflow therefore reports failure. No assertion-failure marker appears in the retained partial log. Full-suite acceptance uses the passing canonical PR CI partitions. A read-only inventory check found no gaps across 1,023 general server files, 161 serialized suites, and 20 workspace projects. The separate chat/native Runner lanes cover the excluded server files. No tests or timeouts were weakened. The local environment has 4 GiB of memory. Use CI for the large Runner E2E typecheck. - The initial report tests failed because the harness could not resolve package imports from the repository root. Relative source imports fixed that failure. - The first broad local run had timeouts from the environment Git bridge. Native Git resolves the ancestry timeout and all runtime tests pass. The added report fixture lacked its required screenshot; the corrected report tests pass. A stale master assertion expected Pi profile 27, while the generated profile is 28. Updated its expectation and exact suite fingerprint; all 78 targeted support tests pass. The clean full support run passes all 2,768 tests. - The three review regression cases failed before repair. They showed remote counts of zero and two, and a completed run that waited for the final diagnostic sink. They now pass. The stalled-sink case keeps persistence pending while the run returns, both bridge stops run once, and the staging lock map becomes empty. - Hosted evidence for this commit: [source verification](https://github.com/paperclipai/paperclip/actions/runs/39335475416), [retained command logs and status table](https://github.com/paperclipai/paperclip/actions/runs/39335475416/artifacts/12228829282). All listed commands exited zero except the incomplete unsharded test command described above. - The physical-launch test drives the executor's spawn callback twice. The native workspace test uses real Git repositories and verifies reuse and invalidation. These are deterministic tests, not a paid provider qualification campaign. ## Risks - Consumers must handle `partial_reuse` and `unknown`. Partial evidence no longer produces a legacy cold/warm boolean. - Native and remote bridged provider process evidence is still unavailable on some paths. The report shows unknown values for those paths. - Receipt emission is best effort. Missing receipts reduce confidence in a label and must not change execution or settlement. - No schema migration, first-party Telemetry event, or endpoint default changes are included. ## Model Used OpenAI Codex, based on GPT-6, used reasoning, repository tools, and code execution. The exact provider model ID and context window are not exposed to this session. The existing implementation checkpoint was preserved from the earlier agent run. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> |
||
|
DottaandPaperclip
|
bc091f0a21 |
fix(recovery): recover retained workspaces before saved messages (#16906)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A stopped run can leave useful files in its original sandbox. > - The server keeps that sandbox when it cannot confirm file copyback. > - The task can then save new user input but offers no safe way to recover the files. > - A retry of the old agent turn can repeat work that already finished. > - This pull request recovers files through the existing workspace owner and delivers only saved human input. > - The user can recover a task without certifying infrastructure facts or losing completed work. ## Linked Issues or Issue Description Refs #16883 and #16835 for publication receipts and verified process termination. Related: #16478 clears stale holds after copyback already succeeded. This change performs the missing export and records its proof. #16901 manages the main GitHub response comment and does not perform workspace repair. **What happened?** A failed run retained its workspace. A later user message stayed saved behind a file-repair hold. The task asked the user to verify process termination and file recovery but offered no repair control. Sending a message could show an optimistic Working state and move the task to Todo before admission succeeded. **Expected behavior** The harness should recover the original files when it can prove ownership and termination. It should preserve newer edits and completed operations. Only waiting user input may start another turn. Missing evidence must keep the source held with a specific explanation. **Steps to reproduce** 1. Interrupt a sandbox run before workspace finalization succeeds. 2. Retain its exact allocation and save a user follow-up. 3. Observe the file-repair hold and missing end-user repair path. ## What Changed - Add source-bound background recovery for native and legacy runs. An accepted native result is not required. - Share the physical copyback lock and durable owner. Resume only the exact stopped allocation, export files, verify persistence and obtain a fresh stop receipt. - Persist retry limits and idempotent requests on the existing recovery record. Limit each server to two background exports. Add an indexed candidate scan and a separate cleanup obligation. - Add a task-editor recovery endpoint. Deny agents and viewers. Reject caller-supplied stop or repair claims. - Deliver waiting human input once through ordinary admission. Preserve pause, approval, budget and assignment gates. Require historical conversational-adapter evidence for legacy-to-Runner continuation. - Include exact GitHub publication receipts in continuation context. Preserve unknown effects and the original failed run. - Show observed recovery progress, a transient-failure retry action and specific service blockers. Keep saved messages visible. Suppress false startup feedback and implicit Todo promotion behind a file hold. Hide unavailable workspace links. - Update execution semantics, capability maps and production-component stories. ## Verification - Red: the original read model failed the new recovery-progress assertion; the other 23 checks in that file passed. - Passed: recovery regressions with 28 cases with real PostgreSQL, files, tar and Git operations. Lifecycle provider calls are controlled test boundaries. Coverage includes native/legacy export, concurrent owners, rollback after export, interrupted archive export with a new database connection, lost replies, historical discovery, durable retry limits, separate cleanup and saved-input delivery. The actual candidate query uses its partial index with 3,000 unrelated history rows. A first automatic repair can bind verified old-controller stop proof without an accepted native result. Review also reproduced and fixed export-only continuation and cleanup lost after a committed repair. Cleanup remains discoverable without waiting input and after task reconciliation. - Passed: two GitHub integration cases verify that continuation context carries the existing publication receipt and does not write another response. Company and task boundaries are checked. - Passed: 293 continuation, workspace-sync, platform-receipt and OpenAPI checks. After file repair, actual dispatch retains pause, cancellation, approval, real budget-policy and assignment gates. - Passed: 174 route checks, including editor admission, agent/viewer denial and rejection of fabricated repair claims. - Passed: 351 task detail, workspace properties and comment-reopen checks after updating the new read boundaries and asynchronous workspace-link expectations. Two new route cases keep held tasks Blocked when a comment is saved. - Passed: 265 task-thread and recovery-notice checks. The broader continuation suite also passed; its new legacy migration case first exposed a gap that the focused recovery suite now covers. - Passed after rebase: `pnpm -r typecheck` and `pnpm build`. The later master merge adds an unrelated permission migration; recovery migration generation and safety checks passed after the merge. - Passed: token gates, module boundaries, map structure and migration checks. The recovery index is migration 0360 after current master. - Browser: inspected the production recovery component and its retry-to-queued transition with fixture data. The first dev dependency load failed; a reload after optimization resolved it. This is UI evidence, not a live provider recovery claim. - Every test shard passed on final commit `bbbc09efb4`. The older local `pnpm test:run` was stopped after the complete CI test matrix passed. Local full typecheck and production build passed. The first CI build compiled and booted, then lost its runner during the clean public npm-install check. An isolated retry passed the build, production boot and clean public npm-install verification. - Greptile reviewed final commit `bbbc09efb4` at 5/5. All review threads are resolved. - The change is not deployed. Hosted recovery remains unverified until the served revision includes this change. ## Risks - Recovery crosses process, provider and persistence boundaries. Missing or mismatched ownership, stop, descriptor or file evidence keeps the hold. It never selects a replacement allocation or silently starts over. - File recovery does not certify external action outcomes or authorize replay. New input still uses normal execution admission. - Historical sources without sufficient durable evidence can remain blocked. Destructive reconstruction stays separate. - The index migration is idempotent. It changes no task data. Snapshot pruning follows repository policy. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with code execution and browser tools. `gpt-6-astra` assisted earlier investigation. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ac9459bae3 |
feat: configure company agent observability (#16874)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Companies use several agent harnesses and execution environments. > - Operators need one place to select where native agent logs and traces go. > - An inference proxy would add a dependency to the agent request path. > - This pull request supplies native OpenTelemetry settings to managed Codex and Claude Code processes. > - Company secrets protect destination credentials. Export failure does not change the task result. ## Linked Issues or Issue Description **Subsystem affected** Company settings, shared contracts, database, server, adapters, and runner. **Problem or motivation** An operator cannot configure a company destination for native agent logs and traces. Manual harness configuration is hard to apply across local and remote execution. **Proposed solution** Add one company destination: Braintrust or a custom OTLP HTTP/protobuf endpoint. Apply a transient snapshot when execution preparation starts. Keep credentials out of API reads, task environment manifests, recovery files, and command arguments. Replace a warm process when its export settings change and preserve its conversation. **Alternatives considered** Multiple destinations can use a user-operated collector for fan-out. This change does not add a collector, an inference proxy, or transcript scraping. **Roadmap alignment** This extends company settings, secret management, and support for multiple harnesses. The maintainer requested this implementation. Searches for observability and Braintrust found no duplicate company destination feature. Related infrastructure tracing work in #16236 has a separate content-free contract. ## What Changed - Add strict schemas, revision checks, encrypted secret storage, activity records, and board-only settings routes. - Add explicit synthetic logs and traces tests with a bounded HTTP request and sanitized results. - Supply native Codex configuration and Claude Code environment values through local and remote launch paths. - Preserve Codex login refresh through the existing managed credential link. Retain unsaved drafts across background reads. Require complete custom header re-entry when the endpoint origin changes. - Separate process export compatibility from conversation identity. Isolate generated Codex configuration and preserve resumable sessions, including remote home refresh after export changes. - Advance the immutable Pi/Copilot source attestations for shared environment and sidecar code changes. Keep their provider payloads unchanged and retain prior identities for decoding. - Redact destination credentials throughout launch diagnostics and HTTP logs. - Add the production settings form, Storybook states and journey, documentation, and feature/access map entries. ## Verification - Real managed local Codex 0.160.0 and Claude Code 2.1.286 tasks exported to Braintrust. API readback and the Braintrust UI showed the test records. A real remote Daytona Codex task exported through the legacy `exec` path. - Live tests exercised content capture off, export disabled, warm conversation reuse, and a destination that accepted connections but never replied. Agent work succeeded and export requests stopped at the configured three-second timeout. The UI connection test returned “Test payload accepted”; the saved API key stayed masked after reload. - All current-head CI test lanes, typechecks, packaging/build checks, and required aggregates passed on `71e994b2be9c8fb6d6e938a4ad7cf4c95fdafe31` ([CI run](https://github.com/paperclipai/paperclip/actions/runs/39319194318)). Greptile completed on that exact commit at 5/5, with zero unresolved review threads. The additional optional Paperclip Review check remains queued waiting for its assigned agent. - Local workspace typecheck and build passed. Focused verification included 239 combined runner tests, 789 API/native-session tests, 107 Codex home/ACP/sandbox-site tests, 204 feature/privacy tests, and the affected Rust suites. Migration, token, map, Storybook catalog, and Storybook build checks passed during acceptance. These checks ran at the relevant implementation or merge boundary; the current-head CI is the complete final gate. - Managed AI credential projections now receive enabled and disabled company export settings; genuine caller-owned Codex homes remain untouched. Two regressions reproduced the skip before the repair. All 276 engine/lifecycle tests and adapter-utils typecheck pass afterward. The launched TOML retains provider/MCP settings, credential continuity, and redaction. - Settings saves, removal, and the real company-deletion owner acquire secret mutation ownership before database locks. PostgreSQL NOWAIT probes cover all three paths; concurrent save/rotation/removal/deletion requests settle correctly. The affected suites passed 137 tests, followed by all 32 service/connection cases after expanded deletion coverage. Server and Codex adapter typechecks pass. - Codex source profiles use the effective CLI argument list. A spawned fixture CLI reproduced both ignored-argument failures; all three precedence/fallback cases now pass. - Real tar staging reproduced remote conversation loss in both API-key and subscription modes before the repair. Both now pass: changing managed settings preserves sessions, archives, native indexes, and refreshed subscription credentials. A corrupt upload preserves the prior config and conversation. - Preserved failure evidence: initial CI caught stale Pi/Copilot source attestations, a missing test-harness helper, and packaging fixture/probe mismatches; those were corrected and covered by focused checks. Superseded heads had intermittent attachment-composer timeouts despite successful uploads; [the original trace and screenshot remain available](https://github.com/paperclipai/paperclip/actions/runs/39309701089/artifacts/12215109203). The same attachment case passed on the final head: [25/25 tests in shard 3](https://github.com/paperclipai/paperclip/actions/runs/39319194318/job/118288637051). - A connection-fault fixture also exposed a stale pooled socket after its hang case; closing fixture responses isolated scenarios and the full affected suite passed. - The broad local test run was not clean: database resource failures, timeouts, an older cached module graph, and a stale optional-resource manifest assumption were recorded. Focused checks passed after the documented fixes. This does not establish a clean full local pass. - Native remote Codex, remote Claude, and the full ACPX Codex bridge remain unqualified. The direct Codex resume test does not establish full bridge acceptance. ## Risks - Native exporters perform network requests from each execution environment. A server connection test does not prove remote reachability. - Prompts, assistant responses, and tool content are enabled by default when supported. Native caps and harness-specific controls still apply. - Caller-owned Codex homes are unsupported for automatic configuration and remain untouched. - Native harnesses and their tool processes can inspect export credentials. The settings form and documentation advise using ingestion-scoped credentials. Retained headers remain bound to their saved scheme, host, and port; changing origin requires fresh values or removal. - Migration 0357 adds a company-scoped row and secret reference. It is safe to reapply. Secret removal and company deletion use the existing secret lifecycle. - Slow or unavailable destinations must not block task settlement. Export settings take effect at the next execution preparation boundary. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and browser verification. The exact serving model ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
d747752496 |
fix(runner): record why an OpenCode turn fails when opencode serve stops (#16863)
When `opencode serve` stops during a turn, the OpenCode driver now fails the active turn itself with a closed-vocabulary cause (server exit code or known signal, allowlisted socket error code, HTTP status, or closed stream) instead of leaving runnerd to close it with `error: null`. The proxy flushes stdout before a fatal exit so the failed turn reaches runnerd, and the server records OpenCode failed turns as `nativeTurnFailure` (live and persisted-recovery paths) so the run-failure diagnostic names the cause. Provider text is never copied. Run status, retries and recovery behaviour do not change. Co-Authored-By: Paperclip Sentry triage <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
8193abb936 |
feat(claude): default qualified macOS and Linux targets to Paperclip Runner (#15997)
## Thinking Path > - Paperclip must run the selected agent on its selected host. > - Claude needs verified packages for each supported target. > - A Mac controller must select the correct Linux provider pack. > - New defaults must follow package and execution evidence. > - This PR adds Claude platform packaging and target-specific pack selection. > - Existing Codex Linux ARM64 behavior remains intact. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15993 and #15994. Related scope: #15422 and #15224. ## What Changed - Reuse the merged four-target daemon producer/consumer matrix. Add target-owned Linux provider packs and carry their immutable identities through release assembly and vendoring. - Reuse master’s landed pinned Claude SDK ARM64 executable admission and retain package-integrity, architecture and no-follow admission. - Select target-specific remote packs so Mac controllers can use Linux hosts without a development pack override. - Preserve read-only runtime context on SSH startup and resume without accepting unverified old content. - Check ordinary installed Claude readiness without injected package authority. Retain Codex and Linux x64 consumer checks. - Extend Claude automatic target selection for the remaining qualified package targets. Retain unsupported-platform and external-override behavior. - Qualify managed Daytona snapshot names through provider IDs during Claude Connect. Pin native acquisition to the measured ID. Preserve a revocation marker after failed probes or target edits, and block native acquisition until requalification instead of falling back to a mutable name. Preserve immutable-image proof, company and account boundaries, and Legacy acquisition. ## Verification - Reconciled with master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through the prerequisite stack, retaining the merged startup fix and latest Codex Linux ARM64 behavior. - [Four-target exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/39267321383) passed on source `04e0202ca81387b17cbe7fd094a8338648356fe9`: all four native builds, public-package assembly, Linux x64 verification and clean installed consumers on Apple Silicon, Intel Mac and Linux ARM64. The later master reconciliation adds shared OpenRouter attribution and changes runtime source. This historical result keeps its original source identity; [later qualification](https://github.com/paperclipai/paperclip/actions/runs/39281100929) passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. It predates the subsequent TypeScript startup fix. - Fresh local product acceptance on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` completed a useful file and follow-up through native Claude/Opus 5.5, High effort and the API-key connection. Both actual browser downloads were independently checked; the revision preserves the original body and session continuity. Total task estimates: $0.518432800. Ordinary web and API creation saved defaults with the native experiment disabled. - Local focused verification passed 993 startup/session tests, 243 UI tests, 12 workflow security tests, direct server/UI TypeScript checks, and token/palette gates before the last review changes. Initial DB-backed snapshot pin/revocation coverage passed 18 cases; the final serial route suites pass 122 credential/permission and 232 snapshot/environment cases after correcting a test hook timeout. Current-head cloud suites pass. - Preserved the failed pre-provider local fixture and its full pending reservation. The successful fixture has no unpriced or pending usage. Historical failed Cloud journeys, accounting holds, source identities and subscription limitations remain in the dated plan. - All Docker/Rust release production was performed in cloud CI. No local Docker image was built. An inadvertently invoked local Rust prerequisite was stopped. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - **Experience needs improvement:** successful connection setup still shows “Connection needs attention” because runtime checks are deferred. Cold provider-pack upload took 122–126 seconds per run while the task mainly showed “Thinking.” The tested Core-only preview excludes private Cloud UI plugins; this is not full Cloud distribution or subscription qualification. Desktop dark theme was tested; mobile and comprehensive accessibility were not. The portable report validates, but gallery interaction could not be inspected because host preview was queued. - The optional supply-chain scan returned neutral/inconclusive because GitHub Actions package coverage and infrastructure parsing are unavailable. It is not counted as a successful supply-chain qualification. Exact package, daemon and image integrity checks passed. - A stale, missing or wrong-architecture pack must fail admission. It must not change runner or target silently. - Provider-pack source changes invalidate affected earlier execution results. Existing unknown-cost and cleanup holds remain intact. - Live subscription execution remains unverified under the API-key qualification direction. Built-in authentication flows are retained. ARM64 cloud-image production is deferred. Merge and production deployment remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
45b67f1d93 |
feat: default new Apple Silicon Claude agents to Paperclip Runner (#15993)
## Thinking Path > - Paperclip lets users choose an agent harness and account. > - The server resolves the runner for new agents. > - Claude still needs the same ordinary native-default journey as Codex. > - Existing agents must retain their runner, model and account. > - This PR defaults new Apple Silicon Claude agents to Paperclip Runner with Opus 5.5. > - Users can connect and work without a runner override. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15994 and precedes #15997. Related scope: #15422, #13978 and #15224. ## What Changed - Resolve new Apple Silicon Claude agents to `paperclip_runner`, `provider: acpx`, `acpxAgent: claude`. Keep explicit Legacy, models, provider routing and saved execution. - Use `claude-opus-5-5` for ordinary Claude defaults and retain explicit overrides. - Reuse existing API-key/subscription connections, credential storage and checks. Generalize the shared Advanced runner control with Claude branding. - Carry configured effort into the shared v9 contract. Preserve compatible permissions, instructions, skills, timeouts and session behavior. Require Legacy for unsupported settings. - Qualify a new immutable managed Claude target at explicit Connect through the existing permission-gated probe. Use fresh scoped discovery for test and hire. Discard stale results and require an explicit Connect after settings change. - Retain merged Codex onboarding recovery, auth preparation, import/effort corrections and Grok Bot gates. - Check the Claude first-task wizard's saved default directly. Update configuration/runtime documentation and exact Opus 5.5 accounting. - Align hire fixtures with the merged host-environment restriction. Native agent-authenticated hires cannot inject host env settings; both creation endpoints verify that no extra agent is saved. ## Verification - Updated onto master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through #15994. Native input v9 carries effort while preserving v7 home modes and v8 Muse. Codex defaults remain unchanged. - Focused local verification passed 285 server tests, 323 UI tests, 124 setup/registry tests, and two v9 permission-pin transport cases. Direct server/UI TypeScript checks passed. An initial 30-second hook timeout was resolved by serial verification with a longer hook budget: all 122 credential/permission cases and 232 snapshot/environment cases pass. Direct server TypeScript checking also passes. - Ordinary web creation with the experiment disabled saved native Claude, Opus 5.5 and High effort without runner/model overrides. The actual product task and follow-up both succeeded on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` with the cloud-built daemon from `04e0202ca81387b17cbe7fd094a8338648356fe9`. The final master reconciliation also changes shared runtime source. Historical proof retains its source identity; a final-source cloud qualification is running before refreshing the preview daemon. - Both actual browser downloads were checked: v1 has three sections/six checkboxes (451 bytes); v2 preserves the original body and adds two Rollback checks (612 bytes). The original attachment stayed byte-identical. The follow-up records session resume and the same provider session. Total estimated API usage: $0.518432800, with no pending/unpriced usage in the successful fixture. - Fixed the live-discovered setup incompatibility: native API-key validation no longer depends on the global Legacy Claude CLI. The existing provider endpoint verifies the key without a paid prompt; subscription/custom routing probes remain intact. - Fixed security review's native-hire credential redirection: agent-supplied URLs, proxy/launch settings and other host environment controls are rejected before inheritance. Native credential references remain supported and partial edits use the saved harness. Added create/hire/update regression coverage. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - Docker and native release builds ran in cloud infrastructure. An inadvertent local Rust prerequisite was stopped; no local Docker build was run. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - Missing packages, authentication rejection, unsupported settings and unavailable models must fail explicitly. Existing agents retain saved execution; explicit runner changes use existing revision/session invalidation. - Subscription follow-up remains unverified; remaining live acceptance uses API keys per human steering. Historical accounting holds and the separate $50 ceiling remain intact. - The subscription acceptance limit remains explicit for review. No Claude database migration is introduced. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
36e4c6657f |
fix(sentry): report a native retry failure only when the retry ends (#16865)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server sends a Sentry `run_failure` event when an agent run
reaches a terminal failure.
> - A native run that waits for a same-run retry is stored as `failed`
with `nativePhase: "retryable_failure"`. That state is not terminal.
> - `reportRunFailure` reported that pending state. Runs that later
recovered, for example after a workspace copy-back retry, still sent a
run failure event.
> - These events make one catch-all Sentry group noisy and hide the real
terminal failures in it.
> - This pull request skips the pending retry state and reports once
when the retry ends as a terminal failure.
> - The benefit is that run failure events match runs that really
failed.
## Linked Issues or Issue Description
**What happened?**
Two staging runs had a workspace copy-back failure during settlement
(`Daytona syncOut tar command failed ... No space left on device`). The
executor stored each run as `failed` with `nativePhase:
"retryable_failure"` and `errorCode: native_session_interrupted`. Sentry
received a `run_failure` event for each run. About two minutes later,
the copy-back retry succeeded and both runs finished as `succeeded` with
`workspaceFinalizeStatus: "succeeded"`.
**Expected behavior**
A run that waits for a same-run retry must not send a terminal run
failure event. If the retry ends as a terminal failure, the server must
send one event at that time.
**Steps to reproduce**
1. Set `PAPERCLIP_SENTRY_RUN_FAILURES=unexpected`.
2. Start a native run that fails retryably. For example, make the
workspace copy-back fail once.
3. See the `run_failure` event in Sentry, and then see the run finish as
`succeeded` after the retry.
## What Changed
- `reportRunFailure` returns without a report for a native run that is
`failed` with `nativePhase: "retryable_failure"`.
- New `isNewTerminalRunFailure(before, after)`: a status change counts
as before. A failed run that leaves the pending retry state but stays
`failed` also counts.
- Three terminal writers can keep the status `failed` → `failed`. They
now use `isNewTerminalRunFailure`, so they still report once:
- the resume reconciler's ambiguous-state write;
- the restart recovery `blocked` write;
- the committed-result projection (`projectCommittedRun`).
- Result-less retries already reset the run to `running` before they
start, so their terminal write still reports through the existing
status-change guard.
- `doc/observability.md` describes the new rule.
## Verification
- `pnpm exec vitest run
src/services/__tests__/run-failure-report.test.ts
src/__tests__/run-failure-sentry.test.ts
src/__tests__/native-session-resumption.test.ts
src/services/native-runtime/native-run-finalizer-telemetry.test.ts
src/services/native-runtime/native-runner-restart-recovery.integration.test.ts
src/__tests__/heartbeat-status-preserving-telemetry.test.ts
src/services/execution-control-reconciliation.test.ts` (from `server/`):
7 files, 363 passed, 9 skipped.
- New tests:
- `run-failure-report.test.ts`: a pending native retry does not report,
and a `terminal_failure` run does. Unit cases cover
`isNewTerminalRunFailure`.
- `native-session-resumption.test.ts`: a sweep that ends an unreported
pending retry sends one event.
- `native-run-finalizer-telemetry.test.ts`: a committed failure for a
pending-retry row sends one event, and a replay sends none.
- `native-runner-restart-recovery.integration.test.ts`: a restart
recovery `blocked` write for a pending-retry row sends one event.
- I removed each of the three writer changes one at a time. The matching
new test failed each time (`expected [] to have a length of 1` / `to
deeply equal [ [ ObjectContaining{…} ] ]`).
- I did not run the full server `tsc --noEmit` locally. The local
container has a 4 GiB memory limit, and the process was killed (exit
137). CI runs the typecheck.
## Risks
- A run that stays in the pending retry state and never resumes no
longer sends a run failure event. The run stays visible on the board as
failed with a scheduled retry. A later terminal write still reports it.
- The finalizer's `failed` + `retryable_failure` write for a failed
provider result also stops reporting. The terminal write that ends that
retry reports instead: retry exhaustion in `recordRetryableFailure`
reports unconditionally, and commit goes through `finalizeNativeRun` or
`projectCommittedRun`.
- A run that was reported in the pending state before this change can
report once more when it becomes terminal after deploy. This affects
only runs in flight during the upgrade.
- There are no schema, migration, or API changes. Run status, retries,
and recovery do not change.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use (shell, code search, test runs) and a read-only code
exploration subagent. Context window: the standard window for that
model.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DottaandPaperclip
|
2cbca4ceb0 |
fix: retain channel tools across task continuations (#16806)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat channels give an agent tools for its assigned task. > - Each later run must resolve the task and its accepted requester again. > - Some native runs store the task in a database column without the older JSON fields. > - Tool discovery read only the JSON fields. GitHub calls also required a new inbound comment. > - This pull request uses the durable task binding and accepted request history. > - Later runs retain their channel tools while current access checks still apply. ## Linked Issues or Issue Description **What happened?** A GitHub task can find `begin_review` on its first run, then receive an empty tool list on a later native run. If discovery is repaired alone, a call can still fail because the later run has no new inbound comment. Slack and voice tool access also read the older JSON task fields. **Expected behavior** A later run for the same task can find and call its assigned channel tools. The current requester, task, company, agent, endpoint, and access rules must still allow the call. Revoked access must fail. **Steps to reproduce** 1. Start a GitHub task and accept its linked requester. 2. Confirm that the runtime can find `begin_review`. 3. Start a later native run with `nativeIssueId`, accepted parent identity, and no JSON task ID or new wake comment. 4. Search for `begin_review` again. Before this change, the search returns an empty list. 5. Repair discovery alone and call `read_pull_request`. Before the request-history fix, the call fails with no active conversation capability. **Paperclip version or commit** Reproduced in source tests before this patch. The branch includes master at `fa50cab5685b9cdbb26ad9c610cc5b6c373d3ff4`. **Additional context** This is a core run-binding bug. The tests use PostgreSQL and synthetic provider responses. They do not send live reviews, Slack messages, or phone calls. ## What Changed - Resolve run task IDs from durable columns, with legacy JSON fields as a fallback. - Use the same binding in the gateway, access policy, identity service, and channel runtime preparation. - Recover accepted channel request history across later runs. Keep company, task, agent, and requester boundaries. - Recover older wake sources through verified parent runs, including after same-user board follow-ups. Clear the board message’s own legacy wake fallback. Keep cycle checks without a fixed limit on valid task history. - Retain the GitHub origin after a board comment from the same accepted user. Skip GitHub origin lookup results from other providers. - Keep ordinary board tools available when the requester changes. Accept legacy user comments without an author-type field. Still reject explicit malformed task IDs. - Prevent a later failure reply from replacing an earlier final GitHub answer. Recheck this rule under the publication lease. - Rotate the native tool contract to replace retained sessions with an older tool list. - Add continuation, publication, revocation, and boundary tests. Update the access and feature maps. ## Verification - Red: the later GitHub assigned-tool search returned an empty list before the binding fix. - Red: the later GitHub read failed with no active conversation capability before the request-history fix. - Green: runtime MCP discovery, GitHub read/begin/submit review, and response publication pass for monitor and same-user board continuations. - Green: Slack history, retained access controls, revocation, and voice gateway continuation pass. - Green: 35 channel, failure-publication, legacy-origin, long-history, and provenance tests pass. Both review findings had failing tests before their fixes. - Green: 107 gateway, operation-credential, identity, and provenance tests pass. These cover both first-run CI failures without changing their tests. - Green: 122 runtime, discovery-policy, and native-session tests pass; 2 unrelated tests are skipped. - Map checks and whitespace checks pass. - Updated to master in `cfcaba22628772d7fe1fb32b57ff7031c7072781`. The sole conflict was two tests added at the same location. Both the continuation test and the publication 403/401 test are retained. - Current local verification: all 179 gateway, credential, requester-identity, access-policy, runtime-context, and session-resume tests passed. Repository typecheck and build passed. Map and whitespace checks passed. - The new review regression first returned no source for a same-user board follow-up over legacy wake metadata. After the repair, all 27 provenance and channel cases passed. The combined GitHub discovery/read/begin/submit path and revoked access are covered. - The broader local channel run passed 176 cases and failed four timing/setup cases: a 15-second Slack authorization timeout, webhook-verification polling, a Slack acknowledgement deadline, and Discord setup polling. All four passed in isolation without source, assertion, or deadline changes. The earlier failure record is retained; the exact load/order cause is not established. Full CI must pass on the latest commit. - Current head: `7b9a08dcbc7b8140b589742f136944270b0496bb`. Repository typecheck and build pass on this commit. Greptile completed on this exact commit with 5/5; all four review threads are resolved. The final-head [CI run](https://github.com/paperclipai/paperclip/actions/runs/39278921653) passed: 55 checks passed and 2 intentionally skipped, with no failures or pending checks. This includes the full sharded test matrix, browser tests, typecheck, build, and Runner verification. Results were delayed during the [GitHub Actions incident](https://www.githubstatus.com/incidents/8tzbpcnl75jp). An additional unsharded local full test run has not completed; the completed full-suite evidence is CI. - Earlier verification: the first PR commit failed two server suites. Those failures were fixed and verified before this update. The original 4 GiB sandbox could not complete full local typechecking or tests; the current checkout completed typechecking and build. The full test suite runs through the CI shards. No full local test-suite pass or live provider delivery is claimed. ## Risks - Request history can restore the wrong authority if its scope is too broad. The resolver checks each accepted parent against the task, company, agent, and requester. Tests cover invalid and cyclic history. - The native contract change starts a new provider session for older contracts. - No database migration or public API schema changes are required. Each tool call still checks live access. ## Model Used - OpenAI Codex, GPT-6 family. The runtime does not expose the exact deployment model ID or context-window size. The agent used reasoning, repository tools, code edits, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
98ca6ff7c5 |
perf: reuse verified provider files during sandbox startup (#16493)
Build provider-pack transport layers once, reuse verified image files in private directories, and upload only missing layers. Preserve archive integrity checks and directory fallback on unsupported hosts. Verified with 44 focused tests, all 48 main CI jobs, cloud Docker Runner checks, and Greptile 5/5 on the merged head. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f3cd3f4ea8 |
fix(runner): require confirmed exit before remote replacement (#16835)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote runs need one runner to own their process identity and listener. > - A health request can fail while that runner continues to work. > - The active monitor treated a failed request as a process exit. > - Recovery could then launch another runner against the occupied listener. > - This pull request retries health reads and requires proof of exit before replacement. > - A brief outage can recover on the same runner without replaying agent work. ## Linked Issues or Issue Description Refs: #16341 adds remote recovery diagnostics. Refs: #16686 retries verification when restoring a saved remote session. This change applies the shared probe policy to active process supervision. **What happened?** The active remote monitor settled its completion promise after a timed-out or rejected health request. The transport treated that result as an exit and started process recovery. An existing runner could still own the listener. Replacement then failed with `runner_ingress_bind_conflict` until the reconnect deadline expired. Missing or changed ownership markers also did not prove that the old process had exited. **Expected behavior** Retry transient read failures on the same runner. Require a matching ownership marker and proof of process exit before replacement. Preserve uncertainty when identity or liveness cannot be verified. **Steps to reproduce** 1. Start a remote runner and leave its listener alive. 2. Make one process-health request time out or reject. 3. Observe completion and the restart decision. Before this change, the monitor settles while the process is alive, and the restart wrapper permits another launch. **Paperclip version or commit** The defect was reproduced on master at `6e89919631`. The patch is based on current master at `c6a0cf4bce`. ## What Changed - Reuse the bounded read-only recovery probe for active supervision. Retry up to three reads within 30 seconds. - Check the saved process generation when available. Accept only the specific process-not-running receipt as exit proof. - Add an exit-confirmation contract to remote process handles. Require it for restart and shutdown completion. Preserve the cause of supervision errors. - Test transient failures, exhausted retries, conflicting identities, real listening processes, and an authenticated runner with a fake provider. Keep confirmed-exit replacement covered. - Document the recovery policy and coverage limits in the execution contract and maps. ## Verification - Red: the timeout and transport-error regressions failed before the patch because completion settled while the runner was alive. The restart-guard regression also failed because replacement was allowed. - Green: 14 server supervision cases passed, including a real listening process. All 120 shared control-plane tests passed. Focused Runner tests passed for authenticated live-runner fault injection and confirmed-exit replacement. - Review regression: a replacement-health test first failed because its error cause was lost. It passed after preserving the cause and detail. The four affected transport cases and Runner typecheck then passed. - `node scripts/check-maps.mjs` passed for 35 areas and 176 capability IDs. - `pnpm -r typecheck` and `pnpm build` passed locally. The sequential local full-suite run was stopped before completion; no local full-suite pass is claimed. Final commit `bcc1b1d339` passed the complete CI matrix: 55 checks passed and two optional Storybook checks were skipped. Greptile is 5/5 with no unresolved conversations. The PR has no merge conflicts. - The first real-listener attempt could not bind inside the execution sandbox. The same test passed with local socket access. The first package-test invocation used the root configuration and selected no tests; the package configuration ran all selected tests. - CI attempt 1: nine test jobs were cancelled together at 12:54:59 UTC. Their test steps report cancellation. The build and uninterrupted jobs passed. The interrupted jobs were retried once. That retry exposed one confirmed-exit message assertion failure in the Runner shard; the other jobs passed. The compatibility correction preserves the prior message. The failing case and four recovery cases then passed locally. The complete CI matrix passed after that correction. - No live sandbox restoration, deployment, or historical task recovery is claimed. ## Risks - A persistent outage now ends supervision with explicit exit uncertainty. It cannot authorize a competing process. Existing ownership and workspace recovery checks remain required. - Health reads can take up to 30 seconds before reporting an unavailable runner. Retries are read-only and bounded. - Local launchers keep their existing completion contract. No database migration or public API change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact serving snapshot and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3858d2341d |
fix(recovery): default saved-monitor waits and keep task recovery quiet (#16686)
## Thinking Path > - Paperclip manages AI agents and their work. > - A task must keep a durable path to its next action. > - A provider can save a task monitor, then fail to yield because it omits internal continuation fields. > - An interrupted provider-pack check can then be reported as permanent artifact incompatibility. > - The task asks its user to certify recovered workspace files, even though this is an operator responsibility. > - This pull request derives the saved monitor continuation, bounds verification retries, and removes that task form. > - Normal recovery stays quiet. Unresolved holds retain inspection and the existing safety checks. ## Linked Issues or Issue Description **What happened?** A yielded completion without `continuation` is rejected even after the provider saves an eligible monitor on the task. Repeated attempts cannot finish. Remote verification also treats timeouts and missing exit status as artifact incompatibility. The task recovery notice exposes raw diagnostics and asks the task user to certify infrastructure files. **Expected behavior** Use current persisted task state to fill routine continuation fields. Retry interrupted verification reads within a fixed bound. Keep normal recovery in the working state. Show a compact notice only when work cannot safely continue. **Steps to reproduce** 1. Save an eligible monitor on the active task. 2. Call `paperclip_finish` with `reportedWorkDisposition: "yielded"` and omit `continuation`. 3. Return a timed-out remote provider-pack command with empty stderr during recovery. Inspect the resulting task notice. **Paperclip version or commit** Source baseline: `aaeff418cf` (master integrated with its current task visibility and saved-source reconstruction rules). Related changes: #15824, #16104, #16451, #16493, #16690. This fix complements saved-result recovery and verified pack reuse. It does not replace their workspace ownership checks. ## What Changed - Allow omitted or null continuation at the provider input boundary. Read current authorized task context and fill a monitor continuation only for the bound run and an eligible saved monitor. Keep accepted results strict. Apply this to Codex, OpenCode, both direct and production-sidecar ACPX paths, and external drivers. Reserve the semantic call identity during the saved-context read and reject cancellation or a changed run/turn. - Bind the shared completion change and continuation helper in a new Copilot v21 source declaration. Preserve v20 history and its decoder support. Qualification stays pending. - Expose the eligible continuation in task context. Completion admission still checks persisted monitor state. No schedule is created or renewed by the default. - Retry read-only provider verification at most twice per command. Hold exhausted reads with a distinct verification-unavailable cause. Reattachment cannot renew the read budget through an unchanged provider-attempt counter. Keep completed verification failures and wrong versions incompatible. - Remove the task file-certification form. Hide the notice only for a matching durable recovery path without an unknown-action replay hold. Keep missed retries, unsafe outcomes, and unresolved file holds visible. - Add production-component stories for automatic recovery, blocked and expired holds, reader access, persisted repair decisions, source reconstruction, queued continuation, and mobile. Preserve master’s guarded saved-source reconstruction choice after confirmed source destruction. Update the execution contract and access maps. ## Verification - Completion, helper, Codex, OpenCode, ACPX, and profile regressions pass. After the sidecar review fix, 271 tests across the production sidecar and three provider suites pass, plus 72 helper/profile tests. The real sidecar MCP bridge tests cover saved-monitor acceptance, no-monitor refusal, cancellation, duplicate call identity, and raw-versus-forwarded semantic receipt hashes. The OpenCode tests also cover eligible-monitor acceptance and refusal without one. - The fresh full native executor suite passes all 743 tests. Four profile, installation-integrity, and ACPX driver suites pass all 168 tests. The earlier CI fingerprint failure required a new immutable source declaration; its follow-up Docker failure required the matching context allowlist entry. Both fixes are verified locally. Targeted server monitor, tool authority, and provider verification suites pass. The probe tests include a real timed-out subprocess, an unrelated responsive timer, transport errors, and completed integrity rejections. - All 90 tests in the merged task notice, reconstruction choice, recovery display, and lineage suites pass. They cover reader access, live/missed/mismatched recovery, unknown outcomes, saved messages, retry refusal, and removal of the file form. Both review findings have failing-baseline/passing-treatment regressions: identical polling across retry expiry, and a failed refresh after cached running activity. - Browser walkthrough passed for automatic recovery, unresolved details, a 390px mobile story, and the merged saved-source reconstruction and queued states. The reconstruction control starts disabled and normal automatic recovery shows only Working. The stories use production components and fixture API data. They do not prove live cloud recovery. - Token gates, map checks, and Storybook catalog generation pass. The exact Docker context integrity command passes locally after adding the new build declaration to its allowlist. - Workspace typecheck and build pass on the final source, including the versioned identity update. The final server and UI guards pass their focused tests, typechecks, builds, and token gates. The final standalone Storybook builds. The saved-monitor default retains master’s stricter child-task visibility filter. Re-review caught an additional production-sidecar validator before forwarding; the fix now defaults at that boundary and has passing regression controls. After preserving master’s new reconstruction capability, 68 targeted reconstruction, tool-authority, and provider-verification server tests pass. The current-master integration at `2a80050a0d` has a fresh 5/5 review with zero unresolved threads. Its complete CI matrix, including Runner tests, is the merge acceptance record. - On the final master integration, local typecheck, build, and all 90 recovery UI tests pass. A concurrent runner run hit five 5-second sidecar startup timeouts; an isolated sidecar run passed 86 tests and timed out in the unrelated unsupported-bootstrap case. Those local deadline failures remain recorded; the canonical CI Runner lanes use their normal deadlines and pass. The unsharded local full-suite attempt did not complete. It spanned source edits; the fresh native executor run above passes. An unrelated workspace fixture timed out once and then failed to reserve 32 contiguous ports on this host. Its unchanged-base control and CI server suite pass. Full-suite acceptance uses the fresh latest-head CI matrix; this PR does not claim a passing unsharded local run. ## Risks The default applies only to an eligible persisted monitor. An explicit invalid continuation still fails validation. The added read occurs only when a yielded report omits its continuation. Admission rechecks current state after that read. An interrupted verification command can add one extra 30-second probe. Completed rejections stay closed. These probes do not change the pack, launch provider work, or replay external tools. Two inconclusive reads hold the original run. They do not schedule another reattachment with an unchanged provider-attempt counter. This PR’s pending Copilot v21 declaration binds the changed shared contract, resolver, direct driver, and production sidecar sources. The previous v20 declaration is immutable and remains decodable, but launch requires the new exact identity. Qualification remains pending. Task users lose the file-certification form. Board-authorized source-only repair remains available through the resolution API with the original evidence and stop-receipt requirements. This branch does not deploy or recover historical held runs. > This is a bug fix and UI polish within existing recovery behavior. ROADMAP.md was checked. ## Model Used OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning, with tool use and code execution. The host does not expose the context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed-area suites; the full local run limitation is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6bec153410 |
fix(runtime): retain cancelled workspaces before reconstruction (#16700)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs must preserve files when Stop ends a turn before its result is saved. > - The cancellation path could skip file retention and leave a prepared remote workspace active. > - An older controller could also admit an extra run while an earlier lost-source hold was unresolved. > - Rejecting every later run then prevented recovery even after that extra run was stopped and its files were reconciled. > - This change retains cancelled workspaces and records the exact proof for a safely reconciled intervening cancellation. > - Recovery can proceed without changing cancelled history, replaying actions, or weakening execution gates. ## Linked Issues or Issue Description Refs: #16583, #16690, #16696. **What happened?** Stop could acknowledge a native turn without recording its unsaved file obligation. Reconstruction also rejected a later cancelled run after its exact environment and files had been reconciled. The task could remain inactive with an `in_progress` status, which the reconstruction endpoint rejected. **Expected behavior** Retain unsaved files after Stop. Permit a board operator to reconstruct from saved sources only after each intervening cancellation has a confirmed stop and a file-only repair record. Preserve the original failed export and the cancelled run. **Steps to reproduce** 1. Prepare a native workspace and cancel the turn before a structured result exists. 2. Inspect its file retention and remote lease state. 3. For a task with an older destroyed export, reconcile the later cancelled run's exact files and unperformed actions. 4. Request reconstruction from verified same-task attachments. ## What Changed - Pin prepared native files before cancellation settlement. Defer teardown if pinning fails. Use the existing stop-only cleanup worker. - Repair active warm and ephemeral sources, plus stopping and confirmed-stopped historical cancellations in bounded startup and periodic batches. Preserve the existing exact-source checks. Bound row-lock waits and advance past a busy source. - Accept only same-owner native run-only cancellations with no result or current coordinator, confirmed stopped leases, and board file-only repair with `not_performed` action outcomes. - Save hashes of each intervening repair and recheck them at queue claim and final dispatch. Reject changed evidence or another later run. - Expose saved-source reconstruction in the issue API for inactive `blocked` and `in_progress` tasks, and hide it when an execution or checkout owner exists. Permit the same inactive source at submission. Keep the source, owner, version, attachment, budget, pause and retry checks. - Update the shared evidence type, recovery runbook and capability maps. Add an optional cancelled-native no-action confirmation to file recovery, resetting evidence when its source changes. No migration, new endpoint, permission or dependency. ## Verification - The new fixture fails on the original validator with `workspace_reconstruction_stale` after valid cancelled-run file repair. - Focused source and Stop retention tests: 94 passed before the final contention case. The subsequent three-suite run passed 123 tests, including real PostgreSQL lock contention and public startup/API/final-dispatch paths. The final narrow-query rerun passed all 123 tests. Review correction: all 143 tests passed across four suites, including real lock contention through the teardown boundary and stopping/stopped allocations. - Six broader cancellation, executor, heartbeat and restart suites: 831 passed, 9 existing skips. - Final correction: all 145 backend tests and all 36 recovery UI tests passed. Real contention covers both warm reusable and ephemeral sources. Optional no-action confirmation defaults to uncertified, and changing source clears evidence. Workspace `pnpm -r typecheck` and token gates passed on these sources. - Maps, module boundaries and Git whitespace checks: passed. Full `pnpm build` passed on the final `4e8c0e73ac` revision as well as the prior reviewed heads. A full local run of the preceding fix completed with 21,724 passes, 109 skips and three failures in webhook body limits, a status-card timeout, and a port-allocation fixture. All three suites then passed unchanged in an isolated rerun (209 tests). The full local run remains recorded as failed; the final head's full CI passed (55 completed checks, no failures). A fresh full local run is also in progress on the frozen reviewed head. - Preserved failures: the first attempt lacked a built plugin SDK; after building prerequisites, the regression reached the expected validator rejection. Treatment tests caught the file-only action's deliberately null continuation owner. A real lock test then caught the database driver's wrapped timeout error. Both defects were corrected and retested. Independent review then found a teardown race after a failed retention write. A second review found active warm sources excluded by the historical policy filter; the regression reproduced both warm failures before correcting the query. Its real-row-lock fixture reproduced the failure; the fixed release guard and historical stopping/stopped cases pass. No checks were weakened. - A pre-deployment walkthrough caught the issue API still hiding reconstruction for an inactive `in_progress` task. The status/capability baseline reproduced three failures, including inappropriate capability flags for execution/checkout owners. The fix uses the same inactive-state requirements as submission; its positive fixture reads the assembled issue API before reconstructing. All 117 tests in those three recovery suites pass, and all 28 heartbeat-recovery tests pass separately. Full workspace typecheck passed again. Full build passed with the exact `4e8c0e73ac` stamp; latest-head independent review is 5/5 with no open findings. All 55 latest-head checks completed with no failures. - Live tenant acceptance remains pending. No successful reconstruction, export or warm-process proof is claimed by these local tests. ## Risks - Stop retains unfinished work, so storage remains until file recovery and explicit cleanup complete. - Reconstruction remains held for unknown or performed external actions, foreign owners, live or rebound environments, accepted results, changed evidence, and unrelated later runs. - The sweep examines at most five candidates per pass. It uses a fair cursor and bounded transaction waits. It performs no provider call and starts no new turn. - Intervening verification reads only the needed JSON fields. The source transaction still serializes recovery for the same task. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
69ac928d2b |
fix(runtime): preserve bounded retries after workspace reconstruction (#16696)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An operator can rebuild a destroyed workspace from saved task attachments. > - The first reconstruction run must pass current source and execution checks. > - A temporary bootstrap failure can create a bounded retry before provider work starts. > - The retry has a new run ID, so the first-run-only reconstruction check rejects it. > - This change verifies the retry chain back to the approved first run and keeps current safety checks. ## Linked Issues or Issue Description Refs #16690. **What happened?** After an operator records workspace reconstruction, a safe bootstrap retry is cancelled before provider startup. The normal retry scheduler creates the successor. The reconstruction check then rejects its immediate predecessor because it expects the original failed export run. **Expected behavior** A verified automatic retry can continue under the same reconstruction decision and remaining retry budget. Changed sources, unrelated executions, uncertain outcomes, pause and later human blocks must still stop startup. **Steps to reproduce** Record reconstruction with a retry budget remaining. Let its first run fail before provider work starts with recorded bootstrap evidence. Use the normal bounded retry scheduler and the locked dispatch gate. Before this change, the scheduler succeeds but dispatch returns false. ## What Changed - Verify the predecessor chain back to the recovery action's exact first continuation. - Check company, task, agent, terminal predecessor state and each scheduler-owned wake. Reject cycles, foreign parents and forged wakes. - Exclude only those verified predecessors from the unrelated-newer-run check. Keep source hashes, original result identity and all current execution gates. - Reuse the existing reconciliation proof for failed predecessors. Keep the original retry limit. - Add real PostgreSQL scheduler and assembled-heartbeat regressions. Update the recovery guide and feature map. ## Verification - The baseline regression failed at final dispatch after the normal retry scheduler created its successor. - The corrected focused group passes 133 tests across reconstruction, retry scheduling and dispatch. It admits two bounded bootstrap retries and refuses a third. The assembled heartbeat enters the provider once for an eligible retry and zero times after a controlled post-claim human block. - An intermediate negative case exposed missing predecessor-outcome validation. Reusing the existing reconciliation proof fixes it. Deleted sources, pause, forged wakes, foreign parents, unrelated runs and new recovery holds remain refused. - Full workspace typecheck and build pass on rebased head `bd301ec8a703fa1cdc9e1db2638440d203e8d57a`. The rebased focused group passes all 133 tests. All CI gates pass on that exact head, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 with no unresolved threads. A clean full local `pnpm test:run` is running; the local source stays frozen for that run. - Module boundaries, maps and diff whitespace checks pass. Live staging acceptance follows the qualified full Cloud deployment. - The earlier full local run for #16690 failed while files changed and its PATH hid `lsof`. All five affected suites then passed in a clean run: 246 passed, three skipped. Those results do not establish a full local pass for this follow-up. ## Risks - Retry ancestry is a rare recovery path. Its primary-key reads grow with the chain and stop at 64 predecessors. Ordinary runs perform no new ancestry reads. - Unknown effects and active recovery obligations stay held. This does not reset budgets, change provider lifecycle policy, grant access or replay completed actions. - No schema migration or new API is included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks; full-run scope is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7c7d2d8d2c |
fix(runtime): continue from saved sources after workspace loss (#16690)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runtime must preserve saved work and prevent duplicate execution. > - Older cleanup could destroy a sandbox before its workspace export finished. > - The existing recovery controls require either the original sandbox or recovered original files. > - Surviving attachments can support a rebuild, but they do not prove that missing edits were recovered. > - This pull request adds an explicit reconstruction decision and one fresh continuation through the normal execution gates. > - Operators can continue useful work while the original failed run remains truthful. ## Linked Issues or Issue Description **What happened?** A historical failed export can have an accepted result, a prepared workspace descriptor, and a confirmed destroyed sandbox. Its notice can also say `new_source_execution_path` and disappear. The task then has neither a truthful original-file repair path nor a supported reconstruction path. **Expected behavior** Keep the missing-file obligation visible. Let an authorized operator choose surviving task attachments and record lost work and external action outcomes. Start one fresh continuation after current execution checks. Keep the old result and failed export intact. **Steps to reproduce** 1. Seed a failed native export with a prepared descriptor and an exact destroyed-lease receipt. 2. Resolve its notice with the historical `new_source_execution_path` bookkeeping value. 3. Read the task. The baseline hides the hold even though no source reconciliation exists. 4. With this change, choose **Continue from saved sources** and submit a reconstruction decision. Related public work: #16451 preserves healthy warm sessions. #16583 prevents destructive cleanup of unexported work. This change handles historical source loss that those fixes cannot undo. Searches of open and closed work found no equivalent reconstruction endpoint. ## What Changed - Add a board-only reconstruction endpoint with `runtime:manage` and task visibility checks. - Validate the exact run, result, descriptor and destroyed lease. Reject active copyback, competing owners, newer work, stale task status, and foreign attachments. - Record attachment hashes, missing work, and external action outcomes. Preserve the original failed run, accepted result, prepared descriptor and terminal failure. - Hold other continuation paths until the durable reconstruction intent admits its successor. Record that successor in the same transaction and preserve the retry count. - Add an operator control that distinguishes reconstruction from original-file recovery. Update the recovery runbook and capability maps. ## Verification - Exact baseline source returns no blocker for the destroyed-source fixture. The same fixture returns a required recovery hold with this change. - New real PostgreSQL and assembled API/heartbeat suite: 25/25 pass. It covers duplicate submissions, lost acknowledgement, context reset, source identity, task ownership, concurrent execution attachment scope, later human blocks, deleted sources and agent pause. - Existing export retry, explicit continuation and recovery-action suites: 386 cases pass. One new-fixture constraint error in the combined run was corrected; the full new suite then passed. - UI recovery suites: 46/46 pass across three files. They cover disabled actions, loss acknowledgement, failure retention and runtime authority. - `pnpm -r typecheck` and `pnpm build` pass. The first sandboxed typecheck attempt could not open the TypeScript IPC socket; the authorized rerun passed. - Module boundaries, map checks, token gates and diff whitespace checks pass. - OpenAPI route coverage: 21/21 pass. The first CI head exposed a missing endpoint registration; this head adds the registration and the focused suite passes. - Review identified a missing dispatch-time check and an isolated attachment cache. Both are fixed. Claim and final handoff recheck the exact successor, sources and current execution gates. A controlled post-claim operator block prevents the assembled heartbeat from entering the provider. The shared attachment cache refreshes uploads and removes deleted selections. - Final focused run: 180/180 pass across six suites, including reconstruction, existing dispatch/stale-queue contracts and UI recovery. - A second review reproduced cancellation by an already satisfied historical connection card while the task was `todo`. Read-only connection availability now works before checkout; active-task mutation checks remain strict. The baseline failed two cases; after the fix all 74 connection and reconstruction tests pass, including revoked access, pending questions and pending approvals. - All current-head CI gates pass on `9d197236da11fa24f1ceadf5589cb2b81179d824`, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 for that exact head with no unresolved threads. The full local `pnpm test:run` is still running its serial server group; no full local pass is claimed. Live staging recovery is pending deployment. ## Risks - Reconstruction cannot restore missing bytes. The operator must identify surviving sources and actions that must not be repeated. The agent must verify those attachments before rebuilding. - This path is limited to exact, confirmed destroyed remote sources with an accepted result. Retained or uncertain sources keep their existing repair path. - No schema migration, credential change, budget increase or warm-session policy change is included. Normal pause, ownership, budget, approval and dispatch gates still apply. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run limitation recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e6dcc6b095 |
Connect personal Muse through secure native execution and Vault OAuth (#16448)
Add company-scoped Muse native control, OAuth, wake receiver and recovery. Recheck current authority after waits and before effects or receipt replay. Preserve task-read scope, assignment expiry, independent holds, nonblocking reassignment and truthful cleanup provenance. Retain source trust on progress comments and verify current Muse V8 authority through real signed-JWT HTTP requests. Preserve the upstream trimmed ordinary authentication lookup. Refs #16448. Preserve captured canonical queue recovery and concurrent environment preparation while retaining the scoped Muse V8 exclusion. Normal migration 0354 and existing Muse SQL remain unchanged. Fresh current-head Greptile review is 5/5 with the project-write finding addressed. Required CI and independent technical review gate the merge; maintainer override is reserved for formal approval rules. Recheck current Muse project-write authority after all receipt/task/run/admission waits, before creation or replay. Real PostgreSQL regressions reproduce four late effects/replays on the old route; the repaired three-file treatment passes 354 cases with no skips, and the scoped server typecheck passes. Ordinary/Dot behavior is preserved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
e30550ef39 |
refactor: remove code comments added by recent PRs (#16681)
CONTRIBUTING.md says PRs add no code comments, but #16485, #16586, #16621 and #16672 added some. This removes the 215 comment-only lines git blame on master attributes to those merge commits, across 12 code files, plus one blank line the removal left doubled. Rewritten (not added) comments in scripts/assert-cloud-image-sentry.mjs and comments in .github workflows are left alone. No code changes. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0528cc7ee0 |
perf: overlap native GitHub and assigned-tool setup (#16620)
Prepare managed GitHub access and fresh assigned-MCP discovery concurrently for remote native Codex. Wait for both before provider construction and drain both on failure through the existing run cleanup. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9a3290fe10 |
perf(runner): overlap default remote artifact preparation (#16608)
Prepare the native runner and default image Codex harness together on providers that declare concurrent sync support. Preserve all executable checks, wait for both operations on failure, and retain serial explicit harness installs. Verified with 881 affected cases, full typecheck/build, exact-head full CI and Greptile Base 5/5. The real Daytona component median fell from 839ms to 432ms; full deployed reply latency remains unproven. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2cc7992369 |
Share the external Runner lifecycle and add strict Muse contracts (#16444)
Add the external Runner foundation with distinct Dot/Grok V6, canonical home-mode V7 and Muse V8 contracts. Keep generated schemas and profiles synchronized. Bound default question-answer command IDs in a disjoint namespace while preserving existing IDs and durable replay. Preserve canonical migration history and add normally generated, replay-safe Muse schema milestones with literal preview-history fixtures. Refs #16444. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c91f7a6e5f |
perf(runner): overlap verified sandbox launch assets (#16585)
Native sandbox launch preparation stages fresh harness files and immutable runtime context in sequence. Overlap them only under the verified concurrent-sync capability, settle every started transfer before launch or cleanup, and preserve serial SSH behavior and error priority. A matched real-Daytona comparison reduces staging from 1,552 ms to 1,017 ms. The larger active-run difference includes model variation and is not attributed to this change. Add an explicit single-attempt diagnostic and synchronize stale Pi evidence metadata with its shipped profile. Validation: 764 native runner/tracing cases, 2,764 support cases, repository typecheck/build, fresh exact-head full CI/E2E/security and Apex 5/5 pass. Observability remains a silent no-op when no endpoint is configured. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f5534717b0 |
fix(runtime): preserve unexported native work during lease cleanup (#16583)
Preserve the exact native workspace source until its export is finalized. Apply the same guard to cleanup sweeps and direct environment teardown. Keep stop-only recovery and healthy warm retention available. Verified with 317 targeted tests, server typecheck, module boundary check, all current-head CI gates, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
6f8ad3d77b |
Add verified session-local stdio MCP transport to Runner (#16526)
## Thinking Path > - Paperclip manages AI agent runs and their tool authority. > - Runner supplies provider sessions with assigned MCP connections. > - The current native connection binding supports an HTTP gateway only. > - Execution hosts also have private local tools with stdio MCP interfaces. > - Those tools need session-local registration and safe concurrency. > - This pull request adds verified local stdio assignments beside the existing gateway. > - Providers receive the existing tool schemas and image content. ## Linked Issues or Issue Description **Problem or motivation** Runner can supply a native provider session with an HTTP MCP gateway, but cannot also assign a verified local stdio MCP server without persistent provider configuration. Execution hosts need a session-local transport for local MCP tools, coordination for calls sharing a resource, and native schema/image delivery. The pinned native Codex client drops MCP images when `structuredContent` is also present. **Proposed solution** Add general stdio MCP transport support using host-assigned executable digests, arguments, environment and a private Unix socket identifying the shared resource. Tool implementations supply their own schemas and responses; Runner forwards their native MCP schemas and content. Register the transport for the current provider session. Serialize calls across processes, keep unknown outcomes fenced, and authorize assigned native Codex tools without a new prompt while routing explicit tool confirmations through existing PRP permission requests. Retain the HTTP gateway and applicable permission policy. **Alternatives considered** Reimplementing tools or translating their actions into provider-specific APIs would tie the transport to individual tool implementations. Global provider config would leak session assignments. Shell commands alone do not deliver native MCP images or preserve MCP approval requests. **Roadmap alignment** This completes the existing Runner transport boundary. Related: #10690 (operator config for the legacy ACPX engine), #11605 (gateway delivery of installed stdio connections), #16486 (preconfigured provider homes). This change assigns local tools on the execution host and leaves those administration paths intact. ## What Changed - Add reserved, verified stdio bindings for native Codex and through the shared ACPX runtime. ACPX clients must support stdio MCP assignments. Read-only sessions omit the binding. No home/project config is rewritten. - Run the backend with only its assigned environment through the verified executable launch path. - Serialize all calls sharing a private Unix socket across sessions. Metadata remains responsive. Queued cancellation prevents execution; active calls remain fenced until acknowledgement. Unknown outcomes require a resource restart. - Authorize host-assigned native Codex tools without a new per-call prompt, matching existing execution-host shell utilities. Preserve explicit tool deny/prompt rules. Route requested MCP tool confirmations through one-time PRP approvals, bound to the active thread and turn. Keep shell escalation disabled and existing tool deny rules effective. ACPX adapters retain their applicable ACPX permission policy. - Preserve schemas and image blocks. For native Codex 0.160/0.162, retain structured metadata as MCP text on image-bearing results to avoid the client's image-loss bug. ACPX adapters receive the original response. - Document transport authority, cancellation and recovery limits. ## Verification - `pnpm -r typecheck` and `pnpm build` passed with pnpm 9.15.4 and Rust 1.97.1 on macOS arm64. - Before Apex, full `paperclip-runner-core` Rust tests passed. After the fixes, all 379 library tests and eight subprocess tests using a generic stdio MCP fixture pass. Those subprocess tests cover schema/image passthrough, assigned-only environment, executable drift, call serialization, cancellation and unknown outcomes. The native Codex suite passed 91 tests with two intentionally ignored; `receipt_limit_rejects_the_call_and_keeps_polling_when_interrupt_fails` timed out waiting for `turn.interrupted`, including an isolated retry. The unchanged pre-fix commit `80917b301a2309f7966a466cf0c198a2112be2d5` reproduces the same timeout in isolation; this local native-suite run is not claimed as fully passing. - All 252 native controller tests pass against the final narrowed approval branch. Focused TypeScript transport/permission suites passed (298 tests); final binding, read-only host, policy and adapter checks passed (105 tests). - **Provider-client smoke coverage:** pinned Codex 0.160.0 through native Runner and Claude SDK 0.3.286 used local stdio MCP servers and loopback model API fixtures without live model credentials. These clients define the smoke-test coverage; this is not an exhaustive adapter compatibility test. Image responses reached the next model request as typed image blocks. Native Codex checks also verified unattended execution without approval requests, explicit per-tool prompt denial without backend execution, disabled-tool rules, and stale-turn approval rejection. - Two Greptile concurrency findings were reproduced with failing subprocess tests and fixed: pending cancellation under output backpressure, and a ctime-based fence reset after chmod. Initial checks exposed Codex CLI option-scope and key-quoting errors, which were corrected. The existing ordinary elicitation regression also exposed an overly broad request branch; the final branch handles explicit tool approvals only. - `pnpm test:run` was attempted. The broad local server suite hit persistent embedded PostgreSQL startup failures and unrelated local Git fixture failures; it was stopped with the failure log retained. Those local results are not claimed as passing. Hosted Linux CI gates passed on the reviewed runtime commit, as recorded below. - First Apex review found three additional defects, now fixed with regressions: keep the resource-lock socket snapshot instead of sampling the socket twice; project native permission requests with durable turn/item IDs while retaining provider IDs in details; validate provider response conversion before marking a request as settling. The retry regression failed before the fix. Focused Codex request/adapter checks pass (23 tests), server permission projection passes (16 tests), and all 379 Runner library tests pass. The broader database-backed bridge suite was attempted but its local PostgreSQL setup timed out; 29 tests did not execute. This local failure is retained, not claimed as passing. - Runtime acceptance before the documentation clarification, on `bd4fccae001157beb5bc76ff17c1920a5db3105a`: standard Greptile 5/5, then Apex 5/5 with no unresolved threads; all 60 checks complete successfully or intentionally skip. Ten CI jobs were interrupted by the same runner shutdown, including one stale job that required cancellation before retry; the same-commit retry passed. Hosted Rust tests also pass, including the native suite with the local baseline timeout noted above. - `git diff --check`, map validation, and pre-push secret/PII scan passed. ## Risks - No live model account or deployment isolation qualification was run. The loopback fixture proves the actual vendor-client request format, not model interpretation. - Stdio MCP support uses the current host-assignment contract: each binding requires a verified executable and a private Unix socket identifying the serialized resource. Other platforms retain the HTTP path. No provider version was upgraded. - A lost acknowledgement blocks subsequent calls until the host restarts the assigned resource. Direct shell calls are outside this MCP fence and retain the existing shell policy. - Preconfigured Claude remains unattended and denies interactive approval requests. Host assignment authorizes native Codex local tools; it must be supplied only for sessions already authorized to use the resource. ## Model Used OpenAI Codex, GPT-6, with reasoning and repository/tool execution. The session does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b294abb9eb |
perf(server): append ordinary run events in one statement (#16569)
Ordinary run events now validate their binding, allocate a sequence and insert the decoded receipt in one atomic PostgreSQL statement. Native replay and retry-exhaustion transactions keep their existing behavior. Verification: exact-head full CI/E2E, Superagent, legacy security and Apex 5/5 passed. PostgreSQL concurrency, rollback and payload controls pass. Controlled network-delay comparison reduced median event persistence from about192ms to48ms; no staging response-time claim is made before deployment. Co-Authored-By: Paperclip <noreply@paperclip.ing> |