mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 17:33:51 +02:00
412914cfa78d5abefcc34d3eb8d4bb86e461caac
99
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicky Leach
|
412914cfa7 | Read native plugin task output files on the execution host (#17285) | ||
|
Nicky Leach
|
2cd6eb27c7 | fix(runner): record the plugin-prepared working directory (#17243) | ||
|
Nicky LeachandPaperclip
|
00331e0254 |
Connect native Runner admission to plugin-owned workspaces (#16978)
## Thinking Path > - Paperclip manages work for AI agents. > - A plugin can own an execution environment and its files. > - #17029 supplies the task driver and bounded setup verification. > - Heartbeat still prepares workspace files on Core and expects a shell transport. > - This PR gives the selected plugin ownership of workspace preparation and native task execution. > - Core keeps admission, tool authority, and recovery checks. ## Linked Issues or Issue Description **Subsystem affected** Heartbeat workspace admission and native Runner execution. **Problem or motivation** A task-only plugin cannot run through Core filesystem staging or shell execution. Native admission and recovery must use the same lease and workspace ownership. **Proposed solution** Use the task driver from #17029. Resolve workspace and project identities without Core file operations. Connect native dispatch and restart recovery to that driver. **Alternatives considered** Keep workspace admission and dispatch in one change. A workspace change alone would leave dispatch on the previous transport. An execution change alone would leave Core staging files for the wrong host. ## What Changed - Enable plugin environment selection for native workspace sessions. - Resolve workspace metadata and authorized project references without Core checkout, copy, or provision operations. - Preserve agent-default and primary workspace policies. Use the pinned workspace provider for plugin-backed project storage. Reject unsupported Core service commands and cross-owner reuse. - Connect native execution to the prepared task driver. Keep assigned MCP authority on Core. - Save exact checkpoint identity, transfer selected account refreshes, and require termination proof before replacement or cleanup. - Complete managed plugin workspaces through their storage provider. Use the same check after a restart. Keep branch validation and leave task-owned files on the plugin. - Include default instruction content in runtime synchronization when no instruction directory exists. - Defer new-agent Copilot verification until the saved-agent lifecycle can acquire the agent-owned computer. ## Verification - Rebased onto `origin/master` at `898144f4b3`. Master advanced once during verification. Both manual conflicts joined import blocks: one in `native-session-executor.ts`, and one in `heartbeat/runtime-execution.ts`. The seven patches retain their logic. The first rebased head passed repository typecheck, build, all 862 focused execution/finalizer cases, and all CI gates. The latest head `93e5a72892` passes server typecheck and all 880 focused execution, finalization, and execution-control notification tests. All 54 CI/review checks pass, with two expected skips. Greptile ran automatically and scored this head 5/5 with no unresolved findings. No new review or Apex request was made for these import-only resolutions. - The successful CI workflow initially left three stale job-check reports. Targeted policy and summary reruns cleared the reports. No source change or test fix was needed. - Repository typecheck and build pass: `pnpm -r typecheck` and `pnpm build`. - Focused native execution and restart tests pass: 795 cases. The account and harness suites pass: 159 cases. Workspace admission passes: 48 cases. The SDK task lifecycle, workspace API, runtime-hook, and instruction checks also pass. - The heartbeat integration case uses the merged workspace provider contract and reaches native dispatch with no project files on Core. The new-agent case confirms that creation leaves the agent in `preparing` until lifecycle verification. - Map validation and the Copilot source-identity check pass. No Runner implementation or source-identity bump is in this PR. - The first rebased workspace run found two stale path assertions and a recovery guard that did not recognize task-owned targets. These are fixed. The storage integration fixture also needed its persisted source binding; the production authorization check stays in place. - Two prior review findings required rejection of Core service startup and cross-owner workspace reuse. The existing admission fixtures now check both failures. The installed-account authentication finding was fixed in merged #17029. - Tests reuse the current fixtures. No new provider test matrix or live Runner process is added for these fixes. - The prior head `2db273ec12` passed CI on its first attempt and reached standard Greptile 5/5. Apex scored 4/5 and found local-folder checks during plugin completion and recovery. - The completion fix reproduces both failures. It passes all 110 cases in the runtime execution, workspace finalizer, and workspace provider integration suites. Server typecheck and map checks pass. CI passed on `62d8335b4e` on its first attempt, and standard Greptile scored 5/5. The next Apex review scored 4/5. It found that a failed plugin inspection could abort restart recovery. - The restart fix records a failed workspace receipt and uses the current run retry policy. A regression first reproduced the aborted pass. It now checks retry state, retained results, and completion of another workspace in the same pass. All 68 finalizer cases and server typecheck pass. CI passed on `7d4016323b` on its first attempt. Standard Greptile and Apex both scored 5/5. Apex left one P2 about lost failure codes. - The diagnostic fix keeps the known branch and owner mismatch codes in receipts and run recovery state. It gives repair guidance for those codes and keeps unknown plugin errors generic. All 96 affected finalizer cases and server typecheck pass. Before the rebase, required CI passed on `c2fa61c6c8` on its first attempt. Standard Greptile and Apex both scored 5/5 on that head, with no unresolved findings. The new head retains these reviewed patches. - The earlier Superagent scan on `c2fa61c6c8` was inconclusive after a pinned merge-base checkout failed or timed out. The current scan on `93e5a72892` passes. - The exact rebased head passes all 48 workspace cases and server typecheck. The first local test launch could not bind PostgreSQL inside the sandbox. The authorized local-server run passed. - The prior head reached standard Greptile 5/5. Its first CI run hit an unchanged Runner cleanup timeout and an external build-runner shutdown. Both jobs passed on the single retry. The timeout cause is not established. - A local regression check found that a retained Core workspace could also enter a plugin task without file transfer. Core now rejects this move before provider work. The existing ledger test covers both directions and confirms that the source files stay intact. - The first active standard review scored 4/5. It found that a fresh plugin retry could enter Core Git preparation. This mode now uses the existing unsupported-storage error before workspace resolution or acquisition. Both unsupported-mode cases pass, and server typecheck passes. - No live Cloud or provider validation was run. ## Risks - The prerequisite #17029 is merged. This PR targets master. - Native plugin tasks support per-turn workspace sessions. Core service commands and fresh retries that import retained files remain unsupported. A workspace storage plugin can use the modes and strategies it declares. - Core-managed credentials support Codex and direct Claude accounts. Other managed credential routes fail closed. Other harnesses can use an installed account. - A provider must preserve its prepared configuration and report process termination correctly. Uncertain termination keeps the task pending cleanup. - Cloud deployment and live provider validation remain separate. ## Model Used OpenAI Codex, GPT-6. The exact model ID and context window are not exposed in this session. Tool use includes code editing, local tests, and the GitHub CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
898144f4b3 |
fix: stop idle execution-control recovery polling (#17094)
Replace idle execution-control polling with workers driven by committed writes and persisted deadlines. Preserve startup recovery, bounded retries, and shutdown safety. Stabilize the Slack setup browser regression by waiting for the restored setup screen before simulating its webhook. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
30b753f614 |
perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path > - Paperclip manages AI agents and their paid work. > - Budgets must prevent new work and stop reported overruns. > - Cost settlement also ran serial lifecycle reconciliation for every eligible company agent. > - That made request latency grow with unrelated idle agents and delayed results on stop delivery. > - This change separates atomic admission and accounting from durable asynchronous enforcement. > - Scoped live usage detects overruns without writing provisional ledger charges. > - The result is constant query scope for ordinary settlement and recoverable stop delivery. ## Linked Issues or Issue Description **What happened?** Settlement and budget checks entered company-wide lifecycle reconciliation. A company with 1,000 unrelated idle agents performed 1,001 accounting transactions for one ordinary cost evaluation. User requests could also wait for provider termination. Running usage checkpoints did not enforce reported monetary exposure. **Expected behavior** Keep the final reservation gate authoritative. Commit charges, scope blocks, and enforcement intent before returning. Deliver stop requests after commit. Read only applicable policies and execution scopes for ordinary charges. Stop long-running work when supported reported or priced usage establishes an overrun. **Steps to reproduce** 1. Create a disposable company with one company budget and idle agents. 2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated idle agents. 3. Compare query counts, company accounting transactions, and duration. 4. Use the controlled partial-usage and delayed-stop fixtures to check enforcement and request responsiveness. **Paperclip version or commit** The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from `master`. Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853, #13075. These cover the affected service charge, cancellation, and accounting paths. This change removes their shared reconciliation overhead. ## What Changed - Batch applicable policy exposure under the existing company accounting lock. Preserve exact fractional money and zero-reservation dispatch fences. - Commit a durable final budget dispatch claim before provider I/O. Retain a separate final issue/run ownership transaction through synchronous handoff, without holding the company accounting lock or awaiting provider work. Preserve native recovery lease checks and duplicate handoff fencing. Browser handoff rechecks current policy; it reuses a consumed rate slot only for the identical policy generation. - Separate ledger spend, live observed spend, and effective held capacity. Use the maximum of reservation and live observation for each unfinished operation. - Deliver budget cancellation through the existing coordinator. Revalidate generations and persist queued/running stop fences before external termination. Bound stop concurrency to four. - Apply agent holds only to the affected agent. Project budgets leave other projects running. Project real company transitions in bounded lifecycle batches. - Replace periodic broad reconciliation with durable pending work, startup repair, receipt replay, and concrete retry/window deadlines. Retain drain and idle-safety checks. - Enforce accepted cumulative usage checkpoints without provisional charges. Final accounting atomically replaces live exposure and settles reservations. - Add operator-gated budget spans, regression coverage, map updates, and a real-PostgreSQL comparison harness. No schema or first-party telemetry change. ## Verification - Full workspace `pnpm -r typecheck` passes on the final ownership fix, rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head `4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks pass. - Focused real-PostgreSQL coverage includes the 292-case rebased accounting/dispatch run, 63 browser cases, 52 run-control/cancellation cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and the exact Muse confirmation regression. Native executor unit coverage passes all 768 cases. Final ownership coverage passes: 74 adapter/use-case cases, nine public regression cases, and the additional deleted-project race. These exercise competing ownership, budget changes and charges between phases, zero/nonzero duplicate claims, rollback, recovery contention, read-only capacity rechecks, and controlled HTTP startup after commit. Another 74 pure/startup/tracing cases pass. The complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its external security scan identified a stale-owner handoff race. The final fix restores synchronous issue/run ownership fencing after the accounting transaction commits. Final head `8e84f55ce8` has fresh Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing security scan. The complete latest-head [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808) passes, including budget, browser, Runner, build, and canary jobs. The unchanged Runner close fixture passed on its sole same-head job retry. - Broad local `pnpm test:run` invocations were stopped before required source updates (rebase and review fixes). They are not claimed as full passes. Final full-suite coverage comes from the complete latest-head CI matrix; local focused coverage and the incomplete broad invocations are reported separately. - Reproduce performance with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-enforcement.ts --baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with `--reverse`. The harness uses disposable real PostgreSQL, 20 warm samples per variant/count, and records raw timings, SQL/lock counts, agent reads, unchanged agent state, and query plans. Final production-source measurement revision: `eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is committed in `4c5f4de8b1`. | Unrelated idle agents | Baseline median / p95 | New median / p95 | Baseline SQL / locks | New SQL / locks | New agent reads | | --- | ---: | ---: | ---: | ---: | ---: | | 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 | | 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 | | 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1 | 0 | - These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64. They isolate the cost-evaluation core and complete admission path, without hosted-region or provider latency. Prior-revision reversed-order controls also preserved the constant query counts; the final measurements above use the final production source. - Complete reserve → final ownership gate → immediate offline provider result is measured separately, with one responsible agent and one active operation at each unrelated-agent count. Admission p95 baseline/new: 0 agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents **24.51 / 74.05 ms**. Baseline admission was already scoped; the stronger final fence adds bounded overhead. New counts stay constant at 53 Drizzle statements, three transactions, two company-accounting locks, and five agent-reading statements bound to the responsible agent. Zero unrelated-agent reads, exactly one provider callback per sample, and no leftover runs or holds. All admission scenarios stay below 250 ms. No external provider calls are included. - The partial-usage/delayed-termination fixture passed 20/20: receipt persistence p95 13 ms, receipt return to observed durable Stop fence p95 28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database request p95 1 ms while termination was blocked. After releasing the controlled termination fixture, acknowledgement p95 was 14 ms. These are separately measured phases; fixture acknowledgement is not live-provider termination latency. - Models and provider deadlines are unchanged. No deployment, live inference, or staging acceptance is claimed. A quiet 30-minute follow-up waits for the merged revision on staging, then repeats classifier and controlled budget-stop acceptance in staging, recording hosted bookkeeping, queue delay, and actual provider acknowledgement separately. - Development failures retained: finalizer notification regressions (fixed; 27/27 affected cases passed), invalid browser admission fixture assumptions (fixed; browser coverage now passes 63/63), and first-head lifecycle/dispatch CI regressions (reproduced and repaired). A prior-head Cases E2E test hit its unchanged announcement-dismiss handler race while opening a filter; the same journey passed on `1ac9b58`. The external security review then found a stale-owner race between accounting commit and provider entry; final verification covers a competing run in that gap while retaining provider I/O outside the accounting lock. Early PostgreSQL startup attempts hit local semaphore pressure; serial reruns succeeded. The final-head Runner lane had one failure among 1,823 tests: the unchanged `timeout-attachment` fixture failed at its first process close after 6.28 seconds. The sole same-head job retry passed all 1,819 active tests (four skipped), including that fixture in 1.85 seconds. Three isolated local probes could not run correctly because the Rust fake-provider fixture binary was absent; cleanup masked the original startup failure. CI builds that fixture through its normal lane. These probes are inconclusive and are not counted as passes. ## Risks - This changes accounting, lifecycle, and cancellation ordering. Durable generations, source/sequence checks, ownership leases, and company locking remain required for correctness. - Provider stop acknowledgement can be slow or fail. Stop intent remains pending until existing termination proof succeeds; cancelled row status alone does not acknowledge delivery. - Mid-run limits depend on provider usage reports or supported price calculations. Providers that report only at completion cannot provide exact mid-call detection. - Signals are process-local wake hints for the supported single-active-writer model. Durable state and startup recovery remain authoritative. - There are no migrations, new queue tables, or cached balances. Repeat the classifier and controlled budget-stop acceptance after normal rollout. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and parallel coding agents. The host does not expose the exact backend model revision or context-window size, so those values are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
572bef4ba7 |
fix: omit re-downloadable Codex remote plugins from failover backups (#17026)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native Codex sessions in remote sandboxes copy the sandbox's Codex
home into a host-side failover backup after every turn
> - Ephemeral sandboxes restore that backup at the start of every turn,
so its size affects both disk use and turn latency
> - Most of each backup is remote plugin data that Codex itself treats
as a cache and re-downloads on startup
> - This pull request leaves that data out of Codex checkpoints, keeping
everything a resumed session needs
> - The benefit is backups roughly 80–90% smaller, less data moved per
turn, and much slower disk growth
## Linked Issues or Issue Description
Bug report described in-PR (bug report template). Related: #16866
(expires failover backups after their issues close).
**What happened?**
Failover backups grow quickly because each Codex checkpoint includes
`$CODEX_HOME` remote plugin data. On one instance, 204 backups held 22.1
GB:
- `codex-home/plugins/cache/openai-curated-remote`: 8.3 GB (about 29 MB
per backup);
- `codex-home/cache/remote_plugin_catalog`: 6.1 GB (one 38 MB catalog
file per backup);
- `codex-home/sessions` (the conversation history): 0.31 GB.
About 80 new backups (9.8 GB) were written per day. Ephemeral sandboxes
also copy all of this into a fresh sandbox before each turn.
**Expected behavior**
Checkpoints keep what a resumed session needs and omit data that Codex
re-downloads itself.
**Steps to reproduce**
1. Run a Codex agent with ChatGPT auth in a remote sandbox environment.
2. Inspect
`<instance>/runtime/paperclip-runner/durable-sessions/*/failover-backups/current/codex-home`.
3. Most bytes are under `plugins/cache/openai-curated-remote` and
`cache/remote_plugin_catalog`.
**Paperclip version or commit:** `master` at
|
||
|
DottaandPaperclip
|
92b238137b |
Connect and configure GitHub Copilot through Apps (#15705)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps and AI Connections store credentials and govern access to agent runtimes. > - Users need a supported saved-token connection and config values that survive save and reopen. > - The release must preserve saved-token access and task lifecycle guarantees. > - This pull request will connect Copilot through the normal Apps and AI Connections path. > - The benefit is a reviewable part of the Copilot production release. ## Linked Issues or Issue Description Refs #15701 and #15703. Supersedes setup work in #15562. Base: #15703. Next: #15706. ## What Changed - Require configuration update authority before Copilot credential-bearing model discovery; default to the saved environment and preserve explicit selection for editors. - Keep Copilot discovery passive during reconnect reconciliation; refresh on explicit opening, account/environment changes and reconnect. - Add GitHub AI token connections with encrypted storage, ownership, grants and audit logging. - Keep repository access separate from Copilot runtime authentication. - Probe the selected environment without sending a model prompt and preserve authenticated model selection through save/reopen. - Keep fresh execution admission consistent with normal Copilot setup; persist task model choices without changing provider or permission policy. - Refresh authenticated models after reconnecting the same saved Copilot account, including recovery from a previous authentication error. - Apply the additive provider-default constraint migration through the normal generated migration workflow. ## Verification Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`. History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`. Remaining source [`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d) contains the four-file authorization repair. The remaining PR trees preserve six nonoverlapping files from the newer public mainline managed-settings fix and the CI-owned lockfile. No unrelated provider is enabled. The security finding is repaired: existing-agent discovery requires `agent_config:update` before connection selection or token resolution. The saved environment is the default. Authorized editors can select a prospective environment. Both denied board-viewer and agent-reader regressions fail on the original source. All49 route tests and server typecheck pass with the fix. Denied requests perform no connection selection, credential resolution or probe. Current-head security scans pass on both remaining PRs; the repaired thread is resolved. Copilot CLI1.0.88 and profile47 retain digest `sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`. The [qualification record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json) reports20/20 Product cells,154 assertions and7/7 protocol cases,46 assertions against frozen normal installed source `c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification model is `gpt-5.6-luna`. Every passing Product case confirms cleanup. Original failures remain recorded; automatic behavior retries are disabled. The route-only authorization repair changes no bound runtime/profile bytes and does not relabel the frozen campaign. The ordinary installed remote journey passed saved-token selection, model discovery, normal UI save/reopen, metadata-only Test, completed task, accepted receipt and exact19-byte browser download. An open model catalog caused no additional discovery requests during117seconds. Actual installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta and native Linux x64 on Daytona. Raw token browser entry was not exercised; encrypted connection creation used the normal API. The first Daytona file attempt timed out during delayed semantic dispatch; its precise stalled layer remains unresolved. Independent ordinary remote delivery and the other19 Product cells passed before one instrumented recheck passed8/8 with unchanged source,prompt,assertions and deadlines. No causal latency repair is claimed. Before the authorization repair, all four final PR heads passed required verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads. Current-head CI/review is running for the two repaired heads. Local focused checks retain129 connection,59 runner,72 shared-eval and88 recovery/workspace passes, plus migration safety and token gates. The additive Copilot migration is361; released master history is preserved. [Cloud normal-package run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704) builds the pre-security source4f3f. All four native producers and Linux package assembly pass; the final clean installed consumers are finishing. All18 retained archives match their hashes;37,863 regular files contain zero selected credentials. That package does not contain the later route repair. The actual final published package must include it and pass the local/Daytona live canaries. The frozen Daytona image remains published and independently verified. No separate full-image scan is claimed. Docker builds run in the cloud. **Remaining:** current-head CI/review, setup and qualification merges, final publication, and ordinary local/Daytona canaries against the actual shipped package. The user explicitly authorizes the maintainer review override; CI remains a merge gate. No production deployment is claimed. Provider USD remains unknown and GitHub-attributed. ## Risks Bad credentials, missing entitlement, runtime installation and unavailable models must remain distinct failures. A connection must not adopt ambient credentials. The constraint migration changes accepted provider values, with no credential-storage migration. Current-master NewAgent fixture extraction is retained. Copilot cases use that shared support and test both passive discovery and explicit reconnect under the same setup lifecycle. The onboarding retry regression shares the component suite’s saved-connection, authenticated-model, draft, agent-list, hire and render lifecycle fixtures. Keeping it beside the existing onboarding failures makes adoption versus duplicate hire observable under the same lifecycle; this is the concrete cohesion reason for the existing large test file. ## Model Used OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this task. Used for code changes, review and tool-assisted verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ac9459bae3 |
feat: configure company agent observability (#16874)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Companies use several agent harnesses and execution environments. > - Operators need one place to select where native agent logs and traces go. > - An inference proxy would add a dependency to the agent request path. > - This pull request supplies native OpenTelemetry settings to managed Codex and Claude Code processes. > - Company secrets protect destination credentials. Export failure does not change the task result. ## Linked Issues or Issue Description **Subsystem affected** Company settings, shared contracts, database, server, adapters, and runner. **Problem or motivation** An operator cannot configure a company destination for native agent logs and traces. Manual harness configuration is hard to apply across local and remote execution. **Proposed solution** Add one company destination: Braintrust or a custom OTLP HTTP/protobuf endpoint. Apply a transient snapshot when execution preparation starts. Keep credentials out of API reads, task environment manifests, recovery files, and command arguments. Replace a warm process when its export settings change and preserve its conversation. **Alternatives considered** Multiple destinations can use a user-operated collector for fan-out. This change does not add a collector, an inference proxy, or transcript scraping. **Roadmap alignment** This extends company settings, secret management, and support for multiple harnesses. The maintainer requested this implementation. Searches for observability and Braintrust found no duplicate company destination feature. Related infrastructure tracing work in #16236 has a separate content-free contract. ## What Changed - Add strict schemas, revision checks, encrypted secret storage, activity records, and board-only settings routes. - Add explicit synthetic logs and traces tests with a bounded HTTP request and sanitized results. - Supply native Codex configuration and Claude Code environment values through local and remote launch paths. - Preserve Codex login refresh through the existing managed credential link. Retain unsaved drafts across background reads. Require complete custom header re-entry when the endpoint origin changes. - Separate process export compatibility from conversation identity. Isolate generated Codex configuration and preserve resumable sessions, including remote home refresh after export changes. - Advance the immutable Pi/Copilot source attestations for shared environment and sidecar code changes. Keep their provider payloads unchanged and retain prior identities for decoding. - Redact destination credentials throughout launch diagnostics and HTTP logs. - Add the production settings form, Storybook states and journey, documentation, and feature/access map entries. ## Verification - Real managed local Codex 0.160.0 and Claude Code 2.1.286 tasks exported to Braintrust. API readback and the Braintrust UI showed the test records. A real remote Daytona Codex task exported through the legacy `exec` path. - Live tests exercised content capture off, export disabled, warm conversation reuse, and a destination that accepted connections but never replied. Agent work succeeded and export requests stopped at the configured three-second timeout. The UI connection test returned “Test payload accepted”; the saved API key stayed masked after reload. - All current-head CI test lanes, typechecks, packaging/build checks, and required aggregates passed on `71e994b2be9c8fb6d6e938a4ad7cf4c95fdafe31` ([CI run](https://github.com/paperclipai/paperclip/actions/runs/39319194318)). Greptile completed on that exact commit at 5/5, with zero unresolved review threads. The additional optional Paperclip Review check remains queued waiting for its assigned agent. - Local workspace typecheck and build passed. Focused verification included 239 combined runner tests, 789 API/native-session tests, 107 Codex home/ACP/sandbox-site tests, 204 feature/privacy tests, and the affected Rust suites. Migration, token, map, Storybook catalog, and Storybook build checks passed during acceptance. These checks ran at the relevant implementation or merge boundary; the current-head CI is the complete final gate. - Managed AI credential projections now receive enabled and disabled company export settings; genuine caller-owned Codex homes remain untouched. Two regressions reproduced the skip before the repair. All 276 engine/lifecycle tests and adapter-utils typecheck pass afterward. The launched TOML retains provider/MCP settings, credential continuity, and redaction. - Settings saves, removal, and the real company-deletion owner acquire secret mutation ownership before database locks. PostgreSQL NOWAIT probes cover all three paths; concurrent save/rotation/removal/deletion requests settle correctly. The affected suites passed 137 tests, followed by all 32 service/connection cases after expanded deletion coverage. Server and Codex adapter typechecks pass. - Codex source profiles use the effective CLI argument list. A spawned fixture CLI reproduced both ignored-argument failures; all three precedence/fallback cases now pass. - Real tar staging reproduced remote conversation loss in both API-key and subscription modes before the repair. Both now pass: changing managed settings preserves sessions, archives, native indexes, and refreshed subscription credentials. A corrupt upload preserves the prior config and conversation. - Preserved failure evidence: initial CI caught stale Pi/Copilot source attestations, a missing test-harness helper, and packaging fixture/probe mismatches; those were corrected and covered by focused checks. Superseded heads had intermittent attachment-composer timeouts despite successful uploads; [the original trace and screenshot remain available](https://github.com/paperclipai/paperclip/actions/runs/39309701089/artifacts/12215109203). The same attachment case passed on the final head: [25/25 tests in shard 3](https://github.com/paperclipai/paperclip/actions/runs/39319194318/job/118288637051). - A connection-fault fixture also exposed a stale pooled socket after its hang case; closing fixture responses isolated scenarios and the full affected suite passed. - The broad local test run was not clean: database resource failures, timeouts, an older cached module graph, and a stale optional-resource manifest assumption were recorded. Focused checks passed after the documented fixes. This does not establish a clean full local pass. - Native remote Codex, remote Claude, and the full ACPX Codex bridge remain unqualified. The direct Codex resume test does not establish full bridge acceptance. ## Risks - Native exporters perform network requests from each execution environment. A server connection test does not prove remote reachability. - Prompts, assistant responses, and tool content are enabled by default when supported. Native caps and harness-specific controls still apply. - Caller-owned Codex homes are unsupported for automatic configuration and remain untouched. - Native harnesses and their tool processes can inspect export credentials. The settings form and documentation advise using ingestion-scoped credentials. Retained headers remain bound to their saved scheme, host, and port; changing origin requires fresh values or removal. - Migration 0357 adds a company-scoped row and secret reference. It is safe to reapply. Secret removal and company deletion use the existing secret lifecycle. - Slow or unavailable destinations must not block task settlement. Export settings take effect at the next execution preparation boundary. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and browser verification. The exact serving model ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8193abb936 |
feat(claude): default qualified macOS and Linux targets to Paperclip Runner (#15997)
## Thinking Path > - Paperclip must run the selected agent on its selected host. > - Claude needs verified packages for each supported target. > - A Mac controller must select the correct Linux provider pack. > - New defaults must follow package and execution evidence. > - This PR adds Claude platform packaging and target-specific pack selection. > - Existing Codex Linux ARM64 behavior remains intact. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15993 and #15994. Related scope: #15422 and #15224. ## What Changed - Reuse the merged four-target daemon producer/consumer matrix. Add target-owned Linux provider packs and carry their immutable identities through release assembly and vendoring. - Reuse master’s landed pinned Claude SDK ARM64 executable admission and retain package-integrity, architecture and no-follow admission. - Select target-specific remote packs so Mac controllers can use Linux hosts without a development pack override. - Preserve read-only runtime context on SSH startup and resume without accepting unverified old content. - Check ordinary installed Claude readiness without injected package authority. Retain Codex and Linux x64 consumer checks. - Extend Claude automatic target selection for the remaining qualified package targets. Retain unsupported-platform and external-override behavior. - Qualify managed Daytona snapshot names through provider IDs during Claude Connect. Pin native acquisition to the measured ID. Preserve a revocation marker after failed probes or target edits, and block native acquisition until requalification instead of falling back to a mutable name. Preserve immutable-image proof, company and account boundaries, and Legacy acquisition. ## Verification - Reconciled with master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through the prerequisite stack, retaining the merged startup fix and latest Codex Linux ARM64 behavior. - [Four-target exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/39267321383) passed on source `04e0202ca81387b17cbe7fd094a8338648356fe9`: all four native builds, public-package assembly, Linux x64 verification and clean installed consumers on Apple Silicon, Intel Mac and Linux ARM64. The later master reconciliation adds shared OpenRouter attribution and changes runtime source. This historical result keeps its original source identity; [later qualification](https://github.com/paperclipai/paperclip/actions/runs/39281100929) passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. It predates the subsequent TypeScript startup fix. - Fresh local product acceptance on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` completed a useful file and follow-up through native Claude/Opus 5.5, High effort and the API-key connection. Both actual browser downloads were independently checked; the revision preserves the original body and session continuity. Total task estimates: $0.518432800. Ordinary web and API creation saved defaults with the native experiment disabled. - Local focused verification passed 993 startup/session tests, 243 UI tests, 12 workflow security tests, direct server/UI TypeScript checks, and token/palette gates before the last review changes. Initial DB-backed snapshot pin/revocation coverage passed 18 cases; the final serial route suites pass 122 credential/permission and 232 snapshot/environment cases after correcting a test hook timeout. Current-head cloud suites pass. - Preserved the failed pre-provider local fixture and its full pending reservation. The successful fixture has no unpriced or pending usage. Historical failed Cloud journeys, accounting holds, source identities and subscription limitations remain in the dated plan. - All Docker/Rust release production was performed in cloud CI. No local Docker image was built. An inadvertently invoked local Rust prerequisite was stopped. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - **Experience needs improvement:** successful connection setup still shows “Connection needs attention” because runtime checks are deferred. Cold provider-pack upload took 122–126 seconds per run while the task mainly showed “Thinking.” The tested Core-only preview excludes private Cloud UI plugins; this is not full Cloud distribution or subscription qualification. Desktop dark theme was tested; mobile and comprehensive accessibility were not. The portable report validates, but gallery interaction could not be inspected because host preview was queued. - The optional supply-chain scan returned neutral/inconclusive because GitHub Actions package coverage and infrastructure parsing are unavailable. It is not counted as a successful supply-chain qualification. Exact package, daemon and image integrity checks passed. - A stale, missing or wrong-architecture pack must fail admission. It must not change runner or target silently. - Provider-pack source changes invalidate affected earlier execution results. Existing unknown-cost and cleanup holds remain intact. - Live subscription execution remains unverified under the API-key qualification direction. Built-in authentication flows are retained. ARM64 cloud-image production is deferred. Merge and production deployment remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
45b67f1d93 |
feat: default new Apple Silicon Claude agents to Paperclip Runner (#15993)
## Thinking Path > - Paperclip lets users choose an agent harness and account. > - The server resolves the runner for new agents. > - Claude still needs the same ordinary native-default journey as Codex. > - Existing agents must retain their runner, model and account. > - This PR defaults new Apple Silicon Claude agents to Paperclip Runner with Opus 5.5. > - Users can connect and work without a runner override. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15994 and precedes #15997. Related scope: #15422, #13978 and #15224. ## What Changed - Resolve new Apple Silicon Claude agents to `paperclip_runner`, `provider: acpx`, `acpxAgent: claude`. Keep explicit Legacy, models, provider routing and saved execution. - Use `claude-opus-5-5` for ordinary Claude defaults and retain explicit overrides. - Reuse existing API-key/subscription connections, credential storage and checks. Generalize the shared Advanced runner control with Claude branding. - Carry configured effort into the shared v9 contract. Preserve compatible permissions, instructions, skills, timeouts and session behavior. Require Legacy for unsupported settings. - Qualify a new immutable managed Claude target at explicit Connect through the existing permission-gated probe. Use fresh scoped discovery for test and hire. Discard stale results and require an explicit Connect after settings change. - Retain merged Codex onboarding recovery, auth preparation, import/effort corrections and Grok Bot gates. - Check the Claude first-task wizard's saved default directly. Update configuration/runtime documentation and exact Opus 5.5 accounting. - Align hire fixtures with the merged host-environment restriction. Native agent-authenticated hires cannot inject host env settings; both creation endpoints verify that no extra agent is saved. ## Verification - Updated onto master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through #15994. Native input v9 carries effort while preserving v7 home modes and v8 Muse. Codex defaults remain unchanged. - Focused local verification passed 285 server tests, 323 UI tests, 124 setup/registry tests, and two v9 permission-pin transport cases. Direct server/UI TypeScript checks passed. An initial 30-second hook timeout was resolved by serial verification with a longer hook budget: all 122 credential/permission cases and 232 snapshot/environment cases pass. Direct server TypeScript checking also passes. - Ordinary web creation with the experiment disabled saved native Claude, Opus 5.5 and High effort without runner/model overrides. The actual product task and follow-up both succeeded on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` with the cloud-built daemon from `04e0202ca81387b17cbe7fd094a8338648356fe9`. The final master reconciliation also changes shared runtime source. Historical proof retains its source identity; a final-source cloud qualification is running before refreshing the preview daemon. - Both actual browser downloads were checked: v1 has three sections/six checkboxes (451 bytes); v2 preserves the original body and adds two Rollback checks (612 bytes). The original attachment stayed byte-identical. The follow-up records session resume and the same provider session. Total estimated API usage: $0.518432800, with no pending/unpriced usage in the successful fixture. - Fixed the live-discovered setup incompatibility: native API-key validation no longer depends on the global Legacy Claude CLI. The existing provider endpoint verifies the key without a paid prompt; subscription/custom routing probes remain intact. - Fixed security review's native-hire credential redirection: agent-supplied URLs, proxy/launch settings and other host environment controls are rejected before inheritance. Native credential references remain supported and partial edits use the saved harness. Added create/hire/update regression coverage. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - Docker and native release builds ran in cloud infrastructure. An inadvertent local Rust prerequisite was stopped; no local Docker build was run. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - Missing packages, authentication rejection, unsupported settings and unavailable models must fail explicitly. Existing agents retain saved execution; explicit runner changes use existing revision/session invalidation. - Subscription follow-up remains unverified; remaining live acceptance uses API keys per human steering. Historical accounting holds and the separate $50 ceiling remain intact. - The subscription acceptance limit remains explicit for review. No Claude database migration is introduced. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f3cd3f4ea8 |
fix(runner): require confirmed exit before remote replacement (#16835)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote runs need one runner to own their process identity and listener. > - A health request can fail while that runner continues to work. > - The active monitor treated a failed request as a process exit. > - Recovery could then launch another runner against the occupied listener. > - This pull request retries health reads and requires proof of exit before replacement. > - A brief outage can recover on the same runner without replaying agent work. ## Linked Issues or Issue Description Refs: #16341 adds remote recovery diagnostics. Refs: #16686 retries verification when restoring a saved remote session. This change applies the shared probe policy to active process supervision. **What happened?** The active remote monitor settled its completion promise after a timed-out or rejected health request. The transport treated that result as an exit and started process recovery. An existing runner could still own the listener. Replacement then failed with `runner_ingress_bind_conflict` until the reconnect deadline expired. Missing or changed ownership markers also did not prove that the old process had exited. **Expected behavior** Retry transient read failures on the same runner. Require a matching ownership marker and proof of process exit before replacement. Preserve uncertainty when identity or liveness cannot be verified. **Steps to reproduce** 1. Start a remote runner and leave its listener alive. 2. Make one process-health request time out or reject. 3. Observe completion and the restart decision. Before this change, the monitor settles while the process is alive, and the restart wrapper permits another launch. **Paperclip version or commit** The defect was reproduced on master at `6e89919631`. The patch is based on current master at `c6a0cf4bce`. ## What Changed - Reuse the bounded read-only recovery probe for active supervision. Retry up to three reads within 30 seconds. - Check the saved process generation when available. Accept only the specific process-not-running receipt as exit proof. - Add an exit-confirmation contract to remote process handles. Require it for restart and shutdown completion. Preserve the cause of supervision errors. - Test transient failures, exhausted retries, conflicting identities, real listening processes, and an authenticated runner with a fake provider. Keep confirmed-exit replacement covered. - Document the recovery policy and coverage limits in the execution contract and maps. ## Verification - Red: the timeout and transport-error regressions failed before the patch because completion settled while the runner was alive. The restart-guard regression also failed because replacement was allowed. - Green: 14 server supervision cases passed, including a real listening process. All 120 shared control-plane tests passed. Focused Runner tests passed for authenticated live-runner fault injection and confirmed-exit replacement. - Review regression: a replacement-health test first failed because its error cause was lost. It passed after preserving the cause and detail. The four affected transport cases and Runner typecheck then passed. - `node scripts/check-maps.mjs` passed for 35 areas and 176 capability IDs. - `pnpm -r typecheck` and `pnpm build` passed locally. The sequential local full-suite run was stopped before completion; no local full-suite pass is claimed. Final commit `bcc1b1d339` passed the complete CI matrix: 55 checks passed and two optional Storybook checks were skipped. Greptile is 5/5 with no unresolved conversations. The PR has no merge conflicts. - The first real-listener attempt could not bind inside the execution sandbox. The same test passed with local socket access. The first package-test invocation used the root configuration and selected no tests; the package configuration ran all selected tests. - CI attempt 1: nine test jobs were cancelled together at 12:54:59 UTC. Their test steps report cancellation. The build and uninterrupted jobs passed. The interrupted jobs were retried once. That retry exposed one confirmed-exit message assertion failure in the Runner shard; the other jobs passed. The compatibility correction preserves the prior message. The failing case and four recovery cases then passed locally. The complete CI matrix passed after that correction. - No live sandbox restoration, deployment, or historical task recovery is claimed. ## Risks - A persistent outage now ends supervision with explicit exit uncertainty. It cannot authorize a competing process. Existing ownership and workspace recovery checks remain required. - Health reads can take up to 30 seconds before reporting an unavailable runner. Retries are read-only and bounded. - Local launchers keep their existing completion contract. No database migration or public API change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact serving snapshot and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3858d2341d |
fix(recovery): default saved-monitor waits and keep task recovery quiet (#16686)
## Thinking Path > - Paperclip manages AI agents and their work. > - A task must keep a durable path to its next action. > - A provider can save a task monitor, then fail to yield because it omits internal continuation fields. > - An interrupted provider-pack check can then be reported as permanent artifact incompatibility. > - The task asks its user to certify recovered workspace files, even though this is an operator responsibility. > - This pull request derives the saved monitor continuation, bounds verification retries, and removes that task form. > - Normal recovery stays quiet. Unresolved holds retain inspection and the existing safety checks. ## Linked Issues or Issue Description **What happened?** A yielded completion without `continuation` is rejected even after the provider saves an eligible monitor on the task. Repeated attempts cannot finish. Remote verification also treats timeouts and missing exit status as artifact incompatibility. The task recovery notice exposes raw diagnostics and asks the task user to certify infrastructure files. **Expected behavior** Use current persisted task state to fill routine continuation fields. Retry interrupted verification reads within a fixed bound. Keep normal recovery in the working state. Show a compact notice only when work cannot safely continue. **Steps to reproduce** 1. Save an eligible monitor on the active task. 2. Call `paperclip_finish` with `reportedWorkDisposition: "yielded"` and omit `continuation`. 3. Return a timed-out remote provider-pack command with empty stderr during recovery. Inspect the resulting task notice. **Paperclip version or commit** Source baseline: `aaeff418cf` (master integrated with its current task visibility and saved-source reconstruction rules). Related changes: #15824, #16104, #16451, #16493, #16690. This fix complements saved-result recovery and verified pack reuse. It does not replace their workspace ownership checks. ## What Changed - Allow omitted or null continuation at the provider input boundary. Read current authorized task context and fill a monitor continuation only for the bound run and an eligible saved monitor. Keep accepted results strict. Apply this to Codex, OpenCode, both direct and production-sidecar ACPX paths, and external drivers. Reserve the semantic call identity during the saved-context read and reject cancellation or a changed run/turn. - Bind the shared completion change and continuation helper in a new Copilot v21 source declaration. Preserve v20 history and its decoder support. Qualification stays pending. - Expose the eligible continuation in task context. Completion admission still checks persisted monitor state. No schedule is created or renewed by the default. - Retry read-only provider verification at most twice per command. Hold exhausted reads with a distinct verification-unavailable cause. Reattachment cannot renew the read budget through an unchanged provider-attempt counter. Keep completed verification failures and wrong versions incompatible. - Remove the task file-certification form. Hide the notice only for a matching durable recovery path without an unknown-action replay hold. Keep missed retries, unsafe outcomes, and unresolved file holds visible. - Add production-component stories for automatic recovery, blocked and expired holds, reader access, persisted repair decisions, source reconstruction, queued continuation, and mobile. Preserve master’s guarded saved-source reconstruction choice after confirmed source destruction. Update the execution contract and access maps. ## Verification - Completion, helper, Codex, OpenCode, ACPX, and profile regressions pass. After the sidecar review fix, 271 tests across the production sidecar and three provider suites pass, plus 72 helper/profile tests. The real sidecar MCP bridge tests cover saved-monitor acceptance, no-monitor refusal, cancellation, duplicate call identity, and raw-versus-forwarded semantic receipt hashes. The OpenCode tests also cover eligible-monitor acceptance and refusal without one. - The fresh full native executor suite passes all 743 tests. Four profile, installation-integrity, and ACPX driver suites pass all 168 tests. The earlier CI fingerprint failure required a new immutable source declaration; its follow-up Docker failure required the matching context allowlist entry. Both fixes are verified locally. Targeted server monitor, tool authority, and provider verification suites pass. The probe tests include a real timed-out subprocess, an unrelated responsive timer, transport errors, and completed integrity rejections. - All 90 tests in the merged task notice, reconstruction choice, recovery display, and lineage suites pass. They cover reader access, live/missed/mismatched recovery, unknown outcomes, saved messages, retry refusal, and removal of the file form. Both review findings have failing-baseline/passing-treatment regressions: identical polling across retry expiry, and a failed refresh after cached running activity. - Browser walkthrough passed for automatic recovery, unresolved details, a 390px mobile story, and the merged saved-source reconstruction and queued states. The reconstruction control starts disabled and normal automatic recovery shows only Working. The stories use production components and fixture API data. They do not prove live cloud recovery. - Token gates, map checks, and Storybook catalog generation pass. The exact Docker context integrity command passes locally after adding the new build declaration to its allowlist. - Workspace typecheck and build pass on the final source, including the versioned identity update. The final server and UI guards pass their focused tests, typechecks, builds, and token gates. The final standalone Storybook builds. The saved-monitor default retains master’s stricter child-task visibility filter. Re-review caught an additional production-sidecar validator before forwarding; the fix now defaults at that boundary and has passing regression controls. After preserving master’s new reconstruction capability, 68 targeted reconstruction, tool-authority, and provider-verification server tests pass. The current-master integration at `2a80050a0d` has a fresh 5/5 review with zero unresolved threads. Its complete CI matrix, including Runner tests, is the merge acceptance record. - On the final master integration, local typecheck, build, and all 90 recovery UI tests pass. A concurrent runner run hit five 5-second sidecar startup timeouts; an isolated sidecar run passed 86 tests and timed out in the unrelated unsupported-bootstrap case. Those local deadline failures remain recorded; the canonical CI Runner lanes use their normal deadlines and pass. The unsharded local full-suite attempt did not complete. It spanned source edits; the fresh native executor run above passes. An unrelated workspace fixture timed out once and then failed to reserve 32 contiguous ports on this host. Its unchanged-base control and CI server suite pass. Full-suite acceptance uses the fresh latest-head CI matrix; this PR does not claim a passing unsharded local run. ## Risks The default applies only to an eligible persisted monitor. An explicit invalid continuation still fails validation. The added read occurs only when a yielded report omits its continuation. Admission rechecks current state after that read. An interrupted verification command can add one extra 30-second probe. Completed rejections stay closed. These probes do not change the pack, launch provider work, or replay external tools. Two inconclusive reads hold the original run. They do not schedule another reattachment with an unchanged provider-attempt counter. This PR’s pending Copilot v21 declaration binds the changed shared contract, resolver, direct driver, and production sidecar sources. The previous v20 declaration is immutable and remains decodable, but launch requires the new exact identity. Qualification remains pending. Task users lose the file-certification form. Board-authorized source-only repair remains available through the resolution API with the original evidence and stop-receipt requirements. This branch does not deploy or recover historical held runs. > This is a bug fix and UI polish within existing recovery behavior. ROADMAP.md was checked. ## Model Used OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning, with tool use and code execution. The host does not expose the context-window limit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed-area suites; the full local run limitation is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e6dcc6b095 |
Connect personal Muse through secure native execution and Vault OAuth (#16448)
Add company-scoped Muse native control, OAuth, wake receiver and recovery. Recheck current authority after waits and before effects or receipt replay. Preserve task-read scope, assignment expiry, independent holds, nonblocking reassignment and truthful cleanup provenance. Retain source trust on progress comments and verify current Muse V8 authority through real signed-JWT HTTP requests. Preserve the upstream trimmed ordinary authentication lookup. Refs #16448. Preserve captured canonical queue recovery and concurrent environment preparation while retaining the scoped Muse V8 exclusion. Normal migration 0354 and existing Muse SQL remain unchanged. Fresh current-head Greptile review is 5/5 with the project-write finding addressed. Required CI and independent technical review gate the merge; maintainer override is reserved for formal approval rules. Recheck current Muse project-write authority after all receipt/task/run/admission waits, before creation or replay. Real PostgreSQL regressions reproduce four late effects/replays on the old route; the repaired three-file treatment passes 354 cases with no skips, and the scoped server typecheck passes. Ordinary/Dot behavior is preserved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0528cc7ee0 |
perf: overlap native GitHub and assigned-tool setup (#16620)
Prepare managed GitHub access and fresh assigned-MCP discovery concurrently for remote native Codex. Wait for both before provider construction and drain both on failure through the existing run cleanup. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b294abb9eb |
perf(server): append ordinary run events in one statement (#16569)
Ordinary run events now validate their binding, allocate a sequence and insert the decoded receipt in one atomic PostgreSQL statement. Native replay and retry-exhaustion transactions keep their existing behavior. Verification: exact-head full CI/E2E, Superagent, legacy security and Apex 5/5 passed. PostgreSQL concurrency, rollback and payload controls pass. Controlled network-delay comparison reduced median event persistence from about192ms to48ms; no staging response-time claim is made before deployment. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
fa9f3f8a08 |
feat(runner): support preconfigured user homes for Codex and ACPX (#16486)
## Thinking Path > - Paperclip manages AI agents and their work. > - Runner launches the selected provider and keeps durable session state. > - Runner currently prepares private provider homes for each session. > - Operators can already have working login, skills, and MCP configuration on the execution machine. > - This PR adds an explicit mode that uses that existing setup. > - Runner keeps its task bridge and session ownership while leaving the user's configuration in place. ## Linked Issues or Issue Description **Problem or motivation:** Rebuilding a provider home is unnecessary when the execution account is already configured. **Proposed solution:** Add `homeMode: "preconfigured"` to Runner adapter config and runnerd transport options. Keep `"isolated"` as the default. Initially support native Codex and ACPX Claude. **Alternatives considered:** Continue copying user configuration into a private home, or add an approval workflow. Both add work outside this feature's scope. **Roadmap alignment:** This extends the existing cross-provider Runner. It does not add another runtime. Related work: #14056 covers managed Codex credentials. #14474 covers command environment delivery. This mode uses user-owned credentials and retains the existing launch boundaries. ## What Changed - Add a closed v7 launch contract for the selected home mode. Existing contracts retain their behavior. - Keep homeMode under operator control. Reject task overrides and agent-authenticated changes, including removal through replacements, adapter changes, or rollback. Ignore stored task values and exclude it from automatic hire inheritance. - Skip managed AI login, assigned-skill materialization, and external MCP setup in preconfigured mode. Preserve the Runner task/completion bridge. - Keep protocol state and provider lifetime fences scoped to the Runner session. Different sessions can share a home. - Fail preconfigured ACPX planning/read-only launches before provider admission until a read-only user-home launch is qualified. Use isolated mode or native Codex for planning. - Reject ACPX approval requests. Keep native Codex on `never`. Local permission rules and hooks remain effective. Initially require the default Runner ACPX `approve-all` setting; other Runner policies are unqualified in this mode and fail before launch. - Keep the internal ACPX bridge in a stable session namespace, preserving an existing user MCP server named `paperclip`. - Reject preconfigured ACPX Codex before launch until a session permission profile can enforce Runner-state denial. Remove the unprotected full-access mode switch; use native Codex or isolated mode for that model family. - Bump Pi to profile 23 because its attestation covers shared Runner source changed here. Preserve profile 22 for historical recovery, and update Docker/package verification inputs. - Fence recovery on mode and home bindings. Fail missing provider-history recovery without starting a replacement session. Exclude the external Codex home from backups and credential cleanup. - Discover remote HOME on the execution machine. Do not project controller HOME or XDG paths onto it. - Separate task instructions from Codex ACP isolation and update the generated provider digest. - Keep remote configured-environment manifests consistent when dropping controller paths. Preserve exact Claude model IDs through per-launch environment settings, without editing the user home or patching the provider. ## Verification - Targeted local checks cover native contracts and launch allowlists, user-config preservation, provider admission, permissions, session recovery, remote environment projection, heartbeat preparation, and operator-only configuration. Runner, shared, and server TypeScript checks and the Rust binary build pass; generated ACPX profiles are current. - Following the latest master integration, all 757 server configuration/executor tests and 115 Runner admission/recovery tests passed. Additional review-fix checks passed: 103 planning contract/host tests, 78 provider-backend tests, 95 route/preparation/inheritance tests, 43 task-validator tests, and 43 workspace-policy tests. Counts describe separate runs and can overlap. - The real pinned Claude ACP/SDK regression reproduces exact-model aliasing without the fix (`claude-sonnet-5-5` becomes `sonnet`) and passes with the fix. All 89 adapter tests pass. User settings remain byte-for-byte unchanged. - Localhost-only Claude SDK probes with synthetic credentials verify the private bridge runs, local `ask` reaches the existing reject callback, local `deny` blocks without a callback, and a user MCP server named `paperclip` remains available. No user model allowlist is required. - Review regressions cover fail-closed provider admission, saved-mode recovery before routing, controller-home path removal without configured-environment manifest drift, pre-launch rejection of unqualified read-only ACPX launches, and denial of task/agent changes to the operator-owned home mode. - Before rebase, `d4d4ee2aebe4cb88c4f4d9ef8e22fed4b36b87be` received standard Greptile 5/5 with no unresolved threads. Prior Apex findings are fixed and covered by regressions. No manual Greptile retrigger is requested for this rebase, per reviewer direction. - The prior PR head and master hit the same GitHub runner encoding crash in workspace-b on three PR attempts, including diagnostic logging ([prior PR CI](https://github.com/paperclipai/paperclip/actions/runs/38818474563/job/116714538405), [prior master CI](https://github.com/paperclipai/paperclip/actions/runs/38819528748/job/116712423193)). Rebased onto `origin/master` at `c3ffa3f0dd` to pull in its bridge-test index correction. The rebased head `edd153db410bea063b645e0000aa68649f287b00` passes all CI checks ([CI run](https://github.com/paperclipai/paperclip/actions/runs/38824676914)), including workspace-b. Greptile's automatic standard review is 5/5 on this exact head, with no unresolved threads; no manual review was requested. - Local workspace-b investigation passed 911 shared tests and 3,868 tests across the remaining 15 workspace projects. The catalog run separately hit an existing ignored `.agents/skills/design-guide/SKILL.md` description-budget violation in this checkout; that file is untracked, outside the PR, and left untouched. Repeating shared tests with CI reporting enabled also passed. - Rebase verification: the final Git tree exactly matches the previously reviewed branch combined with current master. All 192 Runner admission/contract tests, 61 server preparation/provider tests, and 68 bridge tests pass, as do Runner/server TypeScript and generated-profile checks. The real Claude adapter fixture hit an `ENOTEMPTY` cleanup race on its initial run; all 89 adapter tests passed on retry. - Apex on rebased head `edd153db` returned 2/5: ACPX Codex selected full access without enforcing Runner-state denial, and agent configuration replacement could remove the operator mode. Both are fixed in `26e14dbbf`. Preconfigured ACPX Codex is temporarily unqualified; re-enabling it requires session permission enforcement on thread/turn operations and a real filesystem-denial probe. The config guard covers effective replacements, adapter changes, and rollback while allowing ordinary agent merges and operator removal. All 190 focused Runner tests, 103 server tests, both typechecks, and generated-profile checks pass. Standard review and CI passed before the next Apex request. - CI on `26e14dbbf` passed 1,967 Runner-shard tests but hit the previously observed `ENOTEMPTY` fixture cleanup race after successful Claude model admission. `eb83284fb` adds bounded filesystem retries to that fixture's teardown; all 87 adapter tests pass. Assertions and production cleanup behavior are unchanged. Head `eb83284fb42ee19a15d9a93b106c1bbb07e1695e` passes all 59 CI checks ([CI run](https://github.com/paperclipai/paperclip/actions/runs/38829500857)) and standard Greptile at 5/5 with no unresolved threads. Apex also completed at **5/5** on that exact head ([Apex result](https://github.com/paperclipai/paperclip/pull/16486#issuecomment-6201559527)). All review threads are resolved, the branch has no merge conflicts, and the PR remains unmerged. - No live paid-provider or remote-machine smoke was performed. The lockfile remains unchanged under the repository's refresh-bot policy; local dependency installation resolved the changed patch hash. ## Risks This explicit opt-in grants Codex tools access to the execution home (read-only in planning), including credentials. Native Codex separately denies Runner-owned protocol state. ACPX Claude retains its existing provider trust boundary, and the isolated default is unchanged. Providers may refresh credentials and update their own metadata. User configuration can block tools or the completion reporter. Existing blocked results and failure diagnostics remain the recovery path. Edit the local settings and retry; no new approval workflow is added. External provider history remains on the execution machine and is not backed up by Runner. Remote HOME discovery requires `getent` and fails explicitly when unavailable. Other harnesses need separate qualification. The Codex ACP bundle digest changes. Existing sessions tied to the old bundle may need a fresh session. No database migration is required. ## Model Used OpenAI Codex, GPT-6, with reasoning and code execution. The exact runtime model ID was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the feature issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3898f6a45f |
fix(native): honor duplex transport for GitHub callback bridges (#16455)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native remote runners use a session-owned broker for managed GitHub operations. > - That broker uses the existing authenticated sandbox callback bridge. > - The native caller omitted the server-selected duplex flag. An enabled and capable target still used file polling. > - Warm sessions must observe a changed flag and channel loss before the next admission. > - This pull request forwards the existing flag and retires unavailable or changed bridges before retained instruction preparation. > - The benefit is less idle remote traffic without changing credential authority or the default transport. ## Linked Issues or Issue Description **What happened?** A native GitHub callback bridge selected `queue_v1` even when its sandbox target had duplex enabled and supported it. The file bridge continues remote queue checks while the warm session remains available. **Expected behavior** An enabled, capable target should use the existing HTTP/2 bridge. Disabled or unsupported targets and failed channel startup must retain the authenticated file fallback. A changed server flag must take effect at the next normal warm-session boundary. **Steps to reproduce** 1. Create native GitHub access on a sandbox target with the server-selected duplex flag enabled, verified duplex capability and a working channel. 2. Observe the selected callback transport. 3. Before this fix, it is `queue_v1`. The added real-wire regression expects `http2_v1` and fails on the old source. ## What Changed - Forward the existing server-selected duplex flag through the native GitHub broker. - Record that transport selection on the broker. Retire the warm provider and broker through the existing cleanup path when the flag changes. - Read live channel failure when checking broker readiness. Mark normal session cleanup as an orderly completion. - Include the selected flag in instruction preparation identity. Retire a failed broker before adopting retained instructions, while preserving existing ownership and cleanup fences. - Add real local HTTP/2 credential-rotation and file-fallback controls. Add native warm-session flag-change controls. - Document selection, fallback and warm-session behavior. - Scan CI secret expressions in one forward pass, including quoted braces and escaped apostrophes. Preserve direct, bracket, multiline and whole-context references while rejecting filename and literal-string false positives. ## Verification - Current head `41440b07ea45e9e882e36ed495a66e501c8dd4df` resolves the actual CI-policy conflict with master `307eb16c0c59abd9e0626fde9201939b262e7f5f`. It retains upstream's stricter whole-expression credential guard, including quoted secret text, and uses the forward scanner instead of a backtracking regex. All **16 CI policy tests** pass, including the bounded 28- and 10,000-apostrophe controls in both policy modes. The reported old regex still has its recorded two-second timeout (14 passed, one failed). On this final merged source, **809 affected native cases** pass with zero selected skips (65.20s). Full repository typecheck (31.44s) and build (22.13s) pass. Frozen source hashes stayed unchanged. Fresh exact-head full CI, configured security and completed Apex remain pending. - Before master integration, the frozen native repair passed repository `pnpm -r typecheck` passed (33.0s) and `pnpm build` passed (32.1s). - Before the latest master integration, all **800 affected cases passed with zero selected skips**: complete native executor (725), resume (50), GitHub broker (16), environment duplex-gate (4), and heartbeat instruction-copy (5) suites. The final run took 95.3s. Source hashes stayed unchanged through these checks. - The initial head passed 788 affected cases and full CI, but Apex scored it **3/5**. It found a dead HTTP/2 broker could remain reusable and a flag change could fail after retained instruction preparation. Both findings required the new repair. Three added before-fix regressions failed on channel availability, orderly close, and preparation identity. - The first repair run failed three new warm-instruction fixtures because fake provider close left its recorded state ready. The corrected fixture writes a suspended checkpoint without weakening the production ownership guard. A separate real-wire run then failed two availability assertions because its helper spread had frozen the readiness getter. The helper now preserves the getter. All failed attempts remain in the local evidence. - One disposable real Daytona sandbox exercised both transports on the initial transport patch through the production native broker and callback bridge. The provider shell wrapper used the then-current production declarations. During equal two-second idle windows, file mode issued **9 remote commands** and HTTP/2 issued **0**. Both variants rotated synthetic credentials between two run bindings and denied idle requests. Cleanup deleted the sandbox. - Daytona component setup took 3.64s for file mode and 4.74s for HTTP/2. This is an idle-traffic reduction, not a startup-speed claim. That component run predates the availability and preparation repair; it does not qualify the repaired paths on Daytona. The new failure controls use a real local HTTP/2 channel. The credential resolver was synthetic and no model was called. Real GitHub-account grants, staging replies, and the end-to-end latency target remain unverified by this component test. - Repaired-head CI found a policy-test false positive: a broad `secrets.` regex matched an upstream filename. The unchanged test reproduced the failure, and the corrected version passed all 15 controls. Master later supplied a stronger direct/bracket check, which we retained when resolving the merge conflict. Review then found that formatted expressions could stop that check at a quoted brace. The new reported-case baseline failed (one failed, 14 passed). The final check treats quoted strings as complete tokens, respects expression boundaries, and passes all 15 cases. The later master conflict also added `toJSON(secrets)` controls. The old branch pattern failed that whole-context case. Review subsequently found exponential backtracking with escaped apostrophes; the final forward scanner retains the quoted-boundary and whole-context controls and passes all 15 tests, including the bounded long-expression regression. No workflow or credential handling changed. All failed attempts remain recorded. - The full local repository test suite was not rerun. [Prior-head full CI](https://github.com/paperclipai/paperclip/actions/runs/38775801821), including `ci / verify` and `ci / e2e`, passed on previous head `e5f8bdb425081cf08110c0f7423f96609e405920`. All 52 check runs succeeded, two optional Storybook jobs skipped, and legacy Snyk passed. Completed Apex review is **5/5** on that exact head, with no unresolved review threads. ## Risks - Only an explicit server-selected opt-in can use HTTP/2. The existing provider-capability, channel-startup and authenticated file-fallback checks remain in the shared bridge. - Toggling the flag or losing the HTTP/2 channel replaces an idle retained provider at the next run boundary. Instruction preparation must make the same retirement decision. Normal ownership, settled history and cleanup checks remain in effect. - Lower idle traffic may reduce provider/controller pressure, but no staging latency improvement is claimed. This feature remains inactive when the existing flag is off. Other preparation and finalization costs remain. - No schema, API, UI, credential-selection, tracing endpoint or default-setting changes. ## Model Used - OpenAI Codex, based on GPT-6, with tool-assisted reasoning and code execution. The exact hosted snapshot ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
db11d899cf |
fix(native): recover remote sessions while preserving warm runs (#16451)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents keep remote processes and conversation history across warm turns. > - A completed turn can leave handoff authority only in controller memory. > - Losing that controller can block a follow-up even while its sandbox survives. > - This pull request verifies durable ownership and reconnects the exact prior process before handing it to the next run. > - Healthy follow-ups stay warm. Recovery preserves saved work without replaying a completed turn. ## Linked Issues or Issue Description **What happened?** A remote warm turn can finish without suspended backup authority. A replacement controller rejects that prior owner even when its exact process and completed checkpoint remain. A single unavailable identity probe can also stop an otherwise valid reattachment. **Expected behavior** Keep healthy follow-ups warm. Recover a completed session after proving that its controller stopped and that the successor holds current authority. Reuse compatible processes; otherwise save and resume the conversation. Retry unavailable identity reads within one shared deadline. **Steps to reproduce** Use `doc/testing-native-remote-continuity.md`. Its Linux cases execute real turns through PostgreSQL, TLS ingress, and the Rust runner. They kill a separate controller process, recover old checkpoint metadata, and change process configuration. The provider is deterministic. Related: #16432 and #16441 fix live warm-owner handoffs. #16341 adds recovery diagnostics. #14695 covers provider process retention. ## What Changed - Preserve warm process reuse between healthy remote turns. - Verify a committed prior result, stopped controller, matching checkpoint and sandbox, and current task/coordinator ownership before recovering an orphaned warm session. - Reconnect through the prior run route, then use the existing authenticated run attachment to change authority. - Persist a process-configuration digest. Close and save recovered sessions when old or changed configuration cannot support reuse. - Retry unavailable state, marker, and process reads within one 30-second deadline. Keep signals and definitive identity failures single-attempt. - Record closed recovery diagnostics, including unreadable or malformed saved state. - Retire the recovery-only ownership check after verified attachment, so later warm turns use their current authority. - Add real-process Linux regressions and refusal controls. - Expose the existing source-only file-recovery form for retained native holds, with the same board authority, evidence, and exact stop-receipt requirements as legacy holds. Recording recovered files does not change task status or replay a run. ## Verification - `pnpm -r typecheck` passed; the server check was repeated for the third-turn fix. - `pnpm build` passed after the final UI change. The UI typecheck, token gates, and map checks also pass. - Four Linux continuity cases passed: warm reuse, controller loss, legacy checkpoint metadata, and changed configuration. Each creates one conversation and starts exactly three turns, including a healthy follow-up after recovery. The fixture uses expired prior leases; the third turn keeps the runner and provider processes. - Greptile caught the retained recovery gate; the third-turn regression now covers it. An initial test omitted the heartbeat’s saved input and left a test process after failure; the fixture was corrected and its disposable container reset before the passing run. - The current focused executor/restart group passes all 749 tests, including the original failure classifications. - Native file-recovery and UI tests pass (52 tests), including blocked tasks and both native and legacy holds. - After merging current master and retaining its equivalent CI expression check, the CI policy tests pass (15 tests) and all four three-turn Linux continuity cases pass again on `dec418ae55ff21b4fb067132613658cce445f181`. - `pnpm test:run` completed its general server group with 20,572 passes, 103 skips, and three failures: the skill-audit child process and two HTTP test timeouts. The same three suites then passed in isolation (89 tests) without code changes. That local command stopped before later groups; the complete final-head CI, including general, serialized, Runner, build, and browser gates, is green. - Greptile reviewed the combined head `dec418ae55ff21b4fb067132613658cce445f181` at 5/5 with no actionable findings; both earlier findings are resolved. All final-head CI gates and the security scan passed. Staging deployment and retained-incident disposition remain deployment acceptance work. ## Risks - Recovery requires exact durable evidence. Live conflicting controllers, replacement sandboxes, Stop/pause, and uncertain effects must remain blocked. - After controller loss, tool-contract or process-configuration changes can restart the provider. Normal healthy follow-ups retain their processes. - The new idle-recovery path has Linux fake-provider evidence. Live Codex/OpenCode and retained quarantined-state recovery still need staging qualification. - No database schema, telemetry, or runner protocol changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution, and browser tools. The session does not expose a more specific model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks and isolated reruns; full-run limitation recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on `dec418ae55ff21b4fb067132613658cce445f181` - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
afe5b3418d |
fix(native): verify warm ownership before configuration rotation
Track the exact prior warm-owner run ID separately from mutable configuration. Keep configuration retirement, terminal-state checks and failed-close fences. Add current-v5 handoff regressions and document the recovery boundary. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
d5b1975621 |
fix(native): serialize warm idle expiry with turn admission (#16432)
The warm idle timer no longer closes a warm native session while a continuation has reserved or is preparing the same session scope; it re-arms instead. An idle close that has already started is registered in closingWarmNativeSessions so turn admission waits for the checkpoint, matching the other closers. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
3ea39f75c1 |
test: consolidate duplicate owner proofs (#16075)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ff0f976de1 |
fix(native-runtime): retain remote recovery failure reasons (#16341)
Preserve closed marker-read and process-verification reasons through native recovery failures and operator reports. Keep existing transport errors and reattachment ownership checks. Add shell, process-probe, persistence and privacy regression coverage. All current-head CI checks pass and Greptile is 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ebceb2a744 |
trace(server): time workspace, native setup and startup recovery (#16330)
## Thinking Path > - Paperclip manages agents and their work. > - The controller prepares workspaces before it starts a Runner. > - A slow run can spend 30–80 seconds inside the combined workspace phase. > - Startup recovery can also delay queued work for minutes. > - Native runs also have gaps before Runner startup and after the provider result. > - Existing traces do not identify the awaited steps inside those intervals. > - This change adds closed timing boundaries around the existing operations. > - Operators can locate the wait without changing execution or exposing content. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operator-configured OpenTelemetry diagnosis of controller preparation and startup recovery. Related: #16178, #16236 and #16285 introduced the existing preparation timing and sampling contract. #16306 and #16312 reduce separate configuration and copyback work. **Current behavior** The workspace interval combines source resolution, trust checks, provisioning, database persistence, binding and repository admission. Startup awaits accounting and native recovery before queued work resumes, but neither has a separate duration. Native backend setup and post-turn cleanup also contain unmeasured awaits. CPU profiles and a fast database probe cannot identify which await is slow. **Proposed behavior** Record workspace steps and native controller setup and cleanup with the existing hashed run ID. Record startup phases as independent, operator-sampled root spans without custom attributes. Keep the same operation order, results, failures and authorization. **Reason and benefit** These boundaries distinguish source work, database writes, repository admission and restart recovery. This is a diagnosis change. It does not claim a response-time improvement or a solved latency budget. ## What Changed - Add 14 closed workspace step names around 16 existing awaits. Repeated updates and bindings produce separate spans. - Time startup tool reviews, gateway sessions, delivery-worker readiness, accounting, plugin readiness, native recovery, hot restart, orphan reaping, retries, queued work and goal recovery. - Time native GitHub setup, wake binding, backend construction, connector assignments, MCP discovery, tool authority, companion selection and binary resolution. Time selected post-turn GitHub, coordinator and instruction cleanup awaits. - Move preparation timing to a pure shared module. Keep the existing heartbeat import as an alias. Both heartbeat and native timing skip tracer lookup when the endpoint is unset or blank, including when another library registered a provider. - Tracing lookup, callback, status and end faults cannot repeat work or replace its value or error. - Add real PostgreSQL workspace timing coverage and silent no-op/fault controls. Update the tracing contract and paired run maps. - Keep span attributes closed. Startup spans add none; workspace steps use only the existing run hash. No commands, paths, returned records or exception messages are recorded. ## Verification - Diagnostic-gap baseline: the new real workspace test fails against the unchanged workspace implementation because the expected step spans are absent. It passes with the wrappers. - Final affected coverage: **804 distinct cases pass** across seven complete suites: 698 native-executor cases, 39 real PostgreSQL workspace cases, 61 timing/context/startup cases, and six execution-configuration integration cases. The final native and unit invocations pass with no selected skips. The workspace suite passed in the initial six-file invocation on the same final workspace source. - Preserve the initial six-file attempt as failed: **7 failed and 789 passed**. All seven errors occurred during embedded PostgreSQL startup. The log reports local macOS shared-memory exhaustion. A fresh 15-case database group passes, followed by the complete 698-case native suite. These runs overlap; do not add their counts. - The new native timing assertion fails against unchanged native code because the expected spans are absent. It passes with the wrappers and retains the credential-isolation assertions. - Full `pnpm -r typecheck` and `pnpm build`: passed on the final production source. Earlier nullable-closure errors are fixed with erased assertions inside existing guards. The failed attempts remain recorded. - Normalized AST comparison: removing only timing wrappers, timing imports/local handles and erased assertions restores the original workspace, startup and native executor programs. This checks call order and policy expressions; it is not live acceptance. - The expanded head first failed two configuration integration assertions because their test providers had no endpoint. Configure the endpoint explicitly in those two tracing tests, and restore it after each test. All six cases then pass with the original credential cleanup, redaction and exact span assertions. Production is unchanged by this repair. - The first PR head failed one unchanged attachment test at its 15-second deadline in CI. The complete 32-case attachment suite passes locally with that deadline. Do not label the failed CI attempt green. The expanded head needs fresh full CI. - `node scripts/check-maps.mjs`: passed, 35 areas and 176 capability IDs. `node scripts/storybook-catalog.mjs`: 283 files and 2,728 stories. Structural checks only; no browser, play or Storybook build claim. - `git diff --check`: passed. No added prose comments, lockfile changes or new 1,000-line crossings. - A duplicate full local test run was not started. Fresh exact-head full CI and E2E pass on `15a683bb2419960f611584f8c07cdcfc9ce3af0b`. All 56 check runs are successful or optional skipped. Superagent and legacy Snyk pass. Completed Apex review is 5/5 on this same head, with zero unresolved review threads. Commands: ```sh pnpm exec vitest run --project @paperclipai/server --maxWorkers=1 --fileParallelism=false server/src/services/native-runtime/native-session-executor.test.ts server/src/services/execution-preparation-timing.test.ts server/src/startup-recovery-timing.test.ts server/src/services/heartbeat/preparation-timing.test.ts server/src/services/heartbeat/preparation-timing-context.test.ts server/src/services/heartbeat/workspace-preparation.test.ts pnpm -r typecheck pnpm build node scripts/check-maps.mjs node scripts/storybook-catalog.mjs ``` ## Risks - Configured operators receive more spans. They still follow the configured sampler and exporter. - Startup spans measure the awaited phase, including remaining readiness waits. They do not measure earlier worker construction or a provider turn. - Workspace and native setup intervals overlap their containing phases. Do not add them to the parent duration. Native timing covers named awaits, not every statement or every error path. - These spans are not yet deployed or tested against a live latency run. No first-party Telemetry event, run-log write, schema or API changes are included. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact served variant and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d67e3507b2 |
feat(runner): qualify Codex platform packages and cache hosted builds (#16028)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native execution needs a compatible installed daemon and provider dependency. > - The Codex-default work found missing platform packages and clean-install startup failures. > - Release builds also repeat dependency compilation and run preparation serially. > - This prerequisite packages four daemon targets and verifies installed native startup. > - It also caches hosted Cargo dependencies and overlaps JavaScript preparation with compilation. > - Agent creation defaults and experimental gates stay at the current master behavior. ## Linked Issues or Issue Description Refs #15683. Refs #15555. Refs #15422. Refs #15926. Refs #15932. Extract runtime and installation support from #15683. This prerequisite has 65 changed files. Creation, onboarding, imports, UI runner selection and graduation remain in the dependent PR. Feature/access maps and the Storybook source catalog stay synchronized. Existing agents keep their execution choices. ## What Changed - Correct the existing runner test fixtures. Reserve owned credential quorum listeners through the existing bounded fixture helper. Keep foreign listeners intact, run actual credential staging once, and preserve its rejection and unchanged-auth assertions. Reuse the durable-resume outer-limit correction. Wait for actual warm-completion and held-authorization callback entry under the existing recovery deadline and surface early request errors. Preserve every semantic assertion. - Retain bounded installation failure diagnostics through the existing qualification receipt upload. Keep the original error, 10-minute deadline and all success checks. - Keep startup ownership until credential preparation and registration drain. Reject duplicate and post-close startup/resume work. Observe shutdown during normal and adopted authentication readiness. Preserve the primary failure and retain private state when cleanup is unconfirmed. - Package Linux x64/ARM64 and macOS Apple Silicon/Intel daemons. Verify target headers, source identities, manifests, and hashes before assembly and installation. - Reuse the existing release producer/consumer workflows. Cache Cargo dependencies by target, OS, compiler, lockfile, and contract inputs. Rebuild workspace binaries. Permit trusted cache writes only for canonical exact-source master pushes. - Overlap JavaScript preparation with native compilation. Join both commands and failure/cancellation cleanup before artifact verification. - Preserve shell positional arguments in the shared SSH runner and apply environment assignments correctly. Reject environment option keys. Six before/after controls and two real SSH subprocess paths pass with zero provider calls; this fix lands before both Codex and Claude defaults. - Admit verified suspended SSH backups during cold controller continuation, using the same full-scope, integrity and lifecycle checks as sandbox targets. Two positive SSH cases fail before; all 14 positive/negative matrix cases pass afterward. Active ownership and corrupt/malformed/nonempty state remain rejected. - Record measured sandbox platform facts through the existing explicit environment probe. Bind facts to the company, effective provider configuration, immutable OCI image and actual captured image binding. Check the returned cleanup receipt. Reject client-authored proof and stale or named boot identities. Creation and discovery allocate nothing. Preserve connection-only behavior. - Resolve the actual remote Codex target and preserve same-host development overrides. Preserve runtime artifact verification. Use actual protocol/model compatibility rather than a numerical CLI version fence. - Resolve the installed Codex login dependency on 64-bit Linux and Mac independently of the daemon libc baseline. Preserve account homes and safe errors. - Fix actual clean-install TOML table comparison. Preserve strict semantic checks, comments, private writes, and malformed-input rejection. Preserve primary startup failures and bounded cleanup. - Document GNU libc 2.39+ and stock Ubuntu sandbox prerequisites. Qualify ARM startup with a scoped distribution profile and unchanged global restrictions. Preserve Windows and Alpine deferral. - Preserve selected and saved native Codex reasoning effort in the existing UI builder, while excluding it from other providers. This small shared correction prevents import edits higher in the stack from dropping effort. - Reuse existing installed-package checks and assertions. Paperclip tarballs contain no Codex native payloads. Official npm dependencies supply the consumer's Codex binaries. All Docker and native builds stay remote. ## Verification Candidate: `271d434c170e74bdedde65d0b13059a55419c943`. It includes current master `e168512f7bdbb6af6ea01954c6d7ed80b91e903e` and preserves prior candidates through ordinary commits. Current [full CI](https://github.com/paperclipai/paperclip/actions/runs/38346601686) is successful on attempt 2. GitHub reports 54 successful checks and four conditional skips, including required verify and browser aggregates, workspace typecheck/tests/build, Linux DB controls, both runner shards and [isolated native compilation](https://github.com/paperclipai/paperclip/actions/runs/38346601609). Fresh current-head Greptile is 5/5 with all four threads resolved. The passing build's clean Grok/Codex report records CI merge source `7a82a37438e6e448abc7cc5caa60dc2519a97be0`, 18 installed packages, omitted Codex payloads, official host-only npm dependencies and zero provider calls. Pi setup and its no-prompt lease verification also pass. The first attempt's single 60-second Node archive download timeout remains retained; its DNS/TLS/download cause remains unknown. One unchanged-source retry was requested through the normal Build-job dialog with debug logging disabled. GitHub's attempt 2 graph reports the other CI jobs as well; no second retry or source/deadline relaxation was made. The [final exact-source package campaign](https://github.com/paperclipai/paperclip/actions/runs/38347027535) is successful on literal `271d434c170e74bdedde65d0b13059a55419c943`. It passes all four native producers, public Linux package assembly and clean installed consumers on Apple Silicon, Intel Mac and Linux ARM64. The nine qualification-mode skips are intentional; full CI above covers their ordinary gates. Native builds and Docker assembly ran remotely. The predecessor [`dae808` package campaign](https://github.com/paperclipai/paperclip/actions/runs/38342104102) remains literal historical evidence. Only tests and the dated record differ from that source; runtime, package and workflow code is unchanged. The reused credential fixture correction passes all 42 canonical Node 24 controls with synthetic data and independent matching-source review, and the current hosted runner shard passes. Production credential behavior is unchanged. **Merge hold:** current-source CI, fresh 5/5 review and exact-source package qualification pass. Actual code-owner approval and normal protected merge remain pending. No slice has merged or deployed. GitHub CLI authentication needs refresh; the normal signed-in embedded GitHub UI provided this status without credential workarounds. - The master merge preserves both Storybook source-review appendices and the guarded test-home cleanup. All six existing cleanup controls pass without skips. Master's isolated-home helper/test/setup are byte-identical. The two existing whitespace lines in master's unrelated privacy test remain outside this PR's diff. - Canonical Node 24 passes all 24 existing public-install helper tests. Three narrow added cases prove bounded/redacted failure metadata, symlink exclusions, original exception identity, failed diagnostic writes and no false failure on success. The Mac installer now uses `--timing`; at most two 16 KiB tails and 32 structured entries per tail enter the existing receipt upload. Raw argv, URLs, environment, error text and stacks are omitted. A failed receipt explicitly remains FAILED/unqualified/runtime-not-attempted. The 600,000 ms install deadline and every real package/hash/payload/lock/lifecycle/runtime oracle are unchanged. - Canonical Node 24 reproduces the durable-resume outer timeout under controlled fake latency. Only its outer test limit changes to 15 seconds; internal 500 ms/5 second deadlines and usage/effect assertions remain. Controlled warm-recovery and before-bootstrap authorization scheduling delays reproduce their callback polling failures, while the explicit entry signals pass. Early recovery errors surface unchanged. Independent review is clear. Historical mock binaries support the fixture controls; fresh full hosted CI remains decisive. The original hosted callback failure cause is inconclusive. - The composed runner/surfaces/server/E2E TypeScript checks pass. Map checks pass 35 areas/171 capability IDs. Token gates pass. The existing Storybook catalog passes 223 files/2,210 stories; story IDs and classifications remain intact. The current master had 25 inherited stale fingerprints from three changed map recipes. Only those guarded fingerprints and a source-only review note change; all 92 non-map story/component/fixture inputs remain unchanged. Existing classifications are retained. This proves source structure, not visual or live acceptance. - The 65-file contribution audit has no code file crossing from below 1,000 lines to above the limit. Earlier comment cleanup preserves executable emissions, parsed workflows and action pins. Independent cleanup/catalog and three-file reporting reviews are clear. No new workflow, suite, grader or local native/Docker build was added. **Prior `dae808` gate evidence:** fresh Greptile is 5/5 with four resolved threads, isolated native compilation and the browser aggregate pass. Its first runner shard passes the previous two corrections but fails a different held-authorization cell at the implicit one-second polling deadline. It records 1,672 passes/one failure/three skips. This candidate applies the verified entry-signal correction without changing production deadlines or any semantic assertions. Its original hosted cause remains inconclusive until fresh CI. **Previous-source evidence, kept literal:** the [exact `16a532` package campaign](https://github.com/paperclipai/paperclip/actions/runs/38338806076) passes all four native producers, public Linux package assembly, Apple Silicon, Intel Mac and Linux ARM64 installed consumers. Its fresh review is 5/5 with all four threads resolved. Full CI records 52 successful checks, four conditional skips and the two fixture failures corrected here. This is not a green full-CI candidate or proof for the new source. `64c074fb8da1b31f8d6042c4fef48dc4164e759b` passed [full CI](https://github.com/paperclipai/paperclip/actions/runs/38332726938), 54 checks/four conditional skips, [isolated native compilation](https://github.com/paperclipai/paperclip/actions/runs/38332726787/job/115085895328), and fresh Greptile 5/5/all four resolved threads. The hosted transport suite passed 246 tests and startup trust passed 22, closing that source's local fake-binary limitation. These are historical after reconciliation. The [previous exact-source package campaign](https://github.com/paperclipai/paperclip/actions/runs/38332921463) built all four platform daemons and passed public Linux package assembly plus Apple Silicon and Linux ARM64 installed consumers. **Intel Mac failed:** its initial scripts-disabled npm install timed out before lifecycle/runtime assertions. The underlying download/resolution/host cause is unknown because the old helper deleted the npm logs. This is the concrete baseline for the diagnostic change. The failed campaign remains failed; no all-platform signoff or automatic retry is claimed. The earlier `a200495` [four-target campaign](https://github.com/paperclipai/paperclip/actions/runs/38283695248) and [scoped ARM qualification](https://github.com/paperclipai/paperclip/actions/runs/38291535914) remain component evidence. Cargo restores were cold misses; no warm-cache speedup is claimed. Paperclip tarballs contain no Codex payloads. Official npm dependencies supply Codex on the user's platform. Current-source full workspace typecheck/tests/build, Linux DB controls, static/Rust/native tests, all browser shards and exact-source package consumers must pass. New-agent defaults and final managed onboarding belong to #15683. Actual private Cloud composition, warm/cold signup, task/follow-up and rollback compatibility remain later release gates. Source audits found that ordinary managed onboarding does not run the saved platform probe and Cloud distributes named snapshots without a verified final boot-image binding. These dependent release blockers remain open; prequalified E2E fixtures do not close them. Master adds task-workspace migrations; the final image/migrator and previous-image checks must include them. ARM paid task/follow-up and ordinary OS APT installation are not proven. ## Risks - Multi-target packages require source and header agreement. Missing or mismatched artifacts fail explicitly. Existing execution choices remain unchanged. - Linux requires GNU libc 2.39+ and compatible sandbox prerequisites. The scoped hosted ARM preparation does not qualify arbitrary containers or kernels. Windows, 32-bit ARM, and Alpine remain deferred. - Only dependency crates are cached. Preview jobs cannot write trusted caches. Concurrent command cleanup must preserve the primary failure. - Native-free Paperclip packaging retains official Codex npm dependencies. No Codex payload bundling, schema migration, automatic agent migration, or production deployment is included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and parallel agents. The exact serving model snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f329b24ee1 |
feat: give tasks durable project-optional workspaces (#15749)
Give tasks durable company-scoped workspace bindings independent of project assignment, channel defaults and authorized repository preparation. Reuse verified input seeds and sparse output recovery while preserving trust, privacy and controller provisioning boundaries.
Full CI and E2E, Superagent, legacy checks and Greptile Apex 5/5 passed on reviewed head
|
||
|
DottaandPaperclip
|
1e0cfcb4fc |
test: retire redundant suites and consolidate owner coverage (#15978)
Retire duplicate ACPX and Codex proof after migrating unique assertions to their behavioral owners. Consolidate repeated fixtures and exact case matrices while preserving lifecycle, protocol, company, credential, approval, and ordering contracts. Remove the unused runtime-controls compatibility path and document test ownership and preservation evidence. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
db2f00947b |
fix(runner): include Claude ACP in the server package (#15912)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server ships a copy of Paperclip Runner for native agent execution. > - The Claude ACP bridge is a runner dependency, but it was absent from the server package. > - The runner resolves this bridge dynamically, so the import-based packaging check missed it. > - A Claude session then failed before startup and showed a generic connection error. > - This pull request ships the qualified bridge and reports a missing runtime with repair steps. ## Linked Issues or Issue Description **What happened?** A server install with the vendored runner could not load `@agentclientprotocol/claude-agent-acp/package.json`. The runtime test suggested checking the model or provider connection. The task reported an interrupted runner session. **Expected behavior** The server package includes its qualified Claude ACP runtime. A missing bundled runtime reports an installation error and explains how to repair it. **Steps to reproduce** 1. Use a server install with the vendored runner and no Claude ACP bridge. 2. Select the Paperclip Runner ACP harness and the Claude provider. 3. Run the runtime test or open a session. 4. Observe a missing-module failure before Claude starts. **Paperclip version or commit** Reproduced on `e733bddc45`. This branch is based on current master. **Deployment mode** Self-hosted server built from source. Related: #15224 upgrades the Claude bridge. This PR fixes its omission from the server package and keeps the current qualified version. A future bridge upgrade must update the server pin too. ## What Changed - Add Claude ACP `0.73.0` to the server dependencies and bundled dependencies. - Pin the qualified Claude SDK and bridge dependencies in the npm bundle producer graph. Publish its native packages as consumer optional dependencies instead of shipping the producer executable. - Bind source-built vendored runners to the shared pnpm dependency store while retaining the server manifest as the provider authority. Keep published npm and self-rooted deployment roots unchanged. - Include dynamically resolved Claude and Codex bridges in the vendor dependency check. - Preserve the missing-package category through the sidecar, Rust transport, and native session executor. Stop automatic retries for this installation failure. - Show **Runtime missing** in the runtime test and task thread. Explain the reinstall or rebuild, server restart, and runtime test steps. - Add regression tests and document the repair procedure. ## Verification - Red/green comparison with the production sidecar: the same missing-package setup returns a raw require-stack error on the original code and an actionable, non-retryable missing-runtime error on this branch. - The rebuilt server's vendored Claude installation probe passes using the source-server authority selected by the launcher, without starting a provider or making a model request. - Focused server verification: 830 tests passed. Runner installation and sidecar error tests: 83 passed, with 6 platform-specific skips. Packaging tests: 28 passed. Rust transport tests: 7 passed. - Source/deployment authority tests: 3 passed. Red/green packaging tests include a Linux-produced Claude SDK bundle, actual npm pack, and closed executable admission on Linux x64 and macOS ARM64/x64. - Full UI suite: 8,113 tests passed. - `pnpm -r typecheck` and `pnpm build` passed on the rebased branch. - Full current-head CI passed, including all general and serialized test lanes, Runner Rust/Vitest verification, typecheck, build, and all eight e2e shards. Greptile reviewed `d6a2741834ff922956560f2f689382827bbaf9e0` at 5/5 with no unresolved threads. - Local full-suite limitation: an earlier parallel attempt hit Postgres startup failures and timeouts; the affected server tests and a database fixture passed in isolation. The subsequent serial `pnpm test:run` was stopped while still running after complete CI passed. Focused local tests, full UI tests, typecheck, and build completed as listed above. - Token gates and `git diff --check` passed. ## Risks - Bundling the Claude bridge increases the server package size. The producer graph retains the qualified SDK version and existing isolation patch. Native platform declarations move to the published server manifest, while the runtime keeps its exact version, declared-slot, no-follow, and executable digest checks. - The new failure category is permanent. Operators must repair the server installation before they retry the task. - Existing failed runs retain their recorded error. The clearer message applies to new runtime tests and runs. - No database schema or migration changes. ## Model Used OpenAI GPT-6 through Codex. The system reports the model family; the exact deployment ID and context-window size are not exposed in this session. Used reasoning, browser inspection, tool use, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
1b4c803816 |
fix(costs): preserve usage attribution and explain unavailable run costs (#15948)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The cost ledger records usage and controls agent budgets. > - Some runs lose the selected model or API billing category before their usage reaches the ledger. > - The dashboard reports missing costs but does not identify the affected runs or the available actions. > - This pull request preserves usage attribution and adds a compact view of cost gaps grouped by agent. > - Operators can inspect runs, open agent settings, and record saved costs without starting new agent work. ## Linked Issues or Issue Description Refs #15634, #11938, #15879. This follows the merged subscription reporting in #15634. The older proposal in #11938 addresses part of ACP model attribution. This change also rejects conflicting models across a turn, fixes credential-backed billing capture, and explains cost gaps in the existing dashboard. The Pi receipt recovery work in #15879 covers a separate adapter recovery path. This PR preserves unresolved receipts and does not recover provider usage or reprice past entries. **What happened?** ACP results use the requested model even when the runtime selects a model. Native and OpenCode API runs can report unknown billing despite a matching API credential. The dashboard shows notices without links to the affected runs. These gaps make API and subscription totals harder to interpret. **Expected behavior** Record the model selected by the runtime. Identify API billing only from a matching credential. Keep costs unresolved when reliable usage or pricing is absent. Show the affected agents and runs. Offer an accounting retry only when saved evidence is ready. **Steps to reproduce** 1. Run an ACP agent with its model left unset, or use a matching API credential with a native or OpenCode agent. 2. Inspect the recorded model and billing category for the run. 3. Open Costs Overview for a period with unpriced usage or terminal runs awaiting receipts. 4. Inspect the notice and try to find the affected run or the relevant agent settings. **Paperclip version or commit** Current master at `657e71584a0`. The patch is rebased on that commit. **Deployment mode** Self-hosted server and local development. The cost detail tests use a disposable PostgreSQL database. ## What Changed - Preserve the runtime-selected ACP model in checkpoints and final results. Conflicting models across one turn remain unpriced. - Share credential-backed API billing checks across native and OpenCode results. Keep unknown billing when the provider and credential do not match. - Add company-scoped cost detail and accounting retry endpoints. Board viewers can read details but cannot retry. Agents and foreign companies are denied. - Limit each detail category to 25 entries. Use one consistent database snapshot and sparse indexes matching the filters and sort order. Add the unknown-billing count to the existing summary query. - Replace the notice with an expandable count line. Group entries alphabetically by agent. Show the saved blob icon, link each name to configuration, and put run and retry links beside the date. - Preserve estimate and unavailable-price labels on Providers and Billers; keep retry confirmation after the resolved notice disappears. - Fetch details only while expanded. Retain the last loaded details after a refresh error. Retry saved accounting only; do not rerun the agent. - Add capture, database, authorization, UI, and Storybook coverage. Update the implementation contract and cost reporting guide. ## Verification - `pnpm -r typecheck` passed after rebase. - `pnpm build` and `pnpm build-storybook` passed after rebase. - `pnpm check:token-gates` and `git diff --check` passed. - 1,086 focused tests passed across 17 files. These cover ACP, native and OpenCode capture, PostgreSQL reports, route authorization, OpenAPI, Costs UI, and existing accounting regressions. - Desktop and mobile browser checks passed during UI review. They cover lazy expansion, alphabetical groups, loaded blob icons, configuration and run links, keyboard retry, and layout overflow. - `pnpm test:run` was attempted with command-scoped `init.defaultBranch=master` for Git fixtures. It was stopped after a native session recovery failure reproduced on unchanged base `657e71584a0` (`archives a damaged prior epoch before completing guarded same-task replacement with real runnerd`). An MCP upload case failed in the broad run but passed in isolation on both the base and PR branch. These local limitations do not affect the 1,086 passing focused tests. - Current-head CI is green, including all unit test lanes, serialized server suites, runner checks, typecheck, build, policy, and eight browser shards. Apex re-review is 5/5 on `56de60effdb`; all four original findings are addressed and every review thread is resolved. - PostgreSQL applies migration `0333` in the integration fixtures. EXPLAIN ANALYZE on the production detail statements uses the new sparse indexes behind 5,000 healthy records in each table. - Ten provider/biller card cases cover missing and estimated prices; the retry test covers row removal and notice unmount. A live Storybook browser check verifies the confirmation and updated count. - Run `pnpm storybook`. Open **Costs / Subscriptions / Cost Data Quality Details** to inspect the production page with representative fixtures. ## Risks - Model and billing metadata change for future runs. Estimates still require complete usage and a supported price. No past cost amount is rewritten. - A missing final receipt remains unresolved for budget checks. The display changes after fifteen minutes; that time limit never authorizes settlement. - Accounting retry is a write action. It retains the existing company checks, receipt fences, idempotency, and audit logging. - Detail reads are bounded and run only after expansion. They poll every 30 seconds while open. No provider probe or new summary query is added. - Migration `0333` adds two partial indexes; it changes no stored cost amounts or table columns. Index creation scans the existing tables and may briefly block writes during deployment. No lockfile or workflow change is included. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code editing, browser checks, and test execution. The exact serving model ID, reasoning setting, and context window were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
be1bc5d494 |
fix(runner): select bundled Linux assets for remote OpenCode (#15721)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native Runner can execute OpenCode in a remote sandbox. > - Installed releases contain a Linux daemon and the identity of their provider pack. > - OpenCode did not select those assets by default. Cursor did. > - The upload fallback also compared the verified Linux daemon with the controller platform. > - This pull request uses the packaged remote assets for OpenCode and corrects that platform check. > - Users of a matching Paperclip package and Daytona image can start OpenCode without artifact path overrides. ## Linked Issues or Issue Description Refs #15702. Refs #15422. **What happened?** Remote native OpenCode asked for `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` even when the installed release contained the expected pack identity. The daemon upload fallback could then reject a verified Linux daemon when the controller ran on macOS. **Expected behavior** OpenCode should select the release's Linux daemon and provider-pack identity. The controller must still verify the image assets before launch. **Steps to reproduce** 1. Run a complete installed Paperclip release on macOS. 2. Assign native OpenCode to a Daytona environment with the matching Paperclip image. 3. Start a task without remote artifact path overrides. ## What Changed - Select the bundled Linux daemon and provider-pack identity for remote OpenCode. - Skip the controller-platform comparison when the Linux daemon came from the verified bundled selector. - Add regression coverage for OpenCode and Cursor. Make the missing-identity fixture independent of local build outputs. - Document the matching package and image requirement, plus source-development setup. ## Verification - Red: both new regression cases failed before the fix. - Green: the affected runtime suites passed all 686 tests on the rebased head. - `pnpm -r typecheck` and `pnpm build` passed before the rebase. - Full local tests were attempted, then stopped after failures outside this change. That run overlapped a rebase. The clean rerun of the AI connection fixture passed all 111 tests. The affected runtime suites also passed after rebase. All latest-head CI and security checks passed: 54 successful checks and two expected skips. - A real OpenCode/OpenRouter DeepSeek task completed in an isolated Daytona workspace on macOS. Durable bash receipts confirm `df -h` and `uname -s; uname -m`. The result was posted, the task completed, and sandbox cleanup succeeded. - The live test used verified remote development assets from the official image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`. No remote artifact path overrides were set. This was not a new complete release assembly. - The low-trust test policy kept output quarantine. The operator permitted only this disposable test agent's OpenRouter credential binding. ## Risks - A second real external-origin retry executed successfully and saved its answer, but its completion report used unsupported evidence references. Its task stayed in progress. This PR fixes startup; completion-evidence guidance requires a separate fix. External voice playback was not verified. - A plain Daytona sandbox image lacks the provider pack. It still needs a verified full-pack override. - Source-development launchers must provide the remote build assets. This PR does not publish a new image or create a complete release assembly. - Explicit artifact overrides retain precedence. Manifest, executable, package, bridge, and provider-asset checks remain in place. ## Model Used OpenAI GPT-6 through Codex. The exact deployment model ID and context window are not exposed in this session. Capabilities used: reasoning, code editing, shell commands, and browser testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merging Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
556daf9a99 |
feat(dot): run cloud assignments in managed sandboxes (#15691)
## Thinking Path > - Paperclip manages agents and governs their work. > - The Runner owns execution and task authority. > - Dot executes at OpenAI and uses Paperclip through MCP. > - Dot previously required a self-hosted controller. > - Cloud must admit Dot work in a managed sandbox and keep protocol traffic in the tenant. > - This pull request adds managed Dot execution and dedicated pairing routes. > - The same Runner path can later host sandboxed workspace tools. ## Linked Issues or Issue Description **Agent or provider** OpenAI Dot, through the existing dedicated MCP adapter. Related changes: #15402, #15414, and #15581. **Why this adapter is useful** Cloud users need to assign work to Dot and let Dot request work while idle. Neither path should execute agent commands on the control-plane host or require an idle sandbox. **How the agent is invoked** The tenant serves OAuth, signed mailbox events, and scoped tools. Normal admission creates a managed sandbox for each assignment. A verified Rust Runner owns that assignment and relays its tools and results. ## What Changed - Enable the external-agent invitation in Cloud behind the standalone Dot flag. - Launch Dot through the existing managed process launcher and authenticated PRP transport. - Verify the artifact digest and Dot capability before dispatch. Read provider checkpoints from the execution target for recovery. - Reject local execution in Cloud and reject managed workspace access for this release. - Add dedicated public Dot pairing aliases. Preserve the personal assistant consent paths. - Keep Dot events in the tenant when a personal Cloud broker is configured. - Use the app-configured heartbeat for Dot-initiated work. Report terminal admission failures and fence stranded offers before new admission. - Offer normal comments through the active assignment's mailbox. Keep their wakes deferred until a successful history receipt contains the exact IDs; preserve unread comments through finish, batching and recovery without replaying consumed input. - Require a sandbox process fingerprint before dispatch and allow a bounded five-minute controller reconnect gap. Preserve remote process birth without inspecting a colliding controller PID. - Recognize authenticated remote shutdown receipts even without an OS exit code. - Document dedicated pairing aliases and teach route coverage to inspect literal path arrays. - Add opt-in private Daytona qualification and focused contract tests. ## Verification - Workspace typecheck, production build, and UI token gates passed. - Focused onboarding, OAuth, managed environment, artifact metadata, and terminal-fence tests passed. One environment-test timeout passed on an isolated repeat. - A private Daytona fixture passed with the real Rust Runner, signed readiness, assigned app tools, document writes, duplicate receipts, authority fences, completion, and sandbox deletion. - A real OpenAI Dot completed both an operator-assigned task and a Dot-initiated task in managed Daytona sandboxes. Both leases were released. - A real accepted Dot assignment survived a controller gap longer than one minute with the same sandbox process, Runner, session, and assignment. Dot consumed the normal task-comment follow-up and completed exactly once; the lease was released. - A real cancellation fenced an accepted assignment, released its lease, and received Dot's acknowledgement without claiming a global provider stop. - Final local checks: 74 Dot execution, OAuth onboarding and API contract tests after integrating the current agent-lifecycle changes, 9 real Rust driver tests, 21 remote-runtime tests plus the 3 detached-process cleanup tests, and 17 lifecycle tests passed. The final live Daytona fixture passed and deleted its sandbox. - A final real Dot follow-up stayed pending until its exact comment appeared in a successful history receipt. Dot completed the original managed assignment once; its lease was released. - The broad local suite crossed a rebase and was stopped; its failures are not a qualification of this final source. Focused repeats passed. The final head passed all CI gates: workspace types, build, general and serialized test suites, Rust and Runner verification, all browser shards, and the canary dry run. Greptile is 5/5 with both findings resolved. ## Risks - Deploy the tenant app with its current Linux Runner artifact and the corresponding front-door protocol routes together. - This branch has not been deployed to a Cloud tenant. A deployed tenant still needs an OAuth and assignment smoke test. - Dot provider usage and cost are unmetered. Revocation fences Paperclip tools and does not prove a global OpenAI stop. - Missing or changed recovery identity fails closed. No replacement Dot thread is invented. - No schema migration is added. Managed Dot workspace access remains unavailable. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and browser testing. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
57e977be72 |
feat: integrate Pi 1.0 into the experimental Runner (#14921)
## Thinking Path > - Paperclip manages AI agents and their work. > - The experimental Runner owns provider processes and durable sessions. > - Pi needs working task execution and human controls. > - The five-PR stack must preserve changes already on master. > - Each layer now carries the complete integrated source for a safe sequential fallback. > - This PR belongs to native GitHub stack #15602, ending at #14956. ## Linked Issues or Issue Description Refs #14436, #14631, #14743 and #14956. Ship Pi 1.0 through the experimental Paperclip Runner. The five PRs are #14921, #14922, #14923, #14924 and #14956. The user authorized the complete merge after checks pass. Existing `pi_local` execution is unchanged. Accounting and wider provider/platform qualification remain deferred. ## What Changed - Recover missing final replies after workspace finalization changes owners, using accepted-turn evidence without rerunning work or granting external-chat publication. - Preserve the admitted Pi instruction root across warm runs, while retaining changed-root rejection. - Give Pi a bounded 15-second default shutdown grace so stop, drain acknowledgement and durable suspension can complete. Explicit deadlines and other providers retain their existing behavior. - Integrate the Pi 1.0 runtime and master contracts. - Use Pi profile 22. Preserve explicit caller-selected models and exact native thinking levels. Keep Pi's wrapper, helper, extension and question/control behavior unchanged from the qualified profile-19 runtime. - Preserve master's Dot lifecycle and consent fields, configured task environment, status guards and current Codex/Claude dependency versions. Cursor stays qualified. Copilot stays pending; profile 17 binds the changed shared protocol validation sources. - Exclude general AWS IAM credentials from Pi static/custom provider bindings and selected task projections; preserve the provider-scoped Bedrock bearer key. Profile 21 is retained as historical provenance. Rust and cloud install probes use the current declaration. - Patch bundled brace-expansion 5.0.9 to the exact official 5.0.12 payload. Pin the patch and complete runtime closures. Include the patch in normal installed setup tooling. Keep the upstream Pi shrinkwrap as provenance and permit only this exact security correction. - Include current attestation files in the Docker build context. Keep the repository lockfile unchanged from master. CI and private image builds resolve manifest changes before their frozen installation. ## Verification - Full local `pnpm -r typecheck` passes, including Runner Rust, server and UI. Focused integration checks pass: 194 Runner admission/environment tests, 63 profile/credential tests with one expected skip, 152 Dot/UI configuration tests, and Pi transcript/notice tests. - Full local `pnpm build` passes on the final source. - Fresh final-source checks pass: all 698 Rust workspace tests (32 binaries), 156 credential/profile/controller tests with one expected skip, Runner TypeScript typecheck, and 20 package/setup/sandbox tests. - The profile-21 Pi materializer passes on the native host with the official pinned Node 24.21.0 and its npm. It verifies all 150 locked packages, the patched dependency and the exact closure. Setup/package bundle tests and UI token gates pass. - The old hashes were reproduced for all three supported targets before calculating the patched graph. New closure hashes are darwin-arm64 `282022db10150c6632b3444df421342e7d534bdf5d5fb1097a2e79d0625a2bcf`, darwin-x64 `64e251e19009f755c0b04f73ce2138246faab71a961b0f13d75ebfcc34bef12e`, and linux-x64 `713b1fdff42fb56a1518bdc084f181d70bee8ebadc3e4b1d76321ed9108c8410`. Independent native platform execution is separate from graph identity reproduction. - Historical cloud qualification remains unchanged: all seven core cases pass on shipping source `10dc43c9ec65d88c2f782d62afb296d09494f215`, harness `1a4408a48cfb5a1f094a311141c257c92cd7a893`, image `sha256:5b3a775b383591bda1b0c1889e509acc70ce7f37c53f09733c81d59037f02280`, and accepted Sonnet 4.6/low fixture. All 215 canonical files and all seven cleanup checks pass independent verification. These are profile-19 results and are not relabeled as fresh profile-22 runs. - Current Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [The readiness plan](https://github.com/paperclipai/paperclip/blob/codex/pi-production-readiness/doc/plans/2026-10-02-pi-production-readiness.md) preserves campaign and failed-attempt provenance. - Merge only after every PR's current-head CI and fresh review pass. Linux CI covers the full suites, build and browser tests. The local embedded Postgres API-authority suite cannot start on this macOS/Node 26 host, so Linux CI must confirm that suite. ### Fresh profile-22 core qualification — 2026-10-08 All seven accepted core cases pass canonically on Pi profile 22, with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. This model is a fixture; production accepts the caller's explicit Pi provider/model. Runtime/install source: `3241a992f2a7703e59e97ed0fd3e5d6405de4401`. Frozen accepted harness: `1a4408a48cfb5a1f094a311141c257c92cd7a893`. Immutable cloud image: `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`. Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [Hosted Linux image and clean-install verification](https://github.com/paperclipai/paperclip/actions/runs/37868328023) passes, including all 20 source-bound archives, normal CLI/Pi setup, companion import and the production pack reader. This exact installation source includes the latest master integration and the corrected Pi warm instruction-root fence. Full local typecheck/build and current-head hosted CI verify the final stack. All 13 focused real-root regressions pass. The full local executor suite passed 662 tests; 15 database tests could not start the Mac embedded PostgreSQL service. Hosted Linux CI passes the full required verification and E2E checks. These fresh results keep their own source identity; profile-19 results remain historical. | Core path | Canonical campaign | Retained archive SHA-256 | | --- | --- | --- | | File edit, validation, download and Done | `pi-core22-replyfix-0-1791511228` | 23 files; `a473e8603a3dd4737863291f8d3d1e392391f0b16d433c3e0e0e9d8baf7a97b0` | | Pending question and controller restart | `pi-core22-replyfix-1-1791511376` | 33 files; `6b829c4eb74e1f32a89c692a4ae7130dbfc1c6d3cf13915effe2103d9e242c8e` | | Three-turn session/process/workspace continuity | `pi-core22-replyfix-2-1791511587` | 23 files; `7a87021f8f9a3fdd3c58bb4467f8d82c635e3ea4795d6e75f144d9aa14818df8` | | Four typed questions and browser reconnects | `pi-core22-replyfix-3-1791511881` | 42 files; `9e31755252be1f4f9cb0626c984c142d4d1ae5f5bee3a7af08444db8d12c280a` | | Plan approval and completion | `pi-core22-replyfix-4-1791512031` | 22 files; `a0383ce1aab38e7b5a25ce0e9dd3bebea5c037ebd96ae6b29dae19015da2ae2c` | | Same-turn steering and permission denial | `pi-core22-replyfix-5-1791512261` | 39 files; `c929b8c7070f0b66aedc17e65ca46e6beab1e363926ac9f7e2a75fb250f05949` | | Stop during pending permission | `pi-core22-replyfix-6-1791512390` | 33 files; `7f0a58ae0f4d5bfc76149435f4e322537089c5bd16e7ffe9b5ad71f10a621a07` | All 215 canonical files (28714587 bytes) are independently hash-verified. All seven cleanup grades pass, with no owned runtime process or temporary root after each case. Automatic retries are zero. The owned cloud host stopped normally after retention. The prior profile-22 warm attempt remains failed and separately retained: archive SHA-256 `1e54eba5ec72b50cee1534b23d1d1d4f21a090006b8a64501ba70db972abfde5`. Its original canonical classification is preserved. Diagnosis reproduced a product bug comparing an agent-files root against an unset checkpoint-only field. The fix stores the admitted physical root separately from the adopted per-run collection capability. The real-root regression fails before the fix and passes afterward, including rejection of a changed physical root. Fixture, grader, model and all seven accepted case IDs are unchanged; this fresh campaign tests final-reply publication after file registration first. The intermediate restart attempt also remains failed and retained: archive SHA-256 `5dcaefdf1d17cf4cd54fd4cf810f45e736667392339b8ce7caf08bb4e225277f`. Its original canonical classification is preserved. Pi resumed, wrote the verified answer and completed its task; exact runner suspension was proven, but idle stop consumed about 5.2s and left under 3s for the drain acknowledgement. The Pi-only default shutdown grace is now 15s, preserving a full 5s drain round trip and a finite suspension reserve. Explicit caller deadlines, other provider defaults, literal drain receipts and exact suspension identity checks remain unchanged. The timing regression fails before this correction and passes afterward; all 18 focused settlement tests and Runner typecheck pass. The final-source file attempt is also preserved as failed (`candidate_failure`), archive SHA-256 `db6767b6773ea618997927ac77bdb005a5ac81492c7b9c0ffbc900449f829bc9`. Native edit, validation, exact downloadable artifact and Done/succeeded all passed, and the exact final reply was durably recorded. A workspace recovery owner completed before the live heartbeat reached presentation, leaving that reply absent from task chat. Recovery now materializes only a completed final reply from the accepted turn of an ordinary internal Done task, preserving issue/run/contract binding, suppression, external-chat authorization and same-run deduplication. The database regression covers the generated file-preparation receipt, suppression, unapproved external continuation and replay. Server typecheck and all 49 response-selection tests pass; hosted Linux verifies the database regression because embedded PostgreSQL cannot start on this Mac. The delayed-final-answer database regression passes on [the final root-source Linux server shard](https://github.com/paperclipai/paperclip/actions/runs/37868262553/job/113628594152), alongside 1,108 passing tests. The first root Runner shard had one unchanged durable-resume test exceed its 5-second timeout; the identical top-source shard and the isolated exact test passed. One rerun of that failed job and its required aggregate passed without source or test changes. The original failed job log and the single-rerun receipt remain retained. ### October 9 merge verification Current merge head: `5a8fe63512a7166aaef5cf50065a25008aa8b44b`. All current-head checks pass, including `ci / verify` and `ci / e2e`; exact-head Greptile review is 5/5 with no unresolved threads. Current master conflicts are resolved. The user authorized the maintainer override of the code-owner review gate after these checks. The seven retained live core cases remain bound to source `3241a992f2a7703e59e97ed0fd3e5d6405de4401` and its recorded cloud image. ## Risks - The security correction changes the dependency closure and profile identity. Old sessions must reopen on the new profile. Exact identities and credential bindings fail closed. - The runner remains experimental and requires explicit selection. Legacy Pi Local is unchanged. Caller model IDs pass through; the E2E model is a fixture. - Accounting and the broad platform/provider matrix remain deferred. This merge does not publish a release or deploy a service. ## Model Used OpenAI GPT-6 through Codex assisted with reasoning, repository inspection, editing and tool use. The exact serving ID and context window are not exposed in this session. Final live qualification uses Pi 1.0.0 with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bf9dbd18a8 |
fix(runner): recover incompatible Codex models before launch (#15519)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Runner prepares and saves each task's provider configuration before launch. > - A sandbox image can have a supported Codex CLI that is too old for the selected model. > - The current version check stops that task even when the image can run a similar model. > - This pull request selects a compatible model before it saves a fresh execution. > - The task continues with a visible warning, and recovery uses the saved effective model. ## Linked Issues or Issue Description - Refs #15053. This fixes the same CLI and model mismatch for fresh remote Runner tasks. It does not change subscription onboarding. - Refs #14721. This adds a bounded startup choice for Codex CLI compatibility. It does not add user-defined fallback chains or quota failover. - Related PR: #15399 added the model-specific CLI checks that this change preserves at launch. ## What Changed - Check the preinstalled remote Codex CLI before saving a fresh execution that uses a model with a verified CLI minimum. - Select the closest compatible older model in the same class, then the stable Runner default. Consider each candidate once. - Save the effective model before checkpoint selection. Keep the requested agent and task settings unchanged. - Add a task warning and a local `runner.model_fallback` run-log event with both models and the CLI version. - Share executable discovery with the launch verifier. Preserve explicit artifact and install settings, saved executions, and existing artifact checks. - Add regression tests and document the selection and warning behavior. ## Verification - Red: the three new provider-configuration regression cases failed before the fix. They kept the incompatible requested model. - Red: both rejected-probe cases failed before the review fix. They now defer to launch verification. - Green: targeted provider configuration, remote preflight, task warning, and launch-verifier tests pass (680 tests). - `pnpm -r typecheck` passes. - [Full CI](https://github.com/paperclipai/paperclip/actions/runs/37713759744) passes on `eb273fc32661526d90a864b42c73a7da61dd62da`: 47 successful jobs, including all general and serialized Vitest groups, browser shards, Runner verification, typecheck, build, and the canary dry run. - Local extended verification: three serialized shards pass (115 suites). HTTP route tests hit intermittent 15-second timeouts. The authorization suite passes on rerun (130 tests). The document suite passes on both master and the PR head in the same isolated setup (6 tests). The local general run was stopped after the equivalent full CI matrix passed. - `pnpm build` passes. - Server typecheck and build pass again after the probe-error review fix. - `git diff --check` and `pnpm check:module-boundaries` pass. - Example: request `gpt-6.1-sol` on Codex 0.158.0 to use `gpt-6-sol`; on 0.156.0, use `gpt-5.6-sol`. The warning names the requested model, effective model, and CLI version. - This PR has not been deployed to staging. ## Risks - A fallback can have different capabilities. The task warning makes the substitution visible. Later runs can use the requested model after the image CLI is updated. - Preparation adds two remote commands for models with a verified CLI minimum. Failed or invalid version probes retain the existing launch checks. - This only handles known CLI and model mismatches before a fresh provider launch. Authentication, capacity, and artifact failures keep their existing behavior. - No database migration is required. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. This session does not expose the exact deployment model ID, context window size, or reasoning setting. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dd777f4b73 |
feat(dot): complete onboarding and expand governed Runner capabilities (#15414)
## Thinking Path
> - Paperclip manages AI agents, their work, and their permissions.
> - Paperclip Runner supplies the same admitted tool authority to each
provider.
> - The Dot provider in #15402 needs reliable onboarding and useful
agent capabilities.
> - An idle Dot could not start work, assign a human, read skills, or
produce a workspace artifact.
> - Pairing also relied on a second configuration save before ordinary
admission could work.
> - This pull request adds governed idle admission and shared Runner
tools, and completes pairing atomically.
> - Operators can test Dot with fresh data while keeping normal company,
approval, budget, and run ownership checks.
## Linked Issues or Issue Description
**Subsystem affected**
Paperclip Runner, dedicated Dot MCP access, OAuth onboarding,
experimental settings, and empty worktree startup. This PR builds on
merged provider PR #15402. It reuses the merged MCP gateway from #14846
and assistant connection work from #14933 and #15380.
**Problem or motivation**
An idle Dot could see assignments but could not act on a conversation
request until someone created a task first. Its Runner catalog could not
assign tasks to humans or read pinned skills and workspace files.
First-time OAuth discovery and pairing also needed browser fixes, and a
completed pairing did not persist its binding reference on the agent.
**Proposed solution**
Keep Dot within the existing Runner. Admit a visible agent-authored
intake task for idle requests. Add people, human assignment, cross-task,
skill, and optional sandbox workspace tools through shared authority.
Relay assigned app calls through the configured MCP gateway. Add lease
renewal and follow-up references. Save pairing and its configuration
revision atomically. Show prerequisites and provide a complete copy
prompt.
**Roadmap alignment**
This extends the experimental provider in #15402. It uses the existing
governed gateway, task model, skills, artifact path, and native Runner.
It adds no separate execution subsystem.
## What Changed
- Add a standalone OpenAI Dot agent choice with an independent
experimental opt-in. It works with the general Runner option off.
Require Assistant connections (MCP), authenticated sign-in, and public
HTTPS for pairing.
- Use Dot’s own option for company package import. Allow an unpaired Dot
configuration to save after external billing acknowledgement; task
admission still requires pairing. Prepare the shared dev binary for
Dot-only opt-in.
- Label saved Dot agents as OpenAI Dot. Use prerequisite-check copy in
setup and runtime configuration.
- Route Dot creation directly to pairing after explicit external billing
acknowledgement. Hide local CLI, model, and harness setup for Dot.
Preserve the shared Runner implementation and canonical API type. Other
Runner providers still require their general opt-in.
- Add an empty worktree option with fresh signing keys and no production
data copy.
- Fix public-client OAuth negotiation, discovery compatibility, and
optional separate browser authorization origin.
- Add one-use pairing consent preview, clear copied setup instructions,
and atomic binding persistence and cleanup.
- Add idle request admission with stable request IDs and normal
scheduling, permissions, budgets, and task ownership.
- Add identity and people discovery, human task assignment and
reassignment, and authorized cross-task comments and documents.
- Read assigned skill files from pinned manifests. Relay assigned app
calls through the merged gateway without exposing credentials.
- Add an off-by-default workspace bridge. Constrain paths and writes.
Run commands in a deny-by-default OS sandbox with no network or injected
credentials. Reserve mutations before effects and never blindly repeat
uncertain work.
- Add rolling lease renewal, task pagination, bounded operation limits,
and deduplicated follow-up references without comment bodies in
webhooks.
- Add an off-by-default attachment reading setting. Restrict reads to
files on the current assigned task. Verify size and hash, cache bounded
verified copies per run, paginate text or binary bytes, and recheck live
authority before returning.
- Keep file grants operator-owned. Reject agent self-grants across
configuration routes. Preserve attachment consent in create/import
forms. Close generic API file bypasses while retaining current-run
response snapshots and permitted uploads.
- Keep provider limits explicit. Do not inherit a Dot binding,
attachment permission, or workspace permission when hiring another
agent.
- Include the required Markdown format in cross-task document writes and
validate the API title limit. Verify real creation and revision
persistence.
- Restrict command execution to Linux bubblewrap with descendant
containment. macOS retains workspace file tools and artifact publishing,
while refusing command calls. Explain the platform limit in setup.
- Exclude Paperclip instance state from workspace files, uploads,
artifact publication, and sandbox commands. Protect nested directories
and case variants. Fail closed when the directory protection scan
exceeds 4,096 directories.
- Add static UI compression for slow public tunnels. Document setup, the
complete tool inventory, and qualification limits.
## Verification
- Merge preparation on `00ca2c75b` integrates merged base #15402 and
master `fc6304dfe`. The ancestry commit preserves the reviewed follow-up
source tree. The subsequent security fix excludes instance state from
file tools, uploads, artifact publication, and sandbox commands. It
preserves private task authorization and task monitors. It regenerates
the combined tool catalog, seeded catalog digest, and protocol manifest.
Workspace typecheck, full build, and UI token gates pass. All sixteen
real Dot broker cases and 93 company import cases pass after the review
fixes and creator-attribution test correction. The exported
human-assignment catalog and Unicode page boundaries are also fixed. All
ten catalog tests and nine workspace/skill bridge tests pass. All 35
workspace bridge and authority tests passed after the instance-state
fix, including real macOS commands in that intermediate version. The
subsequent document and descendant-containment fixes pass 44 focused
tests across bridge, authority, and setup UI, with six Linux command
cases skipped on macOS. Real cross-task documents pass the route
validator and persist two revisions. macOS refuses command execution and
does not advertise the tool. Full workspace typecheck and build, changed
server/UI typechecks, and token gates pass again on the final commit.
Greptile rates final head
|
||
|
DottaandPaperclip
|
fd8c6b920a |
fix(native): resume connection tasks after approval decisions (#15471)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native tasks can pause while a human decides whether to allow a connection action. > - The next turn needs both the saved decision and complete accounting for the previous turn. > - Cancellation could discard final usage, and complete direct Claude API receipts could remain unpriced. > - A stale blocked or review report could also request approval again after the original card was declined. > - This pull request retains shutdown accounting and rejects approval waits bound to an already resolved action. > - The benefit is reliable continuation with the existing budget and approval controls. ## Linked Issues or Issue Description Refs #15420. Related: #15312 addresses requester ownership during dispatch. This change addresses receipt capture and final-response validation. ## What Changed - Retain usage events after native cancellation. Continue to reject late provider messages and work. - Drain same-turn accounting and terminal events for at most fifteen seconds after a durable governed wait. Keep incomplete accounting blocked. - Estimate complete, unpriced, direct Anthropic API receipts for the exact `claude-sonnet-5` model. Record the rate version and assumptions. Use the one-hour cache-write rate when the receipt lacks cache TTL. - Bind stale approval reports to exact interaction, action-request, or invocation IDs in the same company, task, agent, and run. Cover blocked, review, and response-wake reports. Keep independent reviews valid. - Fence checkpoint and result writes after a controller detaches for restart, including operations waiting for a database lock. Reject stale successful returns before certifying accounting. - Allow bounded subscription teardown only after retaining an actual provider terminal. - Journal the exact governed-wait trigger and disposition before provider interruption. Recover that wait independently of a later saved answer, replay retained accounting, and reject mismatched or unproven terminal evidence. - Give settling governed turns a bounded window before shutdown detaches their controller. - Keep fuzzy external app matches alongside installed capability matches instead of forcing an unrelated provider question for a generic query. - Return up to twenty exact active catalog tool names after an invalid request, after eligibility checks; still reject the request without granting access or creating an approval. - Require retained provider terminal proof before settling a governed wait, including when complete usage arrives before stream closure/error/timeout. Retain harmless numbered cancellation events so restart replay stays contiguous. - Isolate accounting-test OpenCode config from the host plugin directory. - Add regression coverage and document the accounting, restart and connection-search behavior. ## Verification Current PR source: `0cf08efd75f8fb23f7989beda6dbda92587088bf`. The live matrix below measured frozen `09a776bcb1f77422156f8be11e13f8c29f43e7f7`; later review fixes are verified separately and do not relabel those runs. - Repository typecheck and build pass. - Current runner runtime and cancellation suites: 191 tests pass. Six new regressions cover stream end/error/timeout without provider-stop proof and contiguous cancellation acknowledgement/request replay; all six failed before the fix. The existing bounded cleanup case now explicitly supplies terminal proof. All 31 adapter accounting tests pass with isolated fixture config. - New checkpoint-rebinding and approval-criterion suites: 65 tests pass; runner HTTP integration: 29 tests pass. Unchanged executor/control-plane suites: 640 tests pass; database-backed connection suites: 68 tests pass. - Current retained evidence verification covers 222 file hashes across all fifteen original result artifacts. The three-case [published report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37651896525-1/index.html) and all eight screenshot hashes verify. The [three-case recovery report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37662587356-1/index.html) and all six screenshots also verify; the nine-result campaign did not publish. - Evaluation support: 1,805 Vitest tests pass, one skipped; 128 Node tests pass. Eval typecheck and catalog discovery pass. - Full local repository unit run was interrupted before the follow-up edits after three tool-access failures and one runner HTTP failure. Those failures pass in isolation; the 09a full run was interrupted after one rapid Slack callback-ordering failure and seven skill-service failures. All eight pass both isolated and with full-runner environment settings, and all 74 skill-service tests pass together; the subsequent full run reported two 15-second OpenCode accounting timeouts and was stopped with exit 130 to apply review fixes. The timeouts reproduce while copying this host’s 61 MB OpenCode config. All 31 tests pass after isolating config inside each fixture without increasing timeouts or changing assertions. A complete local full-suite pass is not claimed. Repository-wide CI also passes on the final review-fix head in [run 37669185952](https://github.com/paperclipai/paperclip/actions/runs/37669185952). Earlier database-skipped diagnostics and the older ENFILE run are retained and are not full-suite passing evidence. - Three-case live campaign [37651896525](https://github.com/paperclipai/paperclip/actions/runs/37651896525) passes all three original grades on frozen source 09a: 55/55 checks, eight succeeded run records, complete accounting receipts, matching checkpoint identities and no pending approvals. Campaign [37653533353](https://github.com/paperclipai/paperclip/actions/runs/37653533353) adds nine original passes (173/173 checks, eighteen succeeded records) on the identical source. Its other three jobs failed before runner assignment or any step while GitHub could not load the paid environment; those original infrastructure failures are retained. Campaign [37662587356](https://github.com/paperclipai/paperclip/actions/runs/37662587356) completes only those unstarted cells: all three original grades pass (57/57 checks, six succeeded run records). All fifteen exact cases now pass on source 09a: eight FAIL → PASS, seven PASS → PASS, zero new overall failures and zero pending pairs. Total current evidence: 285/285 checks and thirty-two succeeded run records, complete accounting receipts, matching checkpoint identities, no pending approvals or retry records. Earlier campaigns retain forty-seven additional run records and two known same-run recovery attempts; actual provider-call counts and invoices remain unknown. The nine-result campaign skipped publication and its public URL returns 403; original artifacts remain retained. Previous ba9 campaign [37645656840](https://github.com/paperclipai/paperclip/actions/runs/37645656840) completed 2 PASS / 1 FAIL: Claude restart/approval and Codex decline pass, while OpenCode resumes but times out searching for exact tool names and never creates the access card. That failure and incomplete cancelled-run accounting remain preserved; the new catalog error guidance targets this observed dead end. Campaign [37642957312](https://github.com/paperclipai/paperclip/actions/runs/37642957312) remains 0 PASS / 3 FAIL and exposed the now-corrected cross-run marker leak and unknown-criterion approval gap. The original baseline remains 7 PASS / 8 FAIL, first repair 2 PASS / 3 FAIL, and second repair 0 PASS / 3 FAIL. All fifteen selected cases are qualified by their original grades in this bounded trial. These live grades belong to 09a. Its twelve governed-wait checkpoints retain matching same-turn terminal fingerprints, but passing artifacts omit detailed event journals; the later six adversarial regressions qualify the new terminal-proof and replay guards separately. Final-head repository CI passes. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/15471#issuecomment-6044369780) is 5/5, confirms both findings are fixed, and reports no new actionable issues. All review threads are resolved and the PR has no merge conflicts. ## Risks - Governed cancellation drains accounting for up to fifteen seconds. Restart detachment gives a settling batch up to twenty seconds to finish. An incomplete receipt or unproven provider terminal still prevents successful qualification. - Claude prices are estimates, not invoices. The estimate assumes standard global API pricing and uses a conservative cache-write rate. Unsupported models, billers, and billing modes remain unpriced. - Approval identity matching must remain scoped to the current run and the requested approval. It does not authorize execution of a declined call. - Original baseline and final candidate have different merged master context. Exact-case outcomes are before/after observations, not isolated causal attribution to this repair. - No schema migration, fixture, oracle or grader change. Search-result guidance now treats fuzzy external matches as suggestions. Existing app authorization and provider-consent checks remain required. ## Model Used - OpenAI GPT-6 through Codex, with code editing, terminal tools, and test execution. The exact deployment identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (191 current runtime tests and 31 accounting tests; interrupted full-suite history and CI coverage are disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9fb955e9a2 |
fix(codex): gate models on the Codex CLI floor before the ChatGPT backend rejects them (#15399)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Codex adapter and the native runner run Codex in local hosts and in managed sandboxes, and the model catalog lists the models an operator can select > - The Codex backend accepts a new model with ChatGPT sign-in only from a recent enough Codex CLI. An older CLI fails every turn with `The '<model>' model is not supported when using Codex with a ChatGPT account.` > - After #14942 added `gpt-6.1-sol`, an operator selected it for an agent in a managed Daytona sandbox. The sandbox image still shipped Codex 0.156.0. The connection test and runs failed with that sentence, which reads like an account problem > - Paperclip only checked the Codex compatibility window (`>=0.149.0 <0.161.0`), so 0.156.0 passed and the failure surfaced from the backend without a cause > - This pull request records the verified Codex CLI floor for each gated model, compares the installed `codex --version` with that floor in the environment Test and in the remote runner, and names the backend rejection when it still happens > - The benefit is a precise, actionable message ("gpt-6.1-sol requires Codex CLI 0.159.0 or newer; detected 0.156.0; promote a sandbox image with Codex 0.160.0") instead of an opaque 400, and no doomed hello probe ## Linked Issues or Issue Description Refs #14942 **Bug: selecting `gpt-6.1-sol` in a managed sandbox fails with a ChatGPT account error.** **Steps to reproduce** 1. Promote a sandbox image that ships Codex CLI 0.156.0. 2. Create a Codex agent with a ChatGPT sign-in connection, select `gpt-6.1-sol`, and run the environment Test or a task in that sandbox. **Expected behavior** The Test or the run tells the operator that the Codex CLI in the sandbox is older than the model needs. **Actual behavior** The Test reports `codex_hello_probe_failed` with `{"type":"error","status":400,"error":{"type":"invalid_request_error","message":"The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account."}}`. A run fails with "The selected model is not supported by the current ChatGPT connection." **Evidence** - Codex 0.156.1 and older are rejected for `gpt-6.1-sol` with ChatGPT sign-in: https://github.com/openai/codex/issues/49396 - Codex 0.159.0 and 0.159.2 are accepted: https://github.com/openai/codex/issues/49464 and https://github.com/decolua/9router/issues/4471 (same error, fixed by raising the client version identity from 0.155.0 to 0.159.0) - Codex 0.157.0 release notes: "Add GPT-6 Sol and Luna to the model catalog": https://github.com/openai/codex/releases/tag/rust-v0.157.0 - GPT-6.1 Sol is included for Plus, Pro, Business, Enterprise, and Edu with ChatGPT sign-in: https://learn.chatgpt.com/docs/models ## What Changed - `packages/adapters/codex-local/src/index.ts`: add `minimumCodexCliVersionForModel` (`gpt-6.1-sol` → 0.159.0, `gpt-6-sol` and `gpt-6-luna` → 0.157.0, with the sources above), `parseCodexCliVersionOutput`, `codexCliVersionAtLeast`, and `CODEX_CHATGPT_MODEL_REJECTION_RE`. Models without a verified floor return `null`, so nothing probes them. The agent configuration doc describes the floors. - `packages/adapters/codex-local/src/server/cli-version.ts` (new): run `codex --version` where the run would execute it (local, SSH, or sandbox), compare it with the model floor, and build the `codex_cli_version_compatible` / `codex_cli_version_incompatible` checks. Map the backend rejection sentence to `codex_hello_probe_model_rejected` with the detected CLI version and a hint that separates a stale CLI from a plan that does not include the model. - `packages/adapters/codex-local/src/server/test.ts` (CLI lane Test): run the version check before the hello probe when the model has a floor. Skip the hello probe when the CLI is too old (`codex_hello_probe_skipped_cli_version`). When the probe still fails with the backend sentence, report `codex_hello_probe_model_rejected` instead of the generic `codex_hello_probe_failed`. The new codes do not match the auth-failure patterns, so a managed connection is not invalidated. - `packages/adapters/codex-local/src/server/acp.ts` (ACP lane Test): for remote targets, run the same version check against the shared `codex` the ACP server spawns. - `server/src/services/native-runtime/native-session-executor.ts`: after the compatibility-window check, compare the remote Codex with the configured model's floor and fail with `runner_remote_provider_artifact_incompatible: <model> requires Codex <floor> or newer with ChatGPT sign-in, received <version> from the sandbox image; promote a sandbox image with Codex 0.160.0 or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=...`. When a preinstalled Codex fails this check and an npm spec is configured, the existing fallback installs the pinned release. - Tests: adapter metadata, CLI-lane Test (too old, compatible, no floor, backend rejection), ACP-lane Test (too old, compatible, no floor), and the remote runner floor (nine version/model cases). - `doc/adapter-model-audit-2026-10-02.md`: record the floors and the reason. No catalog entry, default model, saved agent configuration, pin, lockfile, or image definition changes. ## Verification Local (Node 25.9, pnpm 9.15.4 via corepack): ```sh pnpm --filter @paperclipai/adapter-codex-local typecheck pnpm --filter @paperclipai/adapter-codex-local exec vitest run # 81 passed pnpm --filter @paperclipai/server exec vitest run \ src/services/native-runtime/native-session-executor.test.ts \ src/services/native-runtime/codex-runtime-compatibility.test.ts -t Codex # 84 passed ``` Also run locally: the full `native-session-executor.test.ts` file (533 passed) and the full Codex adapter suite (81 passed). Follow-up commit `9fd4d3b25` (Greptile P2: the ACP-lane version probe ignored the agent's configured env): the probe now receives the adapter's string-valued `env` entries, the same ones `buildCodexAcpConfig` hands to remote ACP runs, so a `PATH` override selects the same `codex` for the Test as for the run. New test covers a `PATH` + `CODEX_HOME` override and a dropped non-string entry. Re-run on that head: `pnpm --filter @paperclipai/adapter-codex-local typecheck` clean; full Codex adapter suite 503 passed (31 files). Not run here: `pnpm -r typecheck` for `server` (the direct `tsc --noEmit` was killed by the sandbox memory cap; the adapter package typecheck passes and CI covers the server), `pnpm build`, browser suites, and a live sandbox probe. No UI files changed, so token gates do not apply. Manual check for a reviewer: set an agent's Codex model to `gpt-6.1-sol`, point its environment at a sandbox whose `codex --version` prints `codex-cli 0.156.0`, and run the environment Test. The result contains `codex_cli_version_incompatible` with `Detected Codex CLI 0.156.0.` and no hello probe check. With Codex 0.160.0 the Test contains `codex_cli_version_compatible` and runs the hello probe. ## Risks - A floor is enforced for all authentication modes, because the runner does not know the credential method at verification time. An OpenAI API key on Codex 0.157.0 or 0.158.0 with `gpt-6.1-sol` is now rejected before launch. Paperclip pins Codex 0.160.0 everywhere, so this only affects installs that lag the pin, and the message names the fix. - The 0.159.0 floor for `gpt-6.1-sol` is the oldest stable release verified to work; 0.157.0 and 0.158.0 were not verified either way. If OpenAI accepts an older client, the floor can be lowered in one table. - The `codex --version` probe adds one short process run to the Test only for models with a floor. - Rollback: revert this pull request. No data or configuration migrates. ## Model Used Claude Fable 5.1 (`claude-fable-5-1`), Anthropic, with extended thinking, tool use, and web research, operating as a Paperclip agent through Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <noreply@paperclip.ing> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
f669194298 |
fix(runner): propagate configured environment to future turns (#15451)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runners start provider processes and enforce a separate tool environment policy. > - The server resolves task environment bindings at each run boundary. > - Fixed launch allowlists dropped custom variables after resolution. > - Warm process reuse and a blanket fingerprint exclusion also hid configuration changes. > - This pull request carries a bounded, server-selected list of task variable names through each process boundary. > - The benefit is that later turns and tool commands receive configured values while ambient host secrets stay excluded. ## Linked Issues or Issue Description **What happened?** A native Codex agent could not use configured task credentials. Adding the variables in Settings did not fix the next turn. The provider sanitizer, runner launch, Rust provider launch, and shell policy each used fixed allowlists. Effective config fingerprints also excluded all variables with the `PAPERCLIP_` prefix. **Expected behavior** Explicitly bound task variables must reach provider processes and tool commands. Added, changed, and removed values must take effect at the next run boundary. A running turn keeps its original configuration. **Steps to reproduce** 1. Start a native Codex task without a custom environment binding. 2. Add a fake `PAPERCLIP_PAGE_BUCKET` value and a fake Pages credential binding in Settings. 3. Continue the task and inspect the tool environment. 4. Before this fix, those values are absent even from a fresh runner launch. **Paperclip version or commit** Reproduced at `b31558064`. The fix is rebased on current master. **Deployment mode** Self-hosted server with a native process runner. Related changes: [the legacy Codex MCP environment fix](https://github.com/paperclipai/paperclip/pull/13321) and [ambient server-secret exclusion](https://github.com/paperclipai/paperclip/pull/12870). These affect different launch paths. This change preserves their credential boundaries. ## What Changed - Capture scoped task bindings after resolution, before managed provider credential injection. Mint and validate the names-only projection at native dispatch before host inheritance. Legacy adapters retain their previous environment limits. - Strip user-supplied projection markers from agent, environment, project, and routine config. - Carry selected values through the Codex, ACPX, OpenCode, runnerd, and Rust subprocess launch boundaries. - Add selected names to native and ACPX Codex shell include lists. Keep selected values out of command arguments, including selected bootstrap values. - Reject malformed projections, reserved authority and loader names, missing values, null bytes, and oversized input. - Replace a retained native process when projected values change. Keep unchanged processes reusable. - Fingerprint custom namespaced variables while excluding known generated runtime variables. - Document next-run behavior and add regression coverage. ## Verification - Red: the permanent reproduction failed at four launch/tool boundaries and the namespaced fingerprint check. Two control checks passed. - Green: the initial regression plus existing Codex environment and shell tests passed (39 tests). - Server config resolution, fingerprints, and native-session suites passed (653 tests), including addition, rotation, removal, and unchanged warm-session reuse. - Rust regression tests passed. A real shell child received added and rotated values, then lost them after removal. Unselected host variables stayed absent. - `pnpm -r typecheck` and `pnpm build` passed after rebase. The full `pnpm test:run` was attempted but could not complete: fresh embedded PostgreSQL databases fail during bootstrap on this macOS host. An isolated suite and a disposable native `initdb` probe reproduced the failure before test execution. `shmget` reports `No space left on device` because the host has exhausted shared-memory IDs. This is not disk exhaustion. The run was stopped after confirming the external setup failure. CI results will be recorded separately. - Runner boundary suites: 361 tests passed. Two process-launch errors during concurrent binary staging passed on isolated rerun. - Rust Codex provider and process supervisor integration suites: 97 passed, 2 intentionally ignored. - ACPX shell and selected-bootstrap argv regressions: 3 failed before the fix, then all 55 relevant tests passed. - Review compatibility regression: 129-variable and large-value legacy configurations failed before the correction and passed after moving native-only validation to dispatch. - Final review head `3f11e8d25`: repository typechecks and production build passed. CI completed its implementation checks; one general-server shard hit SQL `40P01` in `heartbeat-runtime-skills.test.ts` during its `beforeEach` table truncate (1,199 tests passed in that shard). The shard passed on its single rerun. All CI checks for this head are green. Greptile reviewed this head at 5/5 with no new actionable findings; the compatibility thread is resolved. - No live provider credentials or model calls are required by these tests. ## Risks - Configured task credentials now reach the tools they were configured for. The controller selects names only after existing scope and secret-binding authorization. - TypeScript and Rust validate the same bounded projection. Their reserved-name rules must stay aligned. - A changed projection replaces an idle provider process. Unchanged values preserve reuse. Active turns retain their original environment. - No database migration or API schema change. > This fixes existing runner configuration behavior. It does not add a new roadmap capability. ## Model Used - OpenAI GPT-6 (Codex), with reasoning, local code execution, and repository tools. The session does not expose a more specific API model identifier or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
799e4d556f |
fix: make accounting durable and synchronize cost reporting (#14997)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a6306ba606 |
feat(runner): consolidate Cursor production integration (#15075)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native Runner keeps provider sessions under company authority, approvals, budgets and durable recovery. > - Cursor work was spread across candidate branches. The published branch lacked later plan, permission and cleanup fixes. > - Production also needs public installation and matching runtime assets for local and Daytona execution. > - This pull request consolidates Cursor onto current mainline recovery behavior and completes that installation path. > - The installed v11 release passed focused local and Daytona qualification after the generic mode and lifecycle cleanup. The later model-selection correction and current mainline merge produce v14 artifacts that need matching release qualification. > - Cursor admission is enabled in source; publish only an artifact combination with matching qualification. Native AskQuestion and complete per-run dollar accounting remain excluded. ## Linked Issues or Issue Description Refs: #14435, #14631, #14669, #14699, #14724. This completes the Cursor implementation by @cryppadotta from combined source `22c78242a4e0c2369fecf0c2dc4e7600fbad6706`. It preserves newer mainline recovery, completion and warm-directory behavior. Pi and Copilot remain gated. ## What Changed - Generate named Rust and TypeScript ACPX release profiles from one manifest. Share runtime pins with packaging and server verification. Preserve vendor runtime versions; bind the updated ACPX patch to Cursor profile v14 and reject stale generated declarations at build/typecheck. - Remove ACPX model allowlists, including the former Codex and Pi restrictions and the duplicate developer test-drive gate. Send any explicit model ID unchanged to its provider and verify the effective selection before prompting. The bundled ACPX package forwards unlisted IDs, rejects mismatched acknowledgements, and restores the exact selection after session load. It does not expand Cursor model aliases. Provider rejection, mismatch, or missing model controls fails without a fallback. Model examples live in evaluation fixtures, outside runtime declarations. - Add pinned Cursor execution, contained instructions, exact model verification and Agent/Plan/Ask modes. - Carry an opaque generic `mode` identifier in shared native execution, sidecar, Rust and recovery contracts. The provider adapter owns supported modes, defaults, native translation and acknowledgement. - Keep native RPC recognition, accepted-plan interpretation and permission evidence behind provider adapters. Shared settlement and recovery verify normalized facts and their committed evidence. - Replace the Cursor-only warm-attachment branch with a runner-owned capability. Only Cursor opts into it. Move profile compatibility and optional usage parsing into provider metadata and adapters. - Write generic plan-wait receipts. Read exact historical Cursor receipts through a separate compatibility decoder. Reject mixed formats and preserve existing authority checks. - Carry native plans, semantic questions, todos, child activity, permission identities and partial usage diagnostics through the Runner. - Preserve durable response delivery, cancellation, warm ownership and process retirement. - Finish accepted planning runs successfully. Keep their tasks open for explicit direction. Acceptance does not start implementation. - Ship `paperclipai runtime setup cursor` and its provisioner through the public package. npm installation does not download Cursor. Setup uses the OS account's closure-keyed cache so system-wide npm packages can remain read-only. Run it as the Paperclip service account. - Include Cursor in normal provider packs and Daytona images for macOS ARM64/x64 and Linux x64. - Reject stale release packs by source revision and current ACPX/Cursor pins before assembly writes files. Verify current Cursor version/profile/closure again at runtime. - Ship all three daemon targets and the expected Linux image-pack identity. A macOS controller uses its packaged Linux daemon for Daytona. Image mismatches fail before provider launch. - Use the vendored Runner boundary for installed readiness probes. Verify the actual installed Cursor probe. - Verify compiled public Daytona plugins and their release versions in installed smokes. - Record exact artifacts, the acceptance matrix, retained failures, supported capabilities and rollback behavior in the [readiness report](https://github.com/paperclipai/paperclip/blob/codex/cursor-production-readiness/doc/plans/2026-10-03-cursor-production-readiness.md). ## Verification - Current head `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037` merges mainline `faa8e452c73bae5e044dd6379179a00106abb131`. It keeps Cursor plan and cancellation guards alongside mainline historical-question filtering. The evaluation catalog includes both Cursor and expanded adapter accounting cases (683 total). Recursive typecheck, full build, 696 lifecycle/recovery tests, 45 fixture tests and fixture typecheck passed. Current-head CI passed: 56 successful checks, one neutral and four skipped. [Complete CI](https://github.com/paperclipai/paperclip/actions/runs/37557996535). The fresh Base Greptile review is 5/5 on this exact head, with 304 files reviewed, zero new comments and zero unresolved threads. The user authorized overriding the CODEOWNER review gate after checks passed; no failing checks are overridden. Prior results below retain their own head identities. - Corrective head `3d2b168366258036f6b6a6fccb382c49138cc601` fixes the post-merge Apex finding. Automatic-review and new-evidence reconciliation preserve pending child results and recheck delivery under the status lock before completing. Account repair now excludes unrelated secret consumers and requires the failed agent's identity. Regression coverage includes the commit race, delivery statuses, current-run/current-intent exclusions, repeated reconciliation, both database reconciliation paths, and credential consumer boundaries. All 184 affected tests, server typecheck and server build passed. Current-head Base Greptile review is 5/5, with 304 files reviewed, zero new comments and zero unresolved threads. Current-head CI passed: 56 successful checks, one neutral and four skipped. [Complete CI](https://github.com/paperclipai/paperclip/actions/runs/37535994724). This Base review is distinct from the earlier Apex review. - Merge head `5957c257a` reconciles mainline `b508a05c4`. It preserves both accepted-plan waits and pending-child-completion checks, current provider selectors, task-creation response identities, and mainline ACPX missing-file handling. The combined patch is bound to Cursor profile v14; historical records keep their original identities. - Merge head `5957c257a` passed recursive typecheck, full build, 43 installed ACPX/package contracts, 107 provider UI and plan/recovery tests, 593 database-backed lifecycle tests, 49 profile/native contract tests, 45 Product E2E fixture tests, fixture typecheck, token gates, three provider-free browser task-creation cases, and Runner conformance/replay checks. Its complete CI passed (55 successful checks, one neutral and four skipped), while Apex returned 2/5 with a child-delivery finding addressed below. - The local full-suite attempt again failed the unchanged Git streaming test (360-second timeout) and was stopped. The concurrent local Rust attempt failed four unchanged Codex process/deadline tests; all four passed serially without code changes in 7.29 seconds after removing the competing test load. These failed commands are retained and are not reported as full-suite passes; the fresh Linux CI runs are tracked separately. - The previous head `907bdb2a2778c7ffeb4a662a91460c9d1ddfc9c5` earned Apex 5/5 with zero comments after fixing all three findings: per-user install cache, stale release-pack rejection, and public Linux smoke account/home handling. Its real built installer passed from read-only public packages on macOS ARM64 and Linux x64. All 137 release-registry checks and 64 ACPX package contracts passed. That review does not cover this mainline reconciliation. - Prior `beadd3654` passed the full CI matrix; its one unchanged chat test failure and successful single retry remain in the [CI history](https://github.com/paperclipai/paperclip/actions/runs/37521449327). Historical results below remain attributed to their original builds. - Fixture follow-up `dd59d7e82b103a88b7cbd7d2c38b612c0fbbff7a` removes provider-specific model choices from generic offline ACPX tests. The fake sidecar preserves the model and session identity selected at open through suspension. Affected verification passed: 106 Rust tests and 73 TypeScript tests. This commit changes test code only; the production-code checks below retain their recorded identities. Its CI and Greptile review later passed; those results belong to that historical head. - Model-selection cleanup `9a070808b48960a41fdfd369ae0636b95af82459`: 252 focused Runner tests passed (six platform skips), covering all six ACPX agents, native model acknowledgement, rejected selections, installation integrity and recovery identity. The merged branch passed recursive typecheck, full build, token gates, server admission (19 tests), and the Product E2E catalog (45 tests). The acceptance catalog passed all four tests. The full Rust suite passed: 643 tests, 2 ignored. It verifies sidecar acknowledgement of unlisted models and rejection of model mismatches. The final commits only update Rust tests; production sources match the verified build at `65ec3279ac50185e3cda109b5cfd9b4f56105de0`. No new paid provider calls were made. - The merge preserves both Cursor and the new mainline public-MCP fixture cases. Auto-merge remains disabled; the latest follow-up status is recorded above. The local `pnpm test:run` attempt hit the unchanged Git streaming test's 300-second timeout and was interrupted before merging mainline. The broad Runner attempt found obsolete single-model assertions plus three macOS fixture-path failures caused by a `/private/tmp` override. The assertions are corrected; affected TypeScript checks passed with the standard macOS temporary directory, and the complete Rust suite passed. Neither interrupted command is a full-suite pass. - Earlier declaration-cleanup head `6f4a5e9e2` passed recursive typecheck, build, Rust and focused tests. Its CI later exposed a test expecting duplicated Grok digest literals. The current source fixes that assertion to compare launcher bytes with the shared manifest. Historical successes and failed attempts are retained; no new live provider qualification is claimed. - Previous head `e75fde6098b0ddd8cec765bfb6ecaeecb88a26a6` passed complete CI (56 successful checks, one neutral, four skipped) and Greptile 5/5. [Historical complete CI](https://github.com/paperclipai/paperclip/actions/runs/37489112305). Those results are not claimed for the cleanup head. - Frozen live application: `d7b696f9b8f79095233e9e3d56d23e6a6018dd48`. Public package version: `0.0.0-cursor-verify.3d0c9b7761c6`. The declaration cleanup preserves release pins and does not relabel that tested artifact as a build of the new source. Mainline through `e34abee670` was reconciled while preserving accepted-plan waits, provider-capacity handling, and both Cursor and public-MCP fixtures. - Clean normal installation, explicit Cursor setup and daemon resolution passed on macOS ARM64, macOS x64 under Rosetta, and Linux x64. npm lifecycle hooks ran without silently downloading Cursor. - Historical v11 live matrix: **18/18 passed with cleanup** (nine local, nine Daytona) after the generic mode and lifecycle cleanup. The campaign has 23 attempts; all five failures and their diagnoses remain recorded. Exact case identities, hashes and limits are in the readiness report. All provider calls are real, use the explicit Luna model and company-bound credentials, and run without qualification or runtime-asset overrides. - The immutable Daytona image is `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:d6259b6bba094702c13fc2283bd85550849c1c53145b656fb2746778f9fa1747`. The public Daytona plugin is installed independently and its version is checked. - Recursive typecheck, full build, token gates and Runner contract/conformance/replay checks passed on the frozen application. Its complete Linux CI suite passed. The duplicate local full-suite command was incomplete after timing failures; affected repeats passed, but that command is not reported as a clean pass. - Qualification fixtures passed typecheck, 1,675 Vitest tests (one skip), 128 Node checks, three provider-free browser tests, and 150 focused lifecycle tests after the final diagnostic correction. The affected legacy Cursor command file also passed all five tests after removing its shorter 10-second override; it now inherits the suite’s standard 15-second timeout. Greptile is 5/5 on `e75fde609` with no unresolved review threads. CI results above are recorded separately from historical build results. ## Risks - Cursor v14 includes the updated ACPX dependency patch and release identity. The v11 live matrix and image below remain historical evidence. They do not certify new v14 package/image artifacts. - ACPX accepts models beyond the qualification fixtures. Availability and entitlement depend on the provider. Successful configuration is not a claim of live qualification for every model. - Shared mode is an opaque identifier. Provider adapters own its meaning. Incompatible historical sessions remain fenced; exact committed plan waits and task history remain inspectable. - Native AskQuestion is excluded. Paperclip semantic questions are supported. Authoritative per-run dollar accounting is unavailable; partial counters remain diagnostics and unknown cost is not zero. - Image input, detailed native diffs, deeper child transcripts and native plan-file export remain follow-ups. - macOS x64 has clean-install and daemon-startup proof under Rosetta, not a separate live campaign on Intel hardware. - Release only the tested package/image combination. Merging this PR does not publish npm packages or deploy that image. Later builds need their own release verification. Rollback disables new Cursor admission while preserving records and recovery inspection. - A model can fail an exact instruction: one cancelled-plan attempt returned the wrong summary marker despite correct cancellation. The unchanged repeat passed; both results remain in the report. > ROADMAP.md was checked. This completes existing native Runner/Cursor work; it does not add an independent core feature proposal. ## Model Used OpenAI Codex, GPT-6. The exact serving variant and context window are not exposed in this session. The agent used reasoning, repository inspection, code execution, protocol tests and browser-backed Product E2E tools. Cursor acceptance uses the explicit `gpt-5.6-luna[context=272k,reasoning=medium,fast=false]` model. That is the evaluated provider model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — affected suites passed; full CI and the retained local failed attempts are recorded separately above. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — 56 successful checks, one neutral and four skipped on `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037` - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — fresh Base review passed on `f7ec5cc1f0e30c62a829c016ff2013a2a9d79037`; zero new comments and no unresolved threads. The earlier Apex finding remains fixed. - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a590ab769d |
Give managed agents persistent cryptographic identities (#15352)
Give agents persistent Ed25519 identities encrypted with the existing instance master key. Create keys transactionally for new agents and lazily before supported managed runs, expose public identities in the API and agent UI, and protect private material during runtime delivery and output persistence. Preserve identities in recovery backups while giving imported and development-cloned agents fresh keys. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
6c36c07a4f |
feat(adapters): add GPT-6.1 Sol and refresh shared coding harness pins (#14942)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents run through coding-agent adapters and the native runner. Both use the same installed provider CLIs, model catalogs, and reasoning controls. > - OpenAI released GPT-6.1 Sol (`gpt-6.1-sol`) in Codex. Anthropic released Claude Sonnet 5.5. The static Codex, Bedrock, and OpenCode catalogs do not list these IDs. > - The shared provider pack pins Codex 0.156.0 and OpenCode 1.18.32. The evaluation image pins older Grok, Gemini, Kimi, Cursor, and GitHub CLI releases. Codex 0.156.0 has no bundled metadata for GPT-6.1 Sol. > - A model entry without a current harness, or a harness pin without its runner integrity checks, fails at run time. > - This pull request adds the verified model IDs and moves the harness pins, executable digests, controller checks, and image pins together. > - The benefit is that operators can select the current models, and the native and local adapters share one current CLI installation. ## Linked Issues or Issue Description Refs #13829 and #13838 (the September 22, 2026 model and harness refresh). Related pull requests: #14993 (merged October 5, 2026, superseding #14816) added the direct Claude Sonnet 5.5 entry and refreshed the Claude runtime to Agent SDK 0.3.286 / Claude Code 2.1.286. This pull request does not change the Claude runtime or the direct Claude model list; it keeps the #14993 pins and adds only the Bedrock Sonnet 5.5 ID. After #14993 merged, this branch was rebased onto `master` (October 5, 2026). The six overlapping pin regions (`docker/daytona-runner/Dockerfile`, `docker/daytona-runner/README.md`, `package.json`, `pnpm-workspace.yaml`, `packages/adapters/claude-local/src/index.test.ts`, `packages/paperclip-runner/src/backends/native-backend-factory.test.ts`) were resolved by keeping this pull request's Codex 0.160.0 and OpenCode 1.18.34 pins next to #14993's Claude 0.3.286 / 2.1.286 pins, taking the union of the Sonnet 5.5 model IDs in the Claude test, and merging both README paragraphs. The Sonnet 5.5 effort and CLI-gate lines in the Claude adapter were identical in both pull requests and merged without a diff. #14917 and #14918 reordered the Claude and Codex model lists earlier; the new entries sit where those ordering rules put them. Sources checked on 2026-10-02: - [OpenAI Codex models](https://learn.chatgpt.com/docs/models): GPT-6.1 Sol uses `gpt-6.1-sol`, supports reasoning efforts from Light to Ultra, and has Standard and Fast modes at launch. The page also records that `gpt-5.4` and `gpt-5.4-mini` retired from Codex with ChatGPT sign-in on August 31, 2026, and that `gpt-5.5` retires on October 14, 2026. Neither retirement applies to the OpenAI API. - [Codex CLI releases](https://github.com/openai/codex/releases) 0.157.0 through 0.160.0. The bundled model metadata in the 0.160.0 Linux binary contains `gpt-6.1-sol`. - [Claude Sonnet 5.5](https://platform.claude.com/docs/en/models/sonnet-5-5/overview): Bedrock ID `anthropic.claude-sonnet-5-5`, released September 28, 2026. - [OpenCode releases](https://github.com/anomalyco/opencode/releases) 1.18.33 and 1.18.34 (fixes only). The OpenCode model registry lists both added provider-qualified IDs. - npm `latest` tags for `@xai-official/grok` 1.0.46, `@google/gemini-cli` 0.62.0, and `@moonshot-ai/kimi-code` 2.1.1. [xAI](https://docs.x.ai/docs/models), [Google](https://ai.google.dev/gemini-api/docs/models), and [Kimi](https://www.kimi.com/code/docs/en/kimi-code/models.html) list no newer coding models. - Cursor CLI 2026.10.01-e373342 is the version the official installer resolves. The pinned digest is the SHA-256 of the versioned Linux x64 archive. - [GitHub CLI 2.102.0](https://github.com/cli/cli/releases/tag/v2.102.0) (security fixes). The pinned digest matches the release `checksums.txt`. ## What Changed - Codex adapter: add `gpt-6.1-sol` to the model list, the Fast mode list, and the Ultra effort set. It is the first entry: #14918 orders the list newest version first, and its description notes the ChatGPT app lists GPT-6.1 Sol first. Update the adapter documentation text. - Claude adapter: add `us.anthropic.claude-sonnet-5-5` (Bedrock Sonnet 5.5) to the Bedrock catalog in the newest-Sonnet slot after Opus 5.5; `us.anthropic.claude-sonnet-5` moves into the older-Sonnet group, matching what `sortClaudeModels` from #14917 produces at runtime. Any Sonnet 5.5 ID (direct or Bedrock-qualified) now gets the documented `xhigh` and `max` efforts and requires Claude Code 2.1.284 or later on the CLI lane (the Claude Code changelog entry for 2.1.284 adds `claude-sonnet-5-5`). These two lines are identical to the ones #14993 merged, so the branch carries no diff for them. - OpenCode adapter: add `openai/gpt-6.1-sol` and `anthropic/claude-sonnet-5-5` to the static fallback catalog. - Codex runtime pin 0.156.0 → 0.160.0 in the root and workspace overrides, the runner package, the Codex ACP package patch, the qualified ACPX profiles, the Linux x64 executable digest, the Rust provider backend and its tests, the provider-pack manifest pins, the remote controller pins, the sandbox npm install spec, and the opt-in qualification scripts. - Remote Codex compatibility window: upper bound 0.157.0 → 0.161.0. The minimum stays at 0.149.0. - OpenCode runtime pin 1.18.32 → 1.18.34 in the runner package, the materialization script, the server and Rust qualified versions, the eval and live-session labels, fixtures, and the configuration label. - Evaluation image (`docker/daytona-runner/Dockerfile`): Grok CLI 1.0.46, Gemini CLI 0.62.0, Kimi Code 2.1.1, Cursor CLI 2026.10.01-e373342 with its digest, GitHub CLI 2.102.0 with its digest, Codex and OpenCode version probes, and the refreshed lockfile digest. The Claude Code 2.1.286 probe comes from #14993 and is unchanged here. - `pnpm-lock.yaml` is not part of this pull request. The repository's pull request gate rejects lockfile edits, and the refresh bot regenerates the lockfile on master (the same flow #13838 used). The Dockerfile `PAPERCLIP_RUNNER_LOCK_SHA256` default is the digest of the lockfile that `pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile` (the refresh workflow's command) produces for the combined pins on the rebased branch (`e1856797…`); that lockfile differs from master only in the `@openai/codex` 0.160.0 platform packages, the `@anthropic-ai/claude-agent-sdk` 0.3.286 override that #14993 introduced (the open refresh-bot pull request #14872 carries that part), `opencode-ai` 1.18.34 with its Linux x64 baseline, and the `codex-acp` patch hash. - Documentation: runner README, runner compatibility doc, environment variable example, and a new `doc/adapter-model-audit-2026-10-02.md` with sources and deferred items. - Tests: Codex adapter catalog, server adapter models, Codex compatibility window, native session executor pins, runner package contract, OpenCode materialization, and UI effort options. Unchanged on purpose: Claude Agent SDK 0.3.286 / Claude Code 2.1.286 (already on `master` from #14993), ACP bridges (`acpx` 0.13.1, `claude-agent-acp` 0.73.0, `codex-acp` 1.6.2; newer upstream releases need a separate qualification), the native Grok runtime 1.0.13, Pi 0.84.2 / 0.87.1 (the Pi 1.0 runner stack covers it), and Hermes 0.19.0 (current). `gpt-5.4` and `gpt-5.4-mini` stay in the picker because the OpenAI API still serves them. ## Verification Run on Linux x64 with Node 25.9.0 and pnpm 9.15.4 after `pnpm install --no-frozen-lockfile` (the refreshed lockfile stays local; see above). The results below were re-run on the rebased head (October 5, 2026) for the suites the conflict resolution touches; the other rows are from the original run and are covered by CI on every push: - Rebased head: `packages/adapters/codex-local` 482 passed; `packages/adapters/claude-local` 340 passed, 4 failed (`execute.remote`, `test.probe`, `execute.acp-fallback`, `acp` spawn/env-hardening cases that fail identically on unchanged `master` in this host environment); `server` adapter-models + codex-runtime-compatibility + native-session-executor + adapter-registry 607 passed, 1 failed (the same adapter-registry override-pause case as before, also failing on `master` here); `packages/paperclip-runner` native-backend-factory + qualified-profiles 36 passed; `ui` codex-reasoning-effort + config-fields + model-utils 19 passed. Rust, full typecheck, build, and the Docker image are left to CI as before. - `vitest run` in `packages/adapters/codex-local`: 13 passed. `vitest run` in `packages/adapters/claude-local` (whole package, including the new Sonnet 5.5 gate and effort tests): see the latest CI run and the comment below. `vitest run` in `packages/adapters/opencode-local`: 48 passed, 1 failed (`runtime-config.test.ts` reads the host `PAPERCLIP_OPENCODE_PROVIDERS` variable; it fails the same way on the unchanged base). - `vitest run src/__tests__/adapter-models.test.ts src/services/native-runtime/codex-runtime-compatibility.test.ts src/__tests__/adapter-registry.test.ts` in `server`: 84 passed, 1 failed (`adapter-registry.test.ts` override pause test; it fails the same way on the unchanged base). - `vitest run` in `ui` for `codex-reasoning-effort`, `agent-setup-fields`, `config-fields`, and `ComposerRunSettingsPicker`: 25 passed. - `node --test test/acpx-codex-package-contract.test.mjs scripts/materialize-opencode-binary.test.mjs scripts/runner-protocol-eval-campaign.test.mjs` in `packages/paperclip-runner`: 23 passed. The package contract test verifies the installed Codex ACP executable digest and the 0.160.0 patch pin. - `vitest run src/drivers/acpx src/backends src/drivers/opencode src/live/live-session.test.ts` in `packages/paperclip-runner`: 626 passed, 5 failed, 1 skipped. The 5 failures (`installation-integrity.test.ts` `/proc/self/fd` module loading and one OpenCode answer-selection test) also fail on the unchanged base under Node 25; Linux CI runs Node 24. - `pnpm run test:opencode:qualification` in `packages/paperclip-runner` against the installed OpenCode 1.18.34 executable: passed. - `codex --version` from the installed pack prints `codex-cli 0.160.0`. The Linux x64 executable digest `12eb3e81…652aad` was computed from the `@openai/codex@0.160.0-linux-x64` archive after checking its registry `dist.integrity`. - `pnpm check:token-gates`: all gates clean. - `pnpm run typecheck:typescript` in `packages/paperclip-runner`: passed. Package typechecks ran one at a time; see the comment below for the server and UI results. Not run here, and needed from CI: - Rust tests and `pnpm -r typecheck` / `pnpm build` for the server (no `cargo` in this environment; the server typecheck prepares the runner vendor build). - The Docker evaluation image build and the real-binary Codex startup and session-resume probes (no Docker; the probes need the compiled `paperclip-runnerd`). The trusted CI runner workflow covers them. - Authenticated inference with any new model. This change is metadata and startup validation only. ## Risks - Codex 0.160.0 changes the bundled model catalog and app-server behaviour (authoritative provider catalogs, incremental running-turn tracking). The patched `codex-acp` 1.6.2 bridge is unchanged and declares `^0.148.0`; it worked with 0.156.0 under the same override. If CI probes show a protocol change, the pin can return to 0.156.0 by reverting this pull request. - The compatibility window upper bound moves to `<0.161.0`. Remote images with Codex 0.157 to 0.160 become accepted. Older images stay accepted down to 0.149.0. - Until the refresh bot lands the regenerated lockfile on master, the Dockerfile lockfile digest default does not match the committed lockfile. The trusted CI workflow computes the digest from its own resolution at build time, so this affects only a local build that passes no digest. - Existing saved model selections and effort settings are not changed. Agents on `gpt-5.4` or `gpt-5.5` with ChatGPT sign-in need a model change before the OpenAI retirement dates; that is documented, not enforced. - Rollout order: deploy the controller and runner from this change before promoting a sandbox image that carries these pins. Older controllers reject the new provider-pack pins. ## Model Used - Claude Fable 5.1 (Anthropic, model ID `claude-fable-5-1`), 1M context window, adaptive thinking, tool use. The model ran as a Paperclip agent through the Claude Code harness, performed the web research, edited the code, and ran the tests listed above. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <noreply@paperclip.ing> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
a65ca09508 |
fix(runner): settle accepted results after shutdown failures (#15217)
## Thinking Path > - Paperclip manages AI agents and the tasks they perform. > - The native runner saves tool results and completion reports before it releases a session. > - Large project discovery responses can exceed the durable command limit. > - A shutdown failure can leave a saved answer waiting for workspace repair. > - Recovery reused the old assessment for a different status decision, which violated a database constraint. > - This pull request bounds discovery responses and lets recovery commit the saved result after workspace repair. > - The benefit is a task that reaches its correct final status without another provider turn. ## Linked Issues or Issue Description **What happened?** A native run can save its final answer, fail during shutdown, and leave the task In Progress after workspace repair succeeds. Reconciliation tries to reuse the failed-workspace assessment for a new decision. The one-decision-per-assessment constraint rejects the write. Replaying the old decision can also retain a fresh coordinator lease. Separately, retained session cleanup only recognizes the old `adapter_failed` label. The project-list tool returns full project records, including large descriptions and workspace configuration. A large response exceeds the runner's durable command limit. The settlement diagnostic previously recorded only a failure flag. **Expected behavior** Project discovery stays within the command limit. Recovery finishes the saved result after workspace repair, releases its lease, and preserves the original error for inspection. It does not repeat provider work or relax session ownership checks. **Steps to reproduce** 1. Return large project records from `list_projects` and observe an oversized semantic result. 2. Persist an accepted native completion result, then record a shutdown failure. 3. Finalize with a failed workspace, repeat that attempt, then record successful workspace repair. 4. Reconcile the run. Before this fix, the issue stays In Progress. **Paperclip version or commit** Reproduced on `a386a599983519eb1d399f8b770bfccdb2a74762`. **Deployment mode** Self-hosted server with Paperclip Runner. Related transport work: #12208 drains queued events; #12241 resumes interrupted semantic calls. This change addresses bounded project discovery and accepted-result finalization. ## What Changed - Read bounded project summary projections from the database and return at most 50 authorized summaries with a continuation cursor and explicit description truncation. Agent and run trust boundaries narrow the database candidates; project-specific policies still receive full authorization. Only visible projects determine continuations. The default project-list API remains unchanged. - Record bounded, content-free settlement failure causes for command limits, storage errors, and rejected dispatch. - Include workspace state in assessment identity. Preserve the initial assessment for interrupted finalization, and commit replacement assessment and decision references together. - Release the coordinator lease when an existing decision is replayed, without repeating its effects. - Clear stale errors when recovery succeeds and retain them in `recoveredExecutionFailure`. - Accept both current and legacy close-failure labels in the existing exact-state cleanup path. - Add regression tests and update the tool contracts and recovery documentation. ## Verification - Red: the new project paging, settlement diagnostic, current cleanup label, and repaired-workspace regressions failed on the original implementation. - Green: protocol/catalog/tool checks (117 tests), the full project-tool and finalizer suites (47 tests), cleanup ownership cases (70 tests), and cleanup sweep cases (4 tests) pass. Database-backed pagination covers large descriptions/configuration, complete enumeration, uppercase cursors, agent/run restrictions, project-policy scope contributions, and identical results/cursors when hidden projects are added. - The initial CI failures in interrupted Board waits, contended endpoint proof, and semantic schema validation were reproduced and fixed; all affected cases pass locally. - `pnpm -r typecheck` — passed. - `pnpm build` — passed. - `pnpm test:run` — started before the review corrections; it spanned several source revisions and was stopped after reporting old-behavior and timing failures. It is not claimed green. Fresh project and recovery suites pass; the recovery suite also passes all 25 cases with the broad runner’s isolated home/config. The Slack timing case passed in isolation. Latest-head CI is the authoritative complete test matrix. - Existing authorization suite — 66 tests passed. - Latest-head CI on `8e5763d915aa6f375bdab6601996899ea01496fc` — 55 checks passed, 4 intentionally skipped, no pending or failing checks. - Greptile — 5/5, zero unresolved threads on the same commit. - `git diff --check` — passed. ## Risks - `list_projects` now returns summaries. Callers must follow `nextCursor` and use the authorized project API for full records. Candidate narrowing is only an optimization: project policy and responsible-user authorization remain authoritative. - Assessment identity changes for workspace finalization. Existing evidence remains intact; no migration is needed. - Cleanup still requires matching identities, settled tool evidence, and verified process ownership. Unknown tool outcomes remain blocked from session reuse. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code execution, and GitHub tools. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
018993140f |
feat: let agents name prompt-only tasks (#14761)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Users create tasks with a title and a description. > - A required title adds work when the prompt already explains the request. > - An agent can name the task once it reads that request. > - This pull request accepts prompt-only tasks and starts them with a short prompt slice. > - A scoped title tool lets the assigned agent replace that slice early without changing execution state. > - A live browser eval checks the real agent call, saved title, audit entry, and preservation of user titles. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: task creation, shared contracts, database, server, runner tools, and board UI. **Problem or motivation** Users must currently write a title before they can submit a detailed task prompt. The agent has enough context to write a useful title itself. **Proposed solution** Make the title optional when a description is present. Save the first 120 characters of the normalized prompt as a provisional title. Ask the assigned agent to call `set_task_title` early. Use an atomic provisional-title guard to preserve titles supplied or edited by users. Keep explicit titles supported. Related: #14543 and #14556 concern empty-title submission. This change intentionally enables that submission when a prompt is present, instead of requiring a title. ## What Changed - Add the `titleNeedsGeneration` field with an idempotent migration. Keep existing titles unchanged. - Add `PUT /api/issues/:id/title` and the native and legacy `set_task_title` tool. Enforce company access, active-run ownership, shared, bounded retry receipts across native/HTTP calls, and transactional audit logging. Refresh external-object links after commit, with the same feature gate and plugin detectors as ordinary title edits. - Add early naming guidance in Standard, Ask, and Plan task context. Preserve the description, status, and assignment. - Allow prompt-only root and child task creation, plus draft restoration in the New Task dialog. Keep user titles supported. - Add an opt-in Product E2E suite for prompt-only Standard and Ask tasks, plus an explicit-title control. It checks actual provider calls within the first five tools, persisted state, audit attribution, and the reloaded UI. - Preserve a closed vocabulary of API key maintenance phrases in declared prose while rejecting opaque credential suffixes. Add one bounded naming retry after wording is rejected, without treating the rejected call as a saved title. - Repair the native cleanup receipt check exposed during full verification: accept matching input digests, retain legacy input checks, and reject conflicting receipts. ## Verification - Live Product E2E on `f43478473800e3a46b85c5ee79677efdb15108e7`: **3/3 passed** with native Codex `gpt-5.4-mini`, first attempts only, automatic retries disabled. Standard and Ask each saved “Rotate expired API key” on their first tool call, with matching persisted state and a single same-run audit entry. The explicit-title control retained its user title with zero title writes. All three verified the reloaded browser UI. - Campaign: `local-2026-09-30T21-30-11-021Z`. Earlier failed campaigns are retained separately; they exposed credential-prose handling and prompted the naming recovery fix. No failed result was regraded or deleted. - Reproduce with `pnpm test:e2e:runner -- --id task-titles.runner-codex-mini.local.prompt-title-standard --id task-titles.runner-codex-mini.local.prompt-title-ask --id task-titles.runner-codex-mini.local.preserve-explicit-title --max-automatic-retries 0` and an authorized provider key. - Full `pnpm -r typecheck` and `pnpm build` passed on the latest commit. The runner build used the configured external eval source tree. - Product E2E unit suite: **61 files, 818 tests passed**; E2E typecheck and UI token gates passed. - Title API/native regressions cover prompt-only and explicit child creation, user edits, ownership/company isolation, external reference refresh, cross-surface retry replay, and the 64-key limit without receipt eviction. All passed. Prompt-context coverage: **44 tests passed**. - Rust credential regressions: **35 tests passed**, including benign maintenance qualifiers and opaque credential rejection in every declared prose field. Catalog/report reconciliation: **28 tests passed**. Native recovery: **560 tests passed**. - Broad local `pnpm test:run`: **14,555 tests passed** in the general server group; two suites failed to initialize embedded PostgreSQL and the existing 40,000-file Git streaming stress test exceeded its 300-second macOS timeout. All three suites then passed in isolation (**5 tests passed**) without code or timeout changes. The original full local command exited nonzero and is not being represented as a clean full run. - Latest-head GitHub checks are green: **53 passed, 4 skipped, zero failed or pending**, including all test shards and the canary packaging dry run. Greptile reviewed the same commit at **5/5**, with zero unresolved review threads. ## Risks - The additive database field must reach the server and UI together. The migration uses `IF NOT EXISTS` and defaults existing tasks to a final title. - Title generation depends on the assigned agent running. Tasks without a run keep their provisional title. - Live qualification covers the native Codex path in Standard and Ask modes. API/legacy and Plan behavior have deterministic coverage. - The credential-prose exception validates the entire suffix against a closed maintenance vocabulary. Unknown suffixes, assignments, quoted values, credential prefixes, and diagnostics retain strict checks. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact deployment ID and context window are not exposed in this session. The live eval uses the native Codex `gpt-5.4-mini` profile. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5edf55d735 |
fix: retire failed warm sessions before sandbox stop (#14747)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runner sessions can stay warm between turns in a reusable sandbox. > - Heartbeat stops that sandbox when a turn fails or is cancelled. > - The native executor treated every returned terminal result as a successful warm release. > - A failed session could therefore retain a transport for a stopped sandbox. > - This pull request retires unsuccessful sessions before heartbeat stops their sandbox. > - A retry can start without inheriting that stale transport. Successful turns stay warm. ## Linked Issues or Issue Description **What happened?** A structured failed or cancelled terminal result did not throw. The host kept its native session in the warm cache even though heartbeat stopped the reusable sandbox. Later session retirement could use the stopped provider transport and reject the retry. **Expected behavior** Retire the unsuccessful session and collect its managed files before returning to heartbeat. Keep successful sessions warm. **Steps to reproduce** 1. Run a native session with a warm lifecycle and a reusable sandbox. 2. Return a structured failed or cancelled result from the provider. 3. Stop the sandbox after the executor returns. 4. Retry with a changed native session identity. 5. Verify that the previous session was closed before step 3 and is not closed again during retry. **Paperclip version or commit** Developed from `b54b2dc35`, the current `origin/master` at implementation time. **Deployment mode** Native runner with a reusable Daytona sandbox. The same warm-session release path also serves local providers. Related work: #14735 added incremental managed-file checkpoints for warm turns. #14734 preserves tool outcomes during shutdown. This change fixes the host's handling of unsuccessful terminal results. ## What Changed - Retain a warm session only when its terminal run state is `succeeded`. - Use the existing failed-session retirement path for structured failures and cancellations. - Preserve the existing checkpoint-before-retirement path, including when provider shutdown fails. Collect stopped files after successful shutdown, including edits made during shutdown. - Retire the warm owner if the checkpoint or its receipt callback rejects, then propagate the initiating error. - Add ten regression cases for failure and cancellation, with and without managed files. They check cleanup order, cache removal, retry with a new session identity, and edits preserved when close rejects. - Document the lifecycle rule. ## Verification - The failed and cancelled managed-file regressions fail on unpatched master because the provider is not closed. - All 516 native executor tests pass, including ten new regressions. - `pnpm -r typecheck` and `pnpm build` pass. `pnpm test:run` finished its general-server group with 14,537 passed, 75 skipped, and one existing failure in `native-session-resume.test.ts` (`retainedNativeCleanupJournalMatches`, line 1125). A separate run using the unmodified `origin/master` executor fails the same assertion. The command stops before later local groups; all equivalent CI groups pass on this head. - All 56 PR checks are successful or intentionally skipped. Greptile reviewed this head at 5/5 with no remaining findings. - A real Daytona public-API probe forced a Codex authentication failure, waited for a confirmed sandbox stop, corrected the credential, and retried successfully in the same resumed sandbox. It verified the agent note in canonical storage and deleted the test sandbox. - Pumpkin staging on this exact commit: forced a structured authentication failure, confirmed sandbox stop, corrected the credential, and retried successfully in that same sandbox. Three successful turns kept the same live runner PID/start ticks, native session, runner instance, and provider session. Canonical downloads verified the note, all 8 MiB binary bytes, and deletion persistence. Subsequent checkpoints hashed/copied only 52 and 78 bytes. After sandbox deletion, canonical files still matched. Temporary secrets, agent configuration, and test policy were cleaned up or restored. - Unpatched live probes with both 5-second and 90-second retry delays also succeeded. The unit regressions prove incorrect retention; the original stopped-lease exception was not reproduced in those probes. The staging recovery campaign used the public retry API after an explicit reset of the disposable task’s already-deleted old sandbox/session. The UI did not expose a Retry control on the inspected task or run detail views. - Temporary API-key authentication was used for the campaign. This change does not repair the original managed-subscription authentication 401. ## Risks - Failed and cancelled turns now close their provider sooner. Successful warm-turn behavior is unchanged. - Cleanup uses the existing failed-session path. Its existing handling of cleanup errors is unchanged. - No database migration, API contract change, or dependency change is included. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and test analysis. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (516 relevant executor tests; the full local suite has the independently reproduced baseline failure documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b54b2dc35c |
fix: preserve warm Codex turns with incremental managed file checkpoints (#14735)
## Thinking Path > - Paperclip manages AI agents and keeps their instructions and files durable. > - Native Codex runners can keep a process alive between compatible turns. > - Managed file collection stopped that process after each turn, which defeated warm reuse. > - Agent folders can contain large images and other files, so full copies on every turn are expensive. > - This change keeps one managed directory for the live session and saves only file changes after each turn. > - Ownership, authorization, instruction changes, and process retirement still control when reuse is safe. ## Linked Issues or Issue Description Related: #13710 introduced native warm session reuse. This fixes managed file collection that still forced those sessions to stop. No duplicate open PR or issue was found. **What happened?** With managed instructions and warm native Codex enabled, consecutive turns reused a Daytona sandbox but started a new runner process each time. The managed directory collector required process termination before saving files. **Expected behavior** Compatible turns keep the same process and managed `AGENT_HOME`. Each completed turn saves added, changed, and deleted files before the next turn starts. Unchanged large files do not transfer again. **Steps to reproduce** 1. Use a native Codex agent with managed instructions and a reusable Daytona environment. 2. Enable warm session reuse and run three turns on the same task. 3. Write a large binary on the first turn, edit a small note on each turn, and delete a file on the second turn. 4. Compare process identity across turns and read the canonical files through the public agent-files API. **Paperclip version or commit** Reproduced on `d30b03bd8c17604cdab1533eeeeb087aba30e8b1`. **Deployment mode** Local server with remote Daytona execution; cloud native runner uses the same path. ## What Changed - Retain the managed directory only for the verified owner of a live native Codex session. - Checkpoint each completed turn before releasing the session for reuse. Retry unstable captures, then stop and collect when a warm checkpoint cannot be validated. - Compare metadata and cached hashes, stream only changed file payloads, record deletions, and validate path, content, quota, and authorization before saving. - Rotate sessions when canonical files, loaded instructions, credentials, or launch policy change. Fence stale collection and cleanup callbacks from later owners. - Keep cleanup and recovery aware of the current session owner. Recheck canonical files under the writer lock at handoff, attach the successor collector before fallible bookkeeping, and emit one final save receipt on checkpoint fallback. Preserve storage warnings across unchanged checkpoints. - Add regression coverage and a three-turn Daytona test with independent public API file checks, an unchanged 8 MiB binary, deletion checks, and strict process identity checks. - Document checkpoint consistency, lifecycle behavior, and local run-log counters. - Replace a timing assumption in the Daytona teardown test with explicit transfer-arrival gates after CI exposed an unset release callback. ## Verification - Full local `pnpm -r typecheck` and `pnpm build` passed. Server checks were repeated after the final storage-warning fix. - Runner E2E typecheck and 749 runner E2E unit tests passed. - Focused file checkpoint, directory ownership, instruction collection, native session, and merge tests passed. After review fixes, the managed-directory and native-session suites passed 550 tests, including intervening canonical edits, same-run fresh restore, failed handoff collection, and one-call fallback collection. Server typecheck passed again. The Daytona plugin suite passed 218 tests. The quota-warning regression failed before the fix and passed afterward. - Three real Daytona campaigns passed before the final handoff review fixes. The latest kept PID 547 across all three turns. The first checkpoint copied 8,388,635 bytes; the next two copied 36 and 54 bytes. Public API reads verified the binary, note contents, and deletion after every turn. Test cleanup deleted the sandbox. - The final head was also deployed to an isolated cloud staging instance and passed three UI-triggered native Codex turns with managed instructions. All three retained the same process ID/start time, native session, provider session, runner instance, and Daytona sandbox. Checkpoints copied 8,388,643 bytes on turn 1, then only 52 and 78 bytes on turns 2 and 3; those warm captures also hashed only 52 and 78 bytes. Independent canonical API reads verified every byte of the unchanged 8 MiB binary and the exact note contents after every turn; the deleted file returned 404 after turns 2 and 3. After restoring the original lifecycle and agent-auth configuration, removing the temporary secret, pausing the test agent, and deleting both test sandboxes, independent canonical API reads still verified the entire binary, the final 78-byte three-line note, and the deletion. The native runner flag remained enabled and the final serving revision remained the PR head. - Two earlier staging attempts are preserved as failures and are excluded from the acceptance result: a saved ChatGPT login failed with a provider routing 401, and its subsequent stopped-sandbox retry failed before provider startup with a closed-lease admission error. The successful campaign used a fresh sandbox and a temporary encrypted API-key binding. The stopped-lease retry remains unexplained; this campaign does not establish recovery of that failed sandbox. - All [Paperclip CI gates](https://github.com/paperclipai/paperclip/actions/runs/36750397355) pass on `26ef2ef56a389259246809805c0b34a4747eb86b`, including full test partitions, build, typecheck, runner verification, E2E shards, and the Canary clean public-npm install. Greptile reviewed that exact head at 5/5 with no unresolved review threads or outstanding findings. - Full local repository coverage used the existing CI partitions, but the 40,000-file Git streaming stress test timed out and its local retry was interrupted by macOS thermal emergency sleep; this is not a green full local suite claim. The exact stress test passed on the final head in [CI server shard 2/12](https://github.com/paperclipai/paperclip/actions/runs/36750397355/job/110008294290), in 111.9 seconds. - Repeat the live test with configured credentials and a Linux runner artifact: `pnpm test:e2e:runner -- --id daytona-warm-continuity.runner-codex.daytona.warm-three-turn`. ## Risks - This is a file-level checkpoint, not an atomic snapshot of the whole folder. Background writes after a capture are saved by the next checkpoint or final stopped collection. - Metadata scans still visit all paths. Modified files transfer in full; unchanged files do not rehash or transfer. - Incorrect ownership or reuse could collect the wrong directory. Run ownership fences, current authorization, stable capture validation, and stopped collection fallbacks are covered by tests. - Warm reuse remains opt-in. No database migration or fleet default changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, tool use, and test execution. The exact serving model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dd7fc1f90a |
fix: raise the native journal read limit to 256 MiB (#14711)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native sessions persist control-plane state so they can resume safely. > - The state includes committed provider history needed for recovery. > - The server, runnerd recovery, and durable control plane validate this state before trusting its identity. > - Their differing 64 MiB and 192 MiB limits can reject a valid journal before recovery. > - This pull request aligns all three local state limits at 256 MiB. > - Larger files remain bounded, while recovery can read larger valid histories. ## Linked Issues or Issue Description Refs #13882 Refs #14312 ## What Changed - Raise the server and runnerd recovery limits from 64 MiB to 256 MiB, and align the durable control-plane limit from 192 MiB to 256 MiB. - Add coverage for a valid history above 64 MiB and rejection above 256 MiB. ## Verification - Matching server recovery passed with more than 64 MiB of actual committed event payloads (128 events with 512 KiB deltas). - The real runnerd exact-authority resume regression with the test Codex provider passed with 193 MiB of valid JSON whitespace appended. It crosses the former 192 MiB core limit and confirms the same provider identity. This exercises the runner process and durable control plane with a simulated provider, not a live OpenAI API call. This test used approximately 1.15 GiB peak RSS. - The actual runnerd reader accepted valid 256 MiB JSON and rejected valid 256 MiB + 1 byte. The reader call took 231 ms; the fresh process peaked at 1,244 MiB RSS. - Server tests reject mismatched identity above 64 MiB and files above 256 MiB. - `pnpm -r typecheck`, `pnpm build`, and `git diff --check` passed. - Full local `pnpm test:run`: 13,730 passed, 575 skipped, 7 failed across 6 files. All failures were embedded PostgreSQL startup errors after five attempts. They affected agent hiring, instruction revisions, environment images, reviewed chat bindings, issue monitoring, and legacy continuation authority. The focused journal tests passed; the latest pushed head passed all ordinary CI checks. Superagent is the only blocking check. ## Risks - **Open review concern:** Greptile is 5/5, but Superagent is `ACTION_REQUIRED` with two P2 findings on the server and runnerd readers. Both flag the increased synchronous parsing and memory cost. This PR keeps the requested fixed-limit change small. It does not add a worker parser or a process-wide memory budget. This resource tradeoff needs review before merge. - Large state parsing is synchronous and can consume several times the file size in memory. - Remote checkpoint archive and expanded-size limits remain 64 MiB, so this change alone does not make larger remote checkpoint transfers portable. ## Model Used - OpenAI Codex, GPT-6, with delegated assistance from `gpt-6-luna` at high reasoning effort; tool use and code execution. The GPT-6 context window is not exposed in this task runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
24beb00575 |
feat(runner): add rich ACP transport and durable interaction foundation (#14430)
Add shared rich ACP transport, durable questions and permissions, verified provider packaging, and bounded activity and plan presentation. Keep Cursor, Copilot, and Pi pending their separate provider qualification. Persist interaction settlement before publication, fence failed writes until fresh recovery, and preserve owned-process cleanup. Incorporate reviewed mainline integration with extended harness coverage. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandFry
|
3ca196b0a6 |
feat(agents): persist agent files across tasks without revision history (#14420)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent needs personal files across tasks and sessions. > - AGENTS.md is one file in that directory. Supporting files need the same persistence. > - The Instructions Editor and agent runs must share one current directory. > - Concurrent runs should apply only the files they change. The last sync of the same file wins. > - This pull request uses existing file transport and removes temporary copies after sync. > - Old instruction-only sessions keep their restore contract. New saves do not create revision history. ## Linked Issues or Issue Description Refs #14325. This replaces its revision-oriented design with persistent agent files. Keep #14325 unmerged. Transport prerequisite #14416 merged first at `d172197117a14b80a1eb2d2835a0e7cce2679656`. This PR now targets master and remains below 100 changed files. Related work: #4513 and #8798 cover instruction tooling. This change handles run synchronization, cross-task personal files, browser editing, and old-session restoration. ## What Changed - Keep one current directory per company and agent. Point AGENT_HOME at a temporary working copy for each active run. Keep task files and provider HOME separate. - Restore text, binary files, and nested folders through workspace transport. Exclude remote agent files from task Git snapshots with a self-ignoring file inside the reserved runtime directory; never write through repository-controlled Git metadata. - Collect after the provider and child processes have stopped. Keep resumable conversation state. - Apply changed and deleted files under the agent lock. The last sync wins for the same file. Unrelated concurrent changes survive. - Remove temporary copies after successful sync, rejected sync, and staging failure. Register ownership before copying so restart recovery can remove interrupted preparation. Retry transient synchronization up to three times. Preserve the original remote lease reference until deletion succeeds; restart cleanup never acquires a replacement sandbox. Do not create captured directories or a conflict-review queue for new runs. - Keep browser editing, stale-draft protection, and streaming binary downloads. Keep the instruction entry and text editor limited to 1 MiB. - Keep historical agent-folder sync failures on their affected runs instead of repeating them above current saved instructions. Preserve legacy candidate review and current browser-save errors. Avoid duplicate quota warnings while retaining separate sync failures when they describe a different problem. - Require target-scoped caller grants for peer instruction access, while preserving self edits, responsible-user checks, and protected-change consent. - Treat full storage as a nonblocking run warning, never an agent pause or run-admission failure. Restore already-over-quota saved folders so ordinary agent cleanup can recover; warn on each run until cleanup. The run detail view shows the warning. - Allow 256 MiB per file, 2 GiB per directory, and 100,000 entries. Hash large files as streams. Check editor-save quotas with metadata instead of hashing unrelated files. - Preserve old native inputs, instruction-only copies, paths, digests, and pending legacy candidates. Adopt old revision heads once. New writes do not append history rows. - Add idempotent migration 0287 and verify upgrades from the preview tables and receipts. - Add nine interactive stories under **Agents / Persistent files**, including automatic incoming edits, stale browser drafts, and storage-limit diagnostics. ## Verification - Merge candidate: `4f5390107ec6ffd80a76d1d2e85530e66f21d079`, after merging current master and the landed transport prerequisite. Integration required no manual conflict resolution; the feature remains 99 changed files. Full workspace typecheck, production build, token gates, and 715 focused tests passed on this merge candidate. Fresh Greptile review is 5/5 with no unresolved findings. All 55 checks passed, with four conditional skips, including the build, typecheck, browser E2E, and canary dry run. A single retry recovered four jobs interrupted by runner shutdowns; no source changes were required. - Historical-warning UI fix: all 6,834 UI tests across 640 files passed, including regression coverage for three old failures, legacy preserved edits, and warnings scoped to the affected run. Full workspace typecheck, production build, Storybook build, and token gates passed. Browser-verified Storybook playtests passed for Historical Failures After Successful Save, Storage Limit, and Full Storage Run Warning. - Review follow-ups at `4e20c9fb2`: all 18 focused tests passed, including external Git directories, linked worktrees, symlinks, hardlinks, and distinct I/O failures alongside storage warnings. Server and UI typechecks, token gates, and the production build passed. - Storage warning regressions at `0724f3012`: all 33 directory tests and all five heartbeat-list tests passed, with no skips in their successful runs. They cover repeated runs while full, an already-over-quota saved folder, cleanup, warnings retained after unrelated save failures, and bounded warnings in large result JSON. Server typecheck passed after the final warning fixes. - Full workspace typecheck, production build, and token gates passed during this follow-up. Product E2E harness: 631 tests passed across 52 files; harness typecheck passed. Earlier native session/context and directory/legacy collection suites passed 537 tests; Runner unit/transport suites passed 329 tests. - **Real E2E at `0724f3012` (before this follow-up):** legacy local Codex and native Daytona Codex each passed six tasks, one server restart, seven independent assertions, and cleanup verification. Both prove browser-to-agent edits, agent-to-browser edits, nested/binary restoration, per-file last-sync-wins, a successful run after an oversized save rejection, and cleanup clearing the warning. - Native local Codex also passed the six-task quota flow before the final warning-retention fixes. That pass began at `918d1ed02` while the bounded-result warning fix was being edited, so it is not claimed as exact-final-head evidence. Its final-head rerun failed during embedded PostgreSQL bootstrap before any provider run: the macOS host had 87,365 of 87,381 SysV semaphores occupied. No unrelated services or kernel limits were changed. - The final-source report intentionally records **2/3 cells passed**, preserving the blocked native-local attempt: `tests/runner-e2e/results/agent-files-quota-final-20260928-report/`. Earlier failed attempts and provenance notes remain under `tests/runner-e2e/results/agent-files-quota-final-20260928-input/` and the original campaign directories. - Daytona used immutable image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:5643f0d801417cae3581833a1a3bc6715b325e028602738d2652c44cac5dc6bf` and its exact Linux runner binary. Controller source is `0724f3012`; image source is recorded separately. - Legacy-session compatibility and all three ACP Stop/resume browser regressions passed on the prior validated feature head `169fab46d5af21caa2269b4c1b29b69c933a6951`. They assert the same provider session is retained and interrupted writes are not replayed. Migration upgrade tests also passed earlier. - Nine interactive stories are under **Agents / Persistent files**, including **Full Storage Run Warning**. Its playtest and visual browser inspection passed; the warning states that runs continue and the editor remains available. - Prior-head checks on `4e20c9fb2`: 55 passed, two conditional jobs skipped, no failures or pending checks. All eight browser E2E shards and their aggregate passed. Fresh Greptile review is 5/5 with no findings; all review threads are resolved, the security scan passed, and GitHub reports no merge conflicts. - The broad local follow-up test run was interrupted after host semaphore exhaustion affected isolated PostgreSQL instances. It also encountered the existing macOS long-path fixture failure and two timeout failures. This is not a claim that the full local suite passed. Logs are retained; focused storage/warning tests passed. ## Risks - A later sync can overwrite an earlier edit to the same file, including a saved browser edit. There is no text merge or retained version. This is the intended last-sync-wins policy. - A save that exceeds a storage limit is rejected and its temporary copy is discarded. The run itself continues normally, and later runs restore the last saved files with a warning until cleanup. Transient sync failures get bounded retries. An I/O failure partway through a sync can leave some files updated; a failed receipt does not claim whole-folder success. - Larger folders increase copy time, network traffic, and temporary disk usage. Active runs still need working copies. Terminal runs do not accumulate archives. Operators must provision disk for agents and configured concurrency; these limits are not company-wide quotas. - A restored old native session remains instruction-only until a fresh session starts. Its original conflict fence and existing pending candidates remain compatible. - Provider processes close at the collection boundary. Conversation resume remains available, but warm process reuse is lost. - Backups must include the instance filesystem and database. External bundles keep their existing behavior until explicitly moved to managed storage. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific model ID or context-window size. Reasoning, code execution, and browser tools assisted this change. Real provider E2E uses `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Fry (Paperclip) <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
ad1f7e98ea |
fix: retain diagnostic reasons for native runner failures (#14481)
Retain bounded reasons for runner identity, harness recovery, and provider-pack read failures. Preserve existing ownership and cleanup proofs and compatibility with receipt-gated chat recovery. Verified with executor, recovery, diagnostic privacy, typecheck, build, and full CI checks. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
18e8c121d9 |
fix(runner): include Grok support in public installs with sandbox prerequisites (#14024)
## Thinking Path > - Paperclip manages agents through a shared native runner. > - Built-in harness support should ship with Paperclip's public distribution. > - Grok already speaks ACP; it does not require a new public bridge package. > - Sandbox provisioning owns the native executable and its pinned version. > - The runner must verify that prerequisite without downloading it during npm installation. > - This change separates built-in launcher identity from external runtime identity. > - Clean npm installation and live staging checks verify the distribution boundary. ## Linked Issues or Issue Description Refs #13882, #13973, #13977, #13979. This follow-up now targets master after #13882 was squash-merged. It replaces the private `@paperclipai/grok-acp` workspace package with runner-owned assets. Current master is included so the branch also contains the merged scheduler, complete-event capture, and durable cleanup fixes. ## What Changed - Ship Grok launcher and qualification metadata inside the runner's compiled output and the public server's vendored runner tree. - Remove the separate Grok npm package and all package-manager install hooks for this runtime. - Require the checksum-verified Grok Build 1.0.13 binary at `/opt/paperclip/providers/grok/1.0.13/grok` in the selected execution environment. Provision it explicitly in the Daytona image and CI setup. - Keep native binaries outside the provider pack. Bind the built-in launcher into the pack manifest. - Preserve executable leases, descriptor-backed startup, credential fences, permissions, and exact ACP model admission. - Use `builtin:grok-acp` and `native:grok` as profile identities. Historical package-profile sessions fail closed on resume rather than being silently reinterpreted. - Resolve built-in assets from the authenticated sidecar location, including public server npm layouts. Keep the controller path out of provider environments. - Add clean npm tarball installation verification to the existing trusted canary CI job and the admitted manual EC2 verification path. It stages a unified release version and runs npm lifecycle scripts, then verifies missing-prerequisite rejection and admission after separate provisioning without credentials or inference. - Include the controller-owned provider pack in stamped Cloud images. Unstamped local images omit the pack and remain usable; remote ACPX requires full source provenance. - Correct CLI approval-page metadata for an already authenticated Cloud board user; approval authorization remains unchanged. - Honor explicit native-runner enablement in the Cloud agent picker and direct setup page, keeping the flag disabled by default. - Allow selecting the execution environment before connecting credentials. Include Grok in the existing authenticated hello-probe flow, targeting its pinned native prerequisite for runner setup. - Recover an existing subscription sign-in conflict through an explicit cancel-and-retry action, serialized after cancellation succeeds. - Preserve the selected ACPX harness before normalizing config fields, so new Grok agents use the Grok default model. - Keep the credential-free Cloud provider pack root-owned and readable after runtime UID remapping; verify manifest and referenced asset access under an unrelated unprivileged UID during image builds. - Archive prior failover backups alongside explicitly replaced harness state, preserving evidence while preventing stale backups from blocking a fresh replacement. - Update Daytona image content inputs and contract tests for the built-in assets and explicit provisioner. - Document and regression-test the shared `approve-all` default for Grok setup, saved configuration, and native execution. Explicitly saved restrictions remain unchanged. ## Verification Current merge-repair head `df09eb3e1a619430ad8419a0ee9aedd486689b05` incorporates master `f1a394bd30cb56fb9e479f98b9f50176fe921858` after the base PR was squash-merged. All 12 conflicts came from incoming files identical to the tested pre-squash base. The final tree exactly matches a three-way merge using that original base, preserving built-in Grok distribution and removal of the obsolete private package. All 252 focused runner/UI tests, six npm-isolation tests, and token gates pass. Fresh exact-head Greptile review is 5/5 with no outstanding findings; security scans and EC2 native compilation pass. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([run 36468768035](https://github.com/paperclipai/paperclip/actions/runs/36468768035)). The repository owner explicitly authorized bypassing code-owner approval after all checks passed; no CI checks or repository protection settings are bypassed or changed. The only remaining PR was removed from the completed stack metadata to permit native auto-merge. Earlier integration head `78cb306ecc41b5c96577c26c1d89153b0ef865a1` includes master `3447609d2247e75e55d91493dda91a608364f672` (2026-09-28). Two master advances during verification overlapped the eval catalog; the final merge preserves Grok qualification, completion updates, and bounded API-response reading in all 348 cells. All 77 focused catalog/eval/workflow tests pass. Both native stack layers (#14397) are mergeable, and both exact-head Greptile reviews are 5/5 with successful security scans and no unresolved review threads. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([CI attempts](https://github.com/paperclipai/paperclip/actions/runs/36447124691)). The initial attempt lost two EC2 runners to shutdown signals and stalled a third shard during dependency preparation; all three passed the same-commit failed-job-only retry. Trunk code-owner requirements remain enforced. The review summary’s non-blocking saved-asset offset classification note concerns code already merged in #14301; those runtime files are identical to master and outside this stack’s diff. Historical live evidence below retains its original source revisions. [Final public npm verification](https://github.com/paperclipai/paperclip/actions/runs/36445542764) passed on `76ea70cd4d13786a042af9df82f0fd7a8c85ae30`: 17 public packages, an executed offline lifecycle sentinel, unchanged consumer lock, built-in launcher, missing-prerequisite rejection, and verified separately provisioned binary/command lease. Provisioning and cleanup require no host privilege elevation; only the positive probe mounts the temporary native binary read-only. The verifier is unchanged by the final master merge. All six isolation tests and an offline npm smoke test pass. The prior head had 56 green CI checks and a 5/5 review after two unchanged tests timed out and passed a failed-job-only retry ([CI attempts](https://github.com/paperclipai/paperclip/actions/runs/36444597313)). All 56 recovery-display/lineage tests pass; re-review cleared the already-covered missed-retry concern. Earlier EC2 failures remain retained: [npm lockfile rejection](https://github.com/paperclipai/paperclip/actions/runs/36436311203), [missing compiler in the slim image](https://github.com/paperclipai/paperclip/actions/runs/36440210984), and the aggregate 15-minute test timeouts in those broad runs. Both broad attempts passed typecheck, token gates, Product E2E type/unit checks and build. The focused EC2 lane preserves the existing trusted-actor and immutable-source gates. Earlier documentation/test checkpoint `ff244c4fd78a7ede5a3e00efe09f475f133ef33e` leaves runtime behavior unchanged. 154 focused tests pass across configuration building, native provider resolution, permission policy, credentials, UI configuration, and new-agent setup (including both Grok auth modes); token gates pass. All fresh CI is green for this head: 56 successful checks/statuses and two intentional skips ([run 36367065119](https://github.com/paperclipai/paperclip/actions/runs/36367065119)). Greptile is 5/5 with no new findings. Grok already inherits the shared `approve-all` default, so unattended setup requires no manual permission change. Runtime head `bb5a9307991f1ac567b781970ef11b39d518e19b` fixes a final staging continuation failure before provider startup: explicit replacement archived the old harness but left its failover backups active, which caused `runner_harness_state_mismatch`. The regression fails before the fix and passes after it; all eight adjacent recovery-safety cases also pass. Old backups remain inspectable inside the continuity archive. All fresh CI is green at this head ([run 36360839248](https://github.com/paperclipai/paperclip/actions/runs/36360839248)), with a 5/5 review. One unrelated Cursor test timed out in the initial server shard; the same-commit failed-job rerun passed, and both attempts are retained. Staging deployment is confirmed healthy on this revision. The controller image is `ghcr.io/paperclipai/paperclip@sha256:6ad91c487910ccd2596ff7aed0a3a3ea5233d12b51b83cd6e1402237749b9673`. The final browser-created staging task passed on this exact revision with API authentication: context read → structured human question → controller restart → answer submission → same native provider session resumed → document saved → task Done. The two turns took approximately 119s and 77s. The actual write receipt was applied, and the saved document has exactly one revision containing the selected answer and requested marker. Usage and cost were not reported. [Controller image build](https://github.com/paperclipai/paperclip/actions/runs/36360889243). - Previous integration head `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`: all CI green (53 successful checks/statuses, two intentional skips), including repository typecheck/build/tests, native Runner tests, browser shards, and canary installation checks. [CI run 36358672529](https://github.com/paperclipai/paperclip/actions/runs/36358672529). Greptile is 5/5 with no unresolved findings. - Focused checks cover Grok credentials, executable admission, launcher assets, provider-pack paths/permissions, workflow contracts, setup defaults, CLI authorization, and subscription conflict recovery. All 39 protocol definitions validate. Final integration checks pass 124 catalog/evidence/cache tests and nine project-form tests; token gates pass. Some local dependency checks could not load the stale installed dependency tree; the corresponding fresh EC2 checks pass. - Clean public npm installation passed on EC2 at `8b172ebcf8e02e30662d830c00f3961e3bd459ec` ([run 36164964900](https://github.com/paperclipai/paperclip/actions/runs/36164964900)): 17 unified-version packages, lifecycle scripts enabled, built-in launcher present, no separate Grok package or npm-downloaded binary, missing prerequisite rejected, separately provisioned native executable and command lease verified. No credentials or inference were used. Subsequent changes preserve this npm asset layout. - The immutable Daytona prerequisite image is `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:98957d5be0ac774d086b6402b5849e8e6356fec70fb8c09fca6eb4ed6de918e0`, built from `5a2db471f3ddabe77f9f80e76ed27f996cb97fba`. The previous Cloud controller image was `ghcr.io/paperclipai/paperclip@sha256:fd914e1ab1e45f741e8e078ff452d16f082d7ac05f9b4b3506d3a3c64150d204`, built from `a44f7dbb6b6f77cd9ed893756ca453307f281e5f`; it is superseded by the latest image above. Its EC2 build verified provider-pack access under an unrelated unprivileged UID. - Browser staging at `40f898bc4cba73c1dff4e6344a3983ba0fb247ef` passed full Grok onboarding with the correct `grok-4.7` model, saved credential delivery, and pinned Daytona execution. A browser-created task read context and asked the structured human question. After a controller restart, answering the persisted question resumed the same native provider session, saved the requested document, and completed the task. Actual tool outcomes and durable state agree: one question and one document revision. The two successful turns took 42.7s and 63.1s; usage and cost were not reported. - Restricted policy returned the expected `approval_required` outcome. Functional staging tests explicitly selected `approve-all`; controller authorization and governed approvals remain enforced. Temporary board CLI access was revoked and verified rejected (HTTP 401), and the disposable onboarding agent was paused. Failures remain retained: the pre-fix continuation failure (its task remains blocked; the passing final task is fresh), the original Cloud provider-pack permission failure, the expected restricted-policy denial, the superseded npm staging failure, and an earlier monolithic CI infrastructure timeout. Browser CI exposed a project alias/form race; the final stack uses master's stronger draft-preservation fix and all browser shards pass. Historical full subscription/API protocol and Product rosters retain their original source revisions and do not qualify this packaging revision. No local Docker or Rust build was used. ## Risks The branch includes master’s draft-preservation fix for project URL aliases. It keeps the same project’s edit form mounted and clears prior data when the project or company changes. Custom sandboxes and local execution hosts must provision the pinned binary before Grok starts. Missing, changed, unsupported-platform, and symlinked executables fail admission. The new builtin profile cannot resume sessions created with the former private-package profile. Existing Claude/Codex npm bridge profiles retain their package pins. Grok restricted modes preserve the selected policy but cannot automatically admit Paperclip calls: ACP permission metadata does not independently bind tool authority, so those calls stop with `approval_required`. New Grok configurations default to `approve-all`, including API configurations that omit the mode. Existing explicitly restricted configurations remain restricted; controller authorization and governed approvals remain enforced. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f1a394bd30 |
feat(runner): add Grok Build through native ACP (#13882)
## Thinking Path > - Paperclip manages AI agents and governs their work. > - Its native runner uses structured provider protocols for sessions and tools. > - Grok Build supports ACP over stdio, but the runner did not expose it. > - Native execution requires company-scoped credentials, verified identities, and permission gates. > - This change adds Grok through ACPX for local and Daytona execution. > - Subscription login and explicit API-key execution have separate credential paths. > - Qualification grades real tool outcomes, durable state, and browser workflows. ## Linked Issues or Issue Description Refs #13845, #13847, #13850, #13878, #13901, #13973, #13977, #13979. Add **Grok Build** to `paperclip_runner` with `provider: "acpx"`, `acpxAgent: "grok"`, and model `grok-4.7`. Existing legacy Grok agents keep their adapter. Merge the three companion fixes (#13973, #13977, #13979) before treating the integrated Product qualification as deployed behavior. ## What Changed - Synchronize shared, TypeScript, Rust, server, validation, and UI provider contracts. - Run Grok native ACP stdio through ACPX and the authenticated Paperclip MCP bridge. Verify the pinned executable and exact ACP model identity. - Prefer company subscription login. Support an explicit company-secret API key without automatic paid fallback. Fence refresh and copyback to the same account and remove private runtime credentials after containment. - Preserve selected permissions, cancellation, durable session identity, resume, and restart recovery. Keep unsupported steering and goals unavailable. Preserve missing usage and cost as unknown. - Package checksum-verified Grok Build 1.0.13 for Daytona with an immutable, signed image built on EC2. - Add deterministic admission, protocol, permissions, identity, credential, failure, and cleanup checks. Add the maintained 39-case protocol roster and separate subscription/API Product profiles. - Fix live-test findings in reasoning events, reloads, idle-owner retirement, credential-home cleanup, expired-login model discovery, launcher pinning, and rerun evidence selection. - Align control-plane state readers with the transport's 64 MiB bound while retaining identity, ownership, lifecycle, and size rejection checks. - Stabilize two asynchronous CI assertions while retaining actual outcome and filesystem-evidence checks. ## Verification Current integration head `f114948376056fe0b6b34c1496ae8667b59daa63` includes master `3447609d2247e75e55d91493dda91a608364f672` (2026-09-28). Two master advances during verification overlapped the eval catalog; the final merge preserves Grok qualification, completion updates, and bounded API-response reading in all 348 cells. All 77 focused catalog/eval/workflow tests pass. Both native stack layers (#14397) are mergeable, and both exact-head Greptile reviews are 5/5 with successful security scans and no unresolved review threads. All current-head CI is green: 56 successful checks/statuses and four intentional skips ([CI run](https://github.com/paperclipai/paperclip/actions/runs/36447097232)). Trunk code-owner requirements remain enforced. The review summary’s non-blocking saved-asset offset classification note concerns code already merged in #14301; those runtime files are identical to master and outside this stack’s diff. Historical live evidence below retains its original source revisions. Earlier integration checkpoint: `24fc9b94ca0afb21ccdc8d26dbb2e4b258ad72cb`. Refreshed against master `0f14d2612`, preserving Grok qualification alongside the new accounting and lifecycle suites. All 124 focused catalog, evidence, and service-worker checks pass. The current base workflow includes the explicitly selected public-install verification lane; follow-up #14024 supplies its verifier script. CI at that earlier checkpoint was green (56 successful checks/statuses, four intentional skips), and the review is 5/5 with no unresolved findings. Prior feature CI at `fd73f0a9b1ecdf4094685054028df71739ddc3e1` passed ([run 36148259902](https://github.com/paperclipai/paperclip/actions/runs/36148259902)); that is historical evidence, not a current-head result. Paid Product measurements use frozen integrated source `2d939a92b21dcaf5c77c88b54d96784d2ddd0699`, which combines the feature with #13973, #13977, and #13979. That source passed all 52 CI checks and clean 5/5 review. Later master syncs incorporate upstream changes. Their checks remain separate from these pinned live measurements. | Check | Result and source-pinned report | | --- | --- | | Subscription protocol roster | [39/39 first attempts; 206 assertions](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-36046839612-1/index.html), runtime `bc6833f7`, evals `92bb4b8c` | | API protocol roster | [39/39 first attempts; 206 assertions](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-35926577007-1/index.html), runtime `4a1061c8`, evals `3213dbec` | | Subscription full Product matrix | [16/16 first attempts; 144 assertions; cleanup passed](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36096908572-1/index.html), source `2d939a92` | | Subscription core repetitions | 18/18: tool use, planning approval, and Stop/resume each passed three times in local and Daytona profiles. The full matrix contains repetition one; [repeat two](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36104551060-1/index.html) and [repeat three](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36108100404-1/index.html) each passed 6/6. Total: 28 unique subscription attempts at `2d939a92`. | | API smoke and question continuation | [4/4 first attempts; cleanup passed](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-36147315401-1/index.html), both environments at `2d939a92` | | Historical API Product coverage | [16/16 full matrix](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-35875144860-1/index.html) and 18/18 core repetitions at `4a1061c8`; retained as measurements of that revision | | Native Daytona proof | Three subscription and three API MCP/permissions/resume runs passed at `bc6833f7`. Three expired-login admission and fenced refresh checks passed without inference. All test sandboxes were removed. | | Inspectable artifacts and UI | Current-source screenshots verify planning approval, direct Ask completion, question continuation after controller restart, and two downloadable project revisions. The project downloads pass 12 and 18 tests; all 40 independent artifact oracle checks pass. | | Provider-free checks | 116 eval-validator tests, 39 Grok definitions, and 359 enabled/external campaign cells pass. Continuation regressions above 2 MiB and 16 MiB failed before their fixes; 32 focused recovery/ownership/size checks pass. | The 32 unique current-source Product attempts have no failures, retries, or skipped cells, and all cleanup checks pass. Whole-workflow timing, model identity, image and provider-pack provenance, attempts, and accounting coverage are retained in the canonical reports. The report publisher's conservative `complete=false` flag is preserved; independent audits verify the exact selected source catalog and immutable result rows. Pins: Grok Build `1.0.13 (5e9a58528b76)`, ACPX `0.13.1`, ACP model `grok-4.7`. Linux binary SHA-256: `edf79521581bb5e6b95abef848491a6a742e860da3e237ebe86a280d30dce4c1`. Launcher SHA-256: `f0b698395a3704ed2ffaf84ea19bdb20c36c8a0a70b7c629c7b6ffe144e59e55`. Image: `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:76b24edfd850219e949418b19e4ceba690e84d51d199ade426e484953329b5e9`. Image build source is `4196a4cd`, recorded separately from application source `2d939a92`; each campaign verifies the image signature and provider pack. Original failed campaigns remain available: [continuation bound](https://github.com/paperclipai/paperclip/actions/runs/36057718059), [scheduler/event capture](https://github.com/paperclipai/paperclip/actions/runs/36071063537), and [startup cleanup plus EC2 interruption](https://github.com/paperclipai/paperclip/actions/runs/36080870743). They retain their original grades. No Docker or Rust builds ran on the developer laptop for these follow-ups. ## Risks Merge packaging follow-up #14024 with this base before public release. The follow-up replaces the private Grok bridge package with a built-in launcher and makes the native binary an explicit sandbox prerequisite. Three separate, reviewed fixes are part of the tested integrated behavior: #13973 serializes task-run admission; #13977 captures complete event evidence; #13979 durably reconciles failed Daytona creation. Each has green CI and clean 5/5 review. Failed-create recovery has 277 plugin tests, 92 SDK tests, host-runtime recovery tests, and a real Daytona lost-deletion-receipt proof. The live proof uses a private file for journal persistence; database durability is covered by host tests. Worker death before delivery of a failure envelope remains outside that recovery mechanism. Subscription fixtures stage an authorized company login; interactive browser sign-in is not qualified. Local Product profiles ran on EC2 Linux. The temporary subscription credential was removed from the protected GitHub environment after all subscription audits, with absence verified. Runtime homes and refresh copyback remain ownership-fenced. Protocol results remain pinned to their original revisions; they are not relabeled as tests of the latest feature commit. New binary/model versions require qualification. Missing token usage and model cost remain unknown; runtime estimates do not establish a full bill. Automatic paid Grok scheduling remains disabled pending separate reviewed enablement. The 64 MiB bound can increase memory use for verbose sessions, and larger files still fail closed. No automatic legacy-agent migration occurs. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |