1752 Commits

Include renames

Author	SHA1	Message	Date
Nicky Leach	c0ac2772b0	test: stabilize migration and recovery fixtures (#16601)	2026-10-16 15:19:11 -07:00
DottaandPaperclip	93ab7bad11	Add immutable Muse clients and OAuth grant contracts (#16447) ... Add immutable Muse client assets and the grant-linked private contract. Keep the grant migration safe for replay and partial schema recovery, preserving exact preview receipts and existing data. Keep public wake signals separate from private provider authority. Refs #16447. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 14:09:26 -05:00
DottaandPaperclip	2cc7992369	Share the external Runner lifecycle and add strict Muse contracts (#16444) ... Add the external Runner foundation with distinct Dot/Grok V6, canonical home-mode V7 and Muse V8 contracts. Keep generated schemas and profiles synchronized. Bound default question-answer command IDs in a disjoint namespace while preserving existing IDs and durable replay. Preserve canonical migration history and add normally generated, replay-safe Muse schema milestones with literal preview-history fixtures. Refs #16444. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 13:46:50 -05:00
Nicky LeachandPaperclip	6f8ad3d77b	Add verified session-local stdio MCP transport to Runner (#16526) ... ## Thinking Path > - Paperclip manages AI agent runs and their tool authority. > - Runner supplies provider sessions with assigned MCP connections. > - The current native connection binding supports an HTTP gateway only. > - Execution hosts also have private local tools with stdio MCP interfaces. > - Those tools need session-local registration and safe concurrency. > - This pull request adds verified local stdio assignments beside the existing gateway. > - Providers receive the existing tool schemas and image content. ## Linked Issues or Issue Description **Problem or motivation** Runner can supply a native provider session with an HTTP MCP gateway, but cannot also assign a verified local stdio MCP server without persistent provider configuration. Execution hosts need a session-local transport for local MCP tools, coordination for calls sharing a resource, and native schema/image delivery. The pinned native Codex client drops MCP images when `structuredContent` is also present. **Proposed solution** Add general stdio MCP transport support using host-assigned executable digests, arguments, environment and a private Unix socket identifying the shared resource. Tool implementations supply their own schemas and responses; Runner forwards their native MCP schemas and content. Register the transport for the current provider session. Serialize calls across processes, keep unknown outcomes fenced, and authorize assigned native Codex tools without a new prompt while routing explicit tool confirmations through existing PRP permission requests. Retain the HTTP gateway and applicable permission policy. **Alternatives considered** Reimplementing tools or translating their actions into provider-specific APIs would tie the transport to individual tool implementations. Global provider config would leak session assignments. Shell commands alone do not deliver native MCP images or preserve MCP approval requests. **Roadmap alignment** This completes the existing Runner transport boundary. Related: #10690 (operator config for the legacy ACPX engine), #11605 (gateway delivery of installed stdio connections), #16486 (preconfigured provider homes). This change assigns local tools on the execution host and leaves those administration paths intact. ## What Changed - Add reserved, verified stdio bindings for native Codex and through the shared ACPX runtime. ACPX clients must support stdio MCP assignments. Read-only sessions omit the binding. No home/project config is rewritten. - Run the backend with only its assigned environment through the verified executable launch path. - Serialize all calls sharing a private Unix socket across sessions. Metadata remains responsive. Queued cancellation prevents execution; active calls remain fenced until acknowledgement. Unknown outcomes require a resource restart. - Authorize host-assigned native Codex tools without a new per-call prompt, matching existing execution-host shell utilities. Preserve explicit tool deny/prompt rules. Route requested MCP tool confirmations through one-time PRP approvals, bound to the active thread and turn. Keep shell escalation disabled and existing tool deny rules effective. ACPX adapters retain their applicable ACPX permission policy. - Preserve schemas and image blocks. For native Codex 0.160/0.162, retain structured metadata as MCP text on image-bearing results to avoid the client's image-loss bug. ACPX adapters receive the original response. - Document transport authority, cancellation and recovery limits. ## Verification - `pnpm -r typecheck` and `pnpm build` passed with pnpm 9.15.4 and Rust 1.97.1 on macOS arm64. - Before Apex, full `paperclip-runner-core` Rust tests passed. After the fixes, all 379 library tests and eight subprocess tests using a generic stdio MCP fixture pass. Those subprocess tests cover schema/image passthrough, assigned-only environment, executable drift, call serialization, cancellation and unknown outcomes. The native Codex suite passed 91 tests with two intentionally ignored; `receipt_limit_rejects_the_call_and_keeps_polling_when_interrupt_fails` timed out waiting for `turn.interrupted`, including an isolated retry. The unchanged pre-fix commit `80917b301a2309f7966a466cf0c198a2112be2d5` reproduces the same timeout in isolation; this local native-suite run is not claimed as fully passing. - All 252 native controller tests pass against the final narrowed approval branch. Focused TypeScript transport/permission suites passed (298 tests); final binding, read-only host, policy and adapter checks passed (105 tests). - **Provider-client smoke coverage:** pinned Codex 0.160.0 through native Runner and Claude SDK 0.3.286 used local stdio MCP servers and loopback model API fixtures without live model credentials. These clients define the smoke-test coverage; this is not an exhaustive adapter compatibility test. Image responses reached the next model request as typed image blocks. Native Codex checks also verified unattended execution without approval requests, explicit per-tool prompt denial without backend execution, disabled-tool rules, and stale-turn approval rejection. - Two Greptile concurrency findings were reproduced with failing subprocess tests and fixed: pending cancellation under output backpressure, and a ctime-based fence reset after chmod. Initial checks exposed Codex CLI option-scope and key-quoting errors, which were corrected. The existing ordinary elicitation regression also exposed an overly broad request branch; the final branch handles explicit tool approvals only. - `pnpm test:run` was attempted. The broad local server suite hit persistent embedded PostgreSQL startup failures and unrelated local Git fixture failures; it was stopped with the failure log retained. Those local results are not claimed as passing. Hosted Linux CI gates passed on the reviewed runtime commit, as recorded below. - First Apex review found three additional defects, now fixed with regressions: keep the resource-lock socket snapshot instead of sampling the socket twice; project native permission requests with durable turn/item IDs while retaining provider IDs in details; validate provider response conversion before marking a request as settling. The retry regression failed before the fix. Focused Codex request/adapter checks pass (23 tests), server permission projection passes (16 tests), and all 379 Runner library tests pass. The broader database-backed bridge suite was attempted but its local PostgreSQL setup timed out; 29 tests did not execute. This local failure is retained, not claimed as passing. - Runtime acceptance before the documentation clarification, on `bd4fccae001157beb5bc76ff17c1920a5db3105a`: standard Greptile 5/5, then Apex 5/5 with no unresolved threads; all 60 checks complete successfully or intentionally skip. Ten CI jobs were interrupted by the same runner shutdown, including one stale job that required cancellation before retry; the same-commit retry passed. Hosted Rust tests also pass, including the native suite with the local baseline timeout noted above. - `git diff --check`, map validation, and pre-push secret/PII scan passed. ## Risks - No live model account or deployment isolation qualification was run. The loopback fixture proves the actual vendor-client request format, not model interpretation. - Stdio MCP support uses the current host-assignment contract: each binding requires a verified executable and a private Unix socket identifying the serialized resource. Other platforms retain the HTTP path. No provider version was upgraded. - A lost acknowledgement blocks subsequent calls until the host restarts the assigned resource. Direct shell calls are outside this MCP fence and retain the existing shell policy. - Preconfigured Claude remains unattended and denies interactive approval requests. Host assignment authorizes native Codex local tools; it must be supplied only for sessions already authorized to use the resource. ## Model Used OpenAI Codex, GPT-6, with reasoning and repository/tool execution. The session does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 10:39:38 -07:00
DottaandPaperclip	ce0eff7d1f	perf: reduce fresh workspace clone and sandbox staging waits (#16487) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Each run needs a private workspace and safe persistence. > - Fresh isolated repository sources download upstream Git history again. > - Sandbox staging also verifies an optional seed cache before it uploads the workspace. > - That verification holds a company lock during a full extraction and hash scan. > - This pull request reuses pristine Git objects through an authenticated remote clone. > - It also overlaps seed verification with upload and moves verification outside the publication lock. > - The benefit is less repeated transfer and less serial preparation work, with the same task isolation and recovery contract. ## Linked Issues or Issue Description **What existing behavior does this improve?** Fresh task-private repository preparation and inbound sandbox workspace staging. **Current behavior** Each isolated remote source clones the full upstream history. Optional seed verification then extracts and hashes another full tree before upload. One workspace's verification can hold the company cache lock while other workspaces wait. **Proposed behavior** Use a company and credential scoped cache of pristine upstream objects for GitHub HTTPS sources. Each task still clones from the current remote. It disconnects from the cache before publication. Run seed verification alongside upload. Hold the company lock only for final capacity checks and publication. **Reason and benefit** These operations lie on the workspace preparation path. Removing repeated Git pack transfer and serial verification can reduce startup time. Local measurements below cover these boundaries. They do not establish hosted agent latency. Related: #16425, #16454, #16398, #16461, #16468, #16473. The cache inspection and durable folder proposal in #13376 covers separate product work. ## What Changed - Add a pristine upstream Git object cache for fresh isolated GitHub HTTPS sources. Key it by company, repository and current credential fingerprint. Do not store credentials. - Keep online clone authorization and current remote refs. Use `--reference-if-able --dissociate` so task objects remain independent. A reference failure retries once online without the cache and resolves credentials again. - Cap cache storage at 64 entries and 4 GiB per company, with 2 GiB per entry. Skip contended publication. Bound priming by a cooperative five-second budget and a 4,096-entry scan allowance. Keep the existing free-space reserve. - Overlap optional seed verification and required workspace upload. Settle both before returning or deleting temporary files. Recheck quotas and company directory identity under the publication lock. - Keep the independent fresh upstream reproducibility proof, task commits, dirty source copies, ignored files, nested repositories and durable recovery behavior. - Suppress optional Git server-info generation during dissociation so generated storage passes the existing safe-expiration checks. This uses [the supported Git setting](https://git-scm.com/docs/git-config/2.43.0#Documentation/git-config.txt-repackupdateServerInfo). - Add bounded timing phases, regression coverage and two repeatable local benchmarks. No first-party Telemetry or dependency changes. ## Verification - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Affected adapter suites: 240 tests passed. Native workspace persistence/history and tracing: 33 tests passed. Server checkout/isolation/tracing batch: 53 tests passed. Final object-cache authority, path and capacity suite: 12 tests passed. These batches overlap. - Cache and expiration/restoration lifecycle batch: 34 tests passed. A direct generated-storage check covers cached clones. - Real HTTP Git fixture: a matching cache does not bypass revoked remote access. Other fixtures cover current refs/tags, hidden cached commits, independent objects after cache removal, corruption fallback, credential/company scope, quota limits and publication contention. - Initial restricted runs hit SQLite write and Unix socket permission failures. The adapter batch passed with disposable writable instance state and socket access. The first typecheck rejected the test use of `Promise.withResolvers`; typed deferred gates now use the supported library contract. - Repository benchmark on final head `9047368bf3b7fa2061e00d698b8b51cf4ae1da32`: five alternating samples, real local `file://` Git transport, synthetic 32 MiB history, no injected latency. Median 1143.8 ms to 240.5 ms; ranges 1085.7–1486.6 ms and 188.0–254.8 ms. Received Git pack bytes fell from 33,569,788 to zero. Initial cache priming took 1208.4 ms and is reported separately. - Sandbox benchmark, five alternating samples per mode on 10,000 files with real packing, extraction and verification: with a controlled 3-second transfer delay, median 9.450 s to 7.620 s (ranges 9.280–12.115 s and 7.296–7.983 s). With no injected delay, median 6.808 s to 6.450 s (ranges 6.255–7.029 s and 5.709–6.902 s). The no-delay ranges overlap; local copying competes with verification for disk. Both modes verify final tree contents, ignored-file exclusion and Git HEAD. These are synthetic local fixtures, not hosted agent runs. - Reproduce with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-repository-object-cache.ts` and `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-staging.ts --baseline-ref=307eb16c0c --transfer-delay-ms=3000`. - Prior head `af766bc820` received Greptile 5/5 with no findings. Its E2E shard 7 failed in `plan-task-fixture` cleanup: company deletion raced an `activity_log` foreign key. That fixture has no provider run or Git workspace. The failure is retained here; final-head CI must pass. The corrected corruption fixture now forces a pack and verifies a real damaged archive. Current head `9047368bf3b7fa2061e00d698b8b51cf4ae1da32` has Greptile 5/5 with no review threads or findings. Final head has 55 passing checks and two optional Storybook skips. [The first workspace-shard attempt](https://github.com/paperclipai/paperclip/actions/runs/38802441857/job/116655485350) passed all 1,778 cases, then reported a late `EPIPE` from unchanged `server-utils.ts` while the ACP reuse test was active. That case mocks workspace preparation and passed locally in isolation. [Its single retry on the same commit](https://github.com/paperclipai/paperclip/actions/runs/38802441857/job/116659728034) passed, as did the dependent verification gate. The earlier E2E cleanup fixture passed on this final head. Both initial failures remain recorded here. A full local suite was not duplicated; the affected boundaries were run locally and the complete suite runs in CI. ## Risks - First cache population adds local copying. Cold clones, credential rotation, capacity exhaustion and a corrupt cache retain the online clone path. No eviction is added. Cached data is bounded, but many companies can each consume their allowance. - Individual filesystem calls and cleanup can exceed the cooperative priming budget. Failed reference clones make one additional online attempt. - Parallel verification and upload compete for disk and CPU. The local no-delay benchmark shows a smaller gain than a slow-transfer case. Verification still checks contents before cache publication, and required upload failures remain failures. - Local fixtures do not prove hosted throughput or the 15–30-second end-to-end target. The fresh checkout still writes its working tree, and model startup, copyback and other preparation costs remain. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context window are not exposed in this session. Used reasoning, source editing, shell tools and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal task identifiers - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 12:21:47 -05:00
Devin FoleyandPaperclip	647116dd4b	fix(adapter-utils): allow sandbox agents to read run diagnostics (#16575) ... ## Thinking Path > - Paperclip manages agent work and its execution history. > - Sandbox agents use a callback bridge to reach the control plane. > - The API already provides authorized run diagnostics. > - The bridge denied those read routes before the API could check access. > - Health scans could see task activity but could not inspect failed runs. > - This pull request admits exact GET routes on both bridge transports. > - Linked task results also apply the caller's task-read filter. > - Agents can inspect failures through their existing run identity. ## Linked Issues or Issue Description **What happened?** A sandbox health scan received `403 Route not allowed` for run history, run details, events, and task recovery records. The same agent could read task activity and connected provider state. The bridge stopped these diagnostic requests before they reached the API. **Expected behavior** The bridge should admit the existing diagnostic GET routes. The API must still enforce company and task access and redact run content. Execution and recovery writes must stay denied by these rules. **Steps to reproduce** 1. Start an agent in a sandbox with the normal callback bridge. 2. Use its injected API credential to request company run history. 3. Request a run's details or events. 4. Observe the bridge's route denial before the API handles the request. **Paperclip version or commit** Reproduced with the callback route list on master at `9f4a956b906`. **Deployment mode** Authenticated server with a managed sandbox. Both queue and HTTP/2 transports use this route list. Related work: Refs #15854 adds separate cost-read routes to this list. It does not add run diagnostics. Searches found no duplicate run-diagnostic change. ## What Changed - Admit nine existing diagnostic GET paths: company history and live runs; run details, events, logs, workspace operations and task links; task runs and recovery records. - Reject other methods, extra path segments, execution control, recovery resolution, and provider-trace administration. - Filter linked tasks by the caller's task-read predicate in both activity and original-task queries. Keep those results in the run's company. A readable run must not expose private task titles. - Test both route lists and the HTTP/2 request path. Preserve query parameters and controller denial responses. Add a database regression with public and private tasks linked to a readable company run. - Document bounded diagnostic reads, private-run stubs, access checks, and credential handling in the agent API reference. Update both affected run access maps with the sandbox entry points and verification limits. ## Verification - Passed both bridge suites: 139 tests. - Passed the server run-route, activity-route, and secret-redaction suites: 117 tests, including the new task-link filter test. The cold route-import test needed a 30-second timeout on this host. - Passed `pnpm -r typecheck`, `pnpm build`, the adapter-utils typecheck, both generated capability checks, and `node scripts/check-maps.mjs`. - Attempted `pnpm test:run`. The bundled PostgreSQL binary cannot start on this host. The native session suite reported 15 database startup failures; the privacy suite skipped its 15 cases. Stopped that full local attempt after those failures. No full local-suite pass is claimed. The final 16-case privacy suite also skipped locally for the same host limitation; Linux CI ran all 16 cases successfully. - One unchanged queue-response test hit a file-read race during the combined local run. A separate rerun of both complete bridge suites passed all 139 cases. The transient race was not independently isolated; the original failure is retained in this record. - All CI gates pass on head `97c65cec459d3fc2787083d05ae6ec51e3fd7b70`. Linux CI passed all 16 database-backed run privacy cases, including the new public/private task-link regression. These also cover private-run denial, authorized owner/reviewer reads, redacted list stubs, credential diagnostics, and deleted-task handling: [serialized shard 1](https://github.com/paperclipai/paperclip/actions/runs/38925932547/job/117057907621). - Apex reviewed the final head at 5/5. Both findings are addressed and their review threads are resolved. - After deployment, run a new sandbox audit. Confirm it can list and inspect authorized failed runs without a board credential. This has not been deployed or live-tested by this PR. ## Risks - Sandbox agents can reach more diagnostic GET routes. The existing server access checks and redaction still apply. - Linked task results now omit tasks that the caller cannot read. Authorized task links remain visible. - This change does not grant execution control or recover a failed workspace. - Existing bridge sessions may need a new run to load the updated route list. - No schema, credential, UI, workflow, or lockfile change. ## Model Used OpenAI Codex, based on GPT-6, assisted with implementation and verification. The exact serving model ID and context-window limit are not exposed in this session. Capabilities used: code editing, shell execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 09:04:07 -07:00
Nicky LeachandPaperclip	5fbece9dcc	Synchronize assigned skills through agent runtime hooks (#16501) ... ## Thinking Path > - Paperclip manages shared skills and agent assignments. > - Runtime plugins need installed skills before a task starts. > - Skill content can change without changing the agent row. > - Source access and version pins also affect delivered files. > - This pull request feeds those changes into the durable runtime stream. > - The runtime reader resolves the same immutable skill sources used for execution. ## Linked Issues or Issue Description Builds on merged PR #16499. This pull request now targets `master` and contains only the five skill synchronization commits. **Problem** Plugins receive no durable notification when an assigned skill changes. A saved assignment can also become inaccessible after private-source access changes. **Proposed solution** Invalidate referencing agents at installed skill mutations. Invalidate inherited access when source bindings, project privacy, or policy change. Return authorized skill sources and version IDs through the owning runtime reader. **Alternatives considered** Activity-only subscriptions miss service writers. Per-task synchronization adds preparation work. Copying current files would ignore version pins. **Roadmap alignment** This extends existing skills, source synchronization, and plugin runtime support. ## What Changed - Defer multi-skill source runtime hooks until commit, after publication holds all skill rows. Keep file-save journaling immediate to preserve filesystem rollback. - Add transactional invalidations for skill content, companion files, versions, rename, deletion, policy, source access, and pin-setting changes. - Recognize canonical keys and supported legacy aliases. - Reuse the execution skill resolver, current agent access, and instance version-pin rules. - Resolve and materialize sources outside the agent row lock. Recheck ownership and runtime revision afterward. - Keep task-only mentions and running-task snapshots in the execution path. - Document conservative invalidations for pinned assignments and inherited access. ## Verification - Rebased onto origin/master `9f4a956b90`, which contains merged PR #16499. All five commits are unchanged according to `git range-diff`; no conflict resolutions or source changes were needed. - On rebased head `58b065941`, all 237 focused tests passed: runtime skill hooks/source publication (41), plus content service, assignment routes, policy routes, and project source access (196). - Server typecheck passed after rebuilding Runner with the installed Rust toolchain. Migration numbering/safety and map checks passed; snapshot pruning made no changes. Migration CLI checks required local IPC permission. - Real PostgreSQL coverage includes concurrent source publication/version saves, policy-only tightening/restoration, referencing-agent fanout, company isolation, no-op counters, supporting files, rename/deletion, rollback, private-source access, pinned/latest bytes, and unassignment. - Earlier stack validation passed workspace typecheck/build. Fresh hosted CI and review must verify the rebased head before handoff. - The broad local suite was previously limited by an unchanged diagnostic test that rejects "private" in an error stack under this /private/tmp worktree; that unchanged suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. - Diff whitespace and full PR secret/PII scan passed. - All hosted CI/security checks are green on `58b065941`; fresh standard Greptile and Apex both returned 5/5 with no findings or unresolved threads. The PR targets master and is mergeable without conflicts. ## Risks - Apply migration 0348 after the runtime-stream migration in #16499. - Content invalidations include pinned assignments. The resolved version stays pinned; consumers can avoid writes when its version/hash is unchanged. - Project access and source changes invalidate only agents assigned skills from the affected sources. Company policy still invalidates the company. Source polling bookkeeping and popularity counters do not create work. - The reader rebuilds installed caches without upstream requests. Concurrent changes cause a bounded retry and then a conflict. - Private transport integration and live remote computer qualification remain separate. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 08:28:47 -07:00
Nicky LeachandPaperclip	9f4a956b90	Add durable agent runtime synchronization hooks (#16499) ... ## Thinking Path > - Paperclip manages agents and their runtime configuration. > - Plugins can maintain external agent computers. > - Configuration and file saves currently miss durable plugin delivery. > - A transient activity notification cannot recover after interruption. > - This pull request adds an opt-in agent runtime resource stream. > - The existing dispatcher delivers and acknowledges its committed work. > - Plugins can prepare computers when configuration changes. ## Linked Issues or Issue Description Related: #15631 and #16203 establish agent lifecycle and native durable delivery. **Problem** Plugins cannot reliably react to agent configuration and instruction changes without polling or waiting for execution. **Proposed solution** Add an opt-in `agent_runtime` stream to the existing resource journal. Record changes at the database mutation and managed file audit boundaries. Wake the dispatcher with content-free commit notifications. Provide scoped runtime inputs to the selected lifecycle plugin. **Alternatives considered** Route-only activity events miss service and background writers. Scheduled scans delay changes and repeatedly query unchanged state. **Roadmap alignment** This extends the existing plugin and agent configuration work. ## What Changed - Add durable update/delete events for agent runtime state, including an existing-agent baseline. - Capture configuration changes, instruction edits, deletion, restore, and managed run writeback. - Add commit notifications and a reconnect scan. Register one listener for configured, enabled runtime subscribers. Release it when none remain. - Keep HTTP startup independent of queued plugin callbacks. - Add `ctx.agents.getRuntime`. Require lifecycle capability, invocation company, enabled plugin, and owning lifecycle selection. - Keep legacy agent/project subscriptions isolated. Manual inbox reads retain both legacy streams. - Resolve recovered instruction paths without reading or hashing bundle contents. Use the company reporting graph for runtime health. - Repair missed file events on identical save/deletion retries after a journal failure; preserve content tokens and conditional-write checks. - Correlate file audits with runtime revisions to deduplicate recorded retries without a new table. Bound the lookup with the existing company/timestamp index. - Document the contract, migration, access boundaries, and session-connection requirement. ## Verification - Rebased the full stack onto origin/master after #16507. - Final replayed stack: `pnpm -r typecheck` and `pnpm build` passed. - All 132 affected runtime, skill, connector, publication, instruction, managed-agent, and delivery tests passed after the latest-master rebase. Workspace typecheck and build passed. - Journal failure injection verifies recovery on identical file writes, deletions, and instruction retries from an acknowledged inbox. - Deduplication follow-up: 80 runtime/instruction/delivery tests and server typecheck passed. Repeated retries create no audit/events; old hashes returning after an intervening commit still repair delivery. Built-in/native instruction writer regressions passed all 56 tests. - PR 1 follow-up tests cover valid/cyclic reporting chains, listener registration failure, multiple subscribers, global/company disable, re-enable, unload, and startup with blocked callbacks. - Database coverage includes rollback, journal failure, no-op suppression, file saves/deletion, ordering/replay, owner/company denial, independent-client commits, and real listener loss/reconnect. - SDK/shared suites passed 1,059 tests during implementation. CLI/delivery regressions also passed. - Migration safety, map checks, diff whitespace, and local secrets/PII scan passed. - Hosted CI and security checks are fully green at `b296405e2`. Standard Greptile and Apex both returned 5/5 with no unresolved findings. - The broad local suite encountered an unchanged diagnostic test that rejects the word "private" from an error stack. This worktree lives under /private/tmp. The same unchanged diagnostic suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. ## Risks - Apply migration 0347 before enabling a runtime subscriber. Runtime notifications require session-level PostgreSQL LISTEN. Transaction-mode poolers are unsupported for this opt-in feature. - Database triggers share the mutation transaction. Journal failure prevents configuration commits. Trigger work adds a journal insert to relevant changes. - Filesystem publication and SQL commit are separate boundaries. Direct filesystem edits do not emit hooks. External clients must fence stale revisions and retain conditional-write checks. - The reader returns only that plugin's own instance secrets. Keep them out of UI data, logs, plugin state, and agent files. - No private transport integration or live remote computer qualification is included. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-16 07:53:04 -07:00
DottaandPaperclip	d75b3fc33f	perf(daytona): observe command exit before log-follow closure (#16561) ... Return completed Daytona session commands after recorded exit and a final log snapshot. Keep bounded early status reads, command deadlines, late callback fences and stream recovery. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 07:09:30 -05:00
DottaandPaperclip	8556539364	perf(adapter-utils): batch remote GitHub launcher staging (#16552) ... Stage the nine managed GitHub launcher files through one bounded control command. Preserve per-file locks, content verification, atomic replacement and control-channel dispatch. Real Daytona staging decreased from about one second to 0.24–0.34 seconds. Full CI, E2E, Superagent and Apex 5/5 passed on the reviewed head. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 04:59:21 -05:00
Nicky LeachandPaperclip	fa9f3f8a08	feat(runner): support preconfigured user homes for Codex and ACPX (#16486) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Runner launches the selected provider and keeps durable session state. > - Runner currently prepares private provider homes for each session. > - Operators can already have working login, skills, and MCP configuration on the execution machine. > - This PR adds an explicit mode that uses that existing setup. > - Runner keeps its task bridge and session ownership while leaving the user's configuration in place. ## Linked Issues or Issue Description **Problem or motivation:** Rebuilding a provider home is unnecessary when the execution account is already configured. **Proposed solution:** Add `homeMode: "preconfigured"` to Runner adapter config and runnerd transport options. Keep `"isolated"` as the default. Initially support native Codex and ACPX Claude. **Alternatives considered:** Continue copying user configuration into a private home, or add an approval workflow. Both add work outside this feature's scope. **Roadmap alignment:** This extends the existing cross-provider Runner. It does not add another runtime. Related work: #14056 covers managed Codex credentials. #14474 covers command environment delivery. This mode uses user-owned credentials and retains the existing launch boundaries. ## What Changed - Add a closed v7 launch contract for the selected home mode. Existing contracts retain their behavior. - Keep homeMode under operator control. Reject task overrides and agent-authenticated changes, including removal through replacements, adapter changes, or rollback. Ignore stored task values and exclude it from automatic hire inheritance. - Skip managed AI login, assigned-skill materialization, and external MCP setup in preconfigured mode. Preserve the Runner task/completion bridge. - Keep protocol state and provider lifetime fences scoped to the Runner session. Different sessions can share a home. - Fail preconfigured ACPX planning/read-only launches before provider admission until a read-only user-home launch is qualified. Use isolated mode or native Codex for planning. - Reject ACPX approval requests. Keep native Codex on `never`. Local permission rules and hooks remain effective. Initially require the default Runner ACPX `approve-all` setting; other Runner policies are unqualified in this mode and fail before launch. - Keep the internal ACPX bridge in a stable session namespace, preserving an existing user MCP server named `paperclip`. - Reject preconfigured ACPX Codex before launch until a session permission profile can enforce Runner-state denial. Remove the unprotected full-access mode switch; use native Codex or isolated mode for that model family. - Bump Pi to profile 23 because its attestation covers shared Runner source changed here. Preserve profile 22 for historical recovery, and update Docker/package verification inputs. - Fence recovery on mode and home bindings. Fail missing provider-history recovery without starting a replacement session. Exclude the external Codex home from backups and credential cleanup. - Discover remote HOME on the execution machine. Do not project controller HOME or XDG paths onto it. - Separate task instructions from Codex ACP isolation and update the generated provider digest. - Keep remote configured-environment manifests consistent when dropping controller paths. Preserve exact Claude model IDs through per-launch environment settings, without editing the user home or patching the provider. ## Verification - Targeted local checks cover native contracts and launch allowlists, user-config preservation, provider admission, permissions, session recovery, remote environment projection, heartbeat preparation, and operator-only configuration. Runner, shared, and server TypeScript checks and the Rust binary build pass; generated ACPX profiles are current. - Following the latest master integration, all 757 server configuration/executor tests and 115 Runner admission/recovery tests passed. Additional review-fix checks passed: 103 planning contract/host tests, 78 provider-backend tests, 95 route/preparation/inheritance tests, 43 task-validator tests, and 43 workspace-policy tests. Counts describe separate runs and can overlap. - The real pinned Claude ACP/SDK regression reproduces exact-model aliasing without the fix (`claude-sonnet-5-5` becomes `sonnet`) and passes with the fix. All 89 adapter tests pass. User settings remain byte-for-byte unchanged. - Localhost-only Claude SDK probes with synthetic credentials verify the private bridge runs, local `ask` reaches the existing reject callback, local `deny` blocks without a callback, and a user MCP server named `paperclip` remains available. No user model allowlist is required. - Review regressions cover fail-closed provider admission, saved-mode recovery before routing, controller-home path removal without configured-environment manifest drift, pre-launch rejection of unqualified read-only ACPX launches, and denial of task/agent changes to the operator-owned home mode. - Before rebase, `d4d4ee2aebe4cb88c4f4d9ef8e22fed4b36b87be` received standard Greptile 5/5 with no unresolved threads. Prior Apex findings are fixed and covered by regressions. No manual Greptile retrigger is requested for this rebase, per reviewer direction. - The prior PR head and master hit the same GitHub runner encoding crash in workspace-b on three PR attempts, including diagnostic logging ([prior PR CI](https://github.com/paperclipai/paperclip/actions/runs/38818474563/job/116714538405), [prior master CI](https://github.com/paperclipai/paperclip/actions/runs/38819528748/job/116712423193)). Rebased onto `origin/master` at `c3ffa3f0dd` to pull in its bridge-test index correction. The rebased head `edd153db410bea063b645e0000aa68649f287b00` passes all CI checks ([CI run](https://github.com/paperclipai/paperclip/actions/runs/38824676914)), including workspace-b. Greptile's automatic standard review is 5/5 on this exact head, with no unresolved threads; no manual review was requested. - Local workspace-b investigation passed 911 shared tests and 3,868 tests across the remaining 15 workspace projects. The catalog run separately hit an existing ignored `.agents/skills/design-guide/SKILL.md` description-budget violation in this checkout; that file is untracked, outside the PR, and left untouched. Repeating shared tests with CI reporting enabled also passed. - Rebase verification: the final Git tree exactly matches the previously reviewed branch combined with current master. All 192 Runner admission/contract tests, 61 server preparation/provider tests, and 68 bridge tests pass, as do Runner/server TypeScript and generated-profile checks. The real Claude adapter fixture hit an `ENOTEMPTY` cleanup race on its initial run; all 89 adapter tests passed on retry. - Apex on rebased head `edd153db` returned 2/5: ACPX Codex selected full access without enforcing Runner-state denial, and agent configuration replacement could remove the operator mode. Both are fixed in `26e14dbbf`. Preconfigured ACPX Codex is temporarily unqualified; re-enabling it requires session permission enforcement on thread/turn operations and a real filesystem-denial probe. The config guard covers effective replacements, adapter changes, and rollback while allowing ordinary agent merges and operator removal. All 190 focused Runner tests, 103 server tests, both typechecks, and generated-profile checks pass. Standard review and CI passed before the next Apex request. - CI on `26e14dbbf` passed 1,967 Runner-shard tests but hit the previously observed `ENOTEMPTY` fixture cleanup race after successful Claude model admission. `eb83284fb` adds bounded filesystem retries to that fixture's teardown; all 87 adapter tests pass. Assertions and production cleanup behavior are unchanged. Head `eb83284fb42ee19a15d9a93b106c1bbb07e1695e` passes all 59 CI checks ([CI run](https://github.com/paperclipai/paperclip/actions/runs/38829500857)) and standard Greptile at 5/5 with no unresolved threads. Apex also completed at **5/5** on that exact head ([Apex result](https://github.com/paperclipai/paperclip/pull/16486#issuecomment-6201559527)). All review threads are resolved, the branch has no merge conflicts, and the PR remains unmerged. - No live paid-provider or remote-machine smoke was performed. The lockfile remains unchanged under the repository's refresh-bot policy; local dependency installation resolved the changed patch hash. ## Risks This explicit opt-in grants Codex tools access to the execution home (read-only in planning), including credentials. Native Codex separately denies Runner-owned protocol state. ACPX Claude retains its existing provider trust boundary, and the isolated default is unchanged. Providers may refresh credentials and update their own metadata. User configuration can block tools or the completion reporter. Existing blocked results and failure diagnostics remain the recovery path. Edit the local settings and retry; no new approval workflow is added. External provider history remains on the execution machine and is not backed up by Runner. Remote HOME discovery requires `getent` and fails explicitly when unavailable. Other harnesses need separate qualification. The Codex ACP bundle digest changes. Existing sessions tied to the old bundle may need a fresh session. No database migration is required. ## Model Used OpenAI Codex, GPT-6, with reasoning and code execution. The exact runtime model ID was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the feature issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 17:57:58 -07:00
Nicky LeachandPaperclip	c3ffa3f0dd	test(adapter-utils): fix feedback request index in bridge round trip (#16507) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox agents use the callback bridge to call the Paperclip API. > - The bridge round-trip test checks request delivery and credential removal. > - #12785 added an accepted API-key request before the feedback request. > - The feedback assertions still read request 2, which is now the GitHub credentials request. > - This PR checks request 3 and verifies that all four accepted requests reached the host. > - This restores the shared test gate while retaining the Unicode payload and token checks. ## Linked Issues or Issue Description Related PR: #12785 introduced the stale feedback index. The reported failure appears on #16486 and master. Searches for `workspaces-b`, `sandbox-callback-bridge`, `surrogate`, and `EncoderFallbackException` found no duplicate fix. **What happened?** The `General tests (workspaces-b)` job fails on PRs and master. The hosted PR job and AWS master job both report `System.Text.EncoderFallbackException` for an unpaired `\uD83D`. The runner exits during the test step and does not upload that step's log. The unchanged master test reproduces a deterministic assertion failure at the feedback request. It compares `/runtime-tools/github/credentials` with `/api/companies/company-1/agent-commentary` and prints a diff containing 5,000 crying emoji. The assertion output is the likely trigger for the runner's Unicode logging crash. - [Reported PR job](https://github.com/paperclipai/paperclip/actions/runs/38818474563/job/116719880033) - [Master job with the same runner exception](https://github.com/paperclipai/paperclip/actions/runs/38821128038/job/116717690951) **Expected behavior** The test checks the actual feedback request, verifies its full payload, and confirms that its authorization header is removed. Both shared CI lanes complete normally. **Steps to reproduce** 1. Check out master at `c4a16cbe981f96e2756c7014103c88da67e5514f`. 2. Install dependencies with pnpm 9.15.4. 3. Run `node_modules/.bin/vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts -t 'round-trips localhost bridge requests'`. 4. Observe the feedback assertion reading the preceding GitHub credentials request. **Paperclip version or commit** Master at `c4a16cbe981f96e2756c7014103c88da67e5514f`. **Deployment mode** Built from source. GitHub-hosted PR CI and AWS post-merge CI. ## What Changed - Assert that four accepted requests reached the host. - Check feedback delivery and authorization stripping on request 3. - Retain the large Unicode payload and all existing authentication checks. ## Verification - Unchanged master: the focused round-trip test fails at the stale feedback index. - Fixed branch: all 68 tests in `sandbox-callback-bridge.test.ts` pass with real localhost gateways. - `pnpm --filter @paperclipai/adapter-utils typecheck` passes. - `git diff --check` passes. - Exact CI group: `node scripts/run-vitest-stable.mjs --mode general --group general-workspaces-b` passes (4,800 tests passed, 28 skipped across 17 projects). - `pnpm --filter @paperclipai/adapter-utils build` passes. - Greptile reviewed current head `3de15d3374354809ea84ca4df5f93c2473715f02`: 5/5, success, no comments or actionable findings. - All GitHub checks pass on current head `3de15d3374354809ea84ca4df5f93c2473715f02`: 55 successful checks and 2 intentional skips, including `workspaces-b`, build, typecheck, Runner verification, and e2e. - Full local `pnpm test:run` was started and stopped after the CI test lanes passed; it did not complete locally. No worktree-owned test processes remained afterward. - Full local typecheck and build were attempted. Both stop at the Runner Rust commands because `cargo` is absent from the shell PATH. The changed package typecheck passes. The complete repository typecheck and build pass in CI. - The first sandboxed local reproduction could not bind localhost (`EPERM`). It was repeated with listener permission and reached the deterministic assertion failure described above. ## Risks Low risk. This changes three test lines. Runtime behavior, credentials, workflow settings, and payload coverage stay the same. The GitHub runner Unicode logging defect remains upstream; this fix removes the incorrect assertion that triggers it in this test. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository inspection, and command execution. The exact runtime model identifier and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 16:32:20 -07:00
3 people	8aaef59339	fix(pi-local): add stdout compaction to stop quadratic run-log growth (#10249) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `pi_local` adapter runs the `pi` CLI in `--mode json` and records its NDJSON stream as a run-log per turn > - In JSON mode `pi` emits one `message_update` event per streaming token, each carrying the **full cumulative assistant message**, so a single turn's run-log grows quadratically with output length (measured ~77 MB for one thinking-model turn) > - The root cause is in `pi`, but the Pi project declined to add a compact mode (earendil-works/pi#3905, closed the same day), so Paperclip has to filter on the consumer side > - This PR adds a dedicated, config-gated compactor at the `pi_local` stdout boundary (`stdoutLogMode`: `raw` | `compact`, default `compact`) that strips the cumulative snapshots while preserving streaming deltas, terminal events, and parser/UI output > - The benefit is linear run-log growth (~150× smaller in practice) with **no change** to `parsePiJsonl` results, token usage, or live typing — keeping storage and replay bounded ## Linked Issues or Issue Description - **Refs #5699** — same root cause ("full accumulated partial state" → 46 MB log, per that PR). This PR takes a **different, single-concern approach**: a dedicated `stdout-compaction.ts` module with an explicit `stdoutLogMode` config (default `compact`, fail-safe fallback to `raw`), scoped to `pi_local` only. #5699 instead filters inline in `buffered-on-log.ts`, bundles an unrelated `claude-local` change, and is currently **conflicting** + stale (last updated 2026-05-14). Happy to coordinate with @ukint-vs on a merged direction. - **Refs earendil-works/pi#3905** — proposed adding a compact JSON mode upstream in `pi`; closed by the maintainer, which is why this filters on the Paperclip side rather than the producer. - **Related #1846** (run-log/retention TTL) — complementary: this PR reduces what's stored per run; #1846 addresses how long rows live. - No public GitHub issue exists that is an exact duplicate of the `pi_local` quadratic-growth symptom; the problem is tracked via #5699 above. (No internal/instance-local references used.) ## What Changed - **`packages/adapters/pi-local/src/server/stdout-compaction.ts`** (new, 123 lines) — `createPiStdoutLogCompactor(mode)`. `raw` passes lines through unchanged; `compact` strips the cumulative `message`/`assistantMessageEvent.partial` from `message_update` events while preserving streaming deltas, terminal events, and passthrough of unknown event types. Default mode `compact`; an invalid explicit mode fails safely to `raw`. - **`packages/adapters/pi-local/src/server/execute.ts`** — read the mode via `resolvePiStdoutLogMode(config)` and apply the compactor at the stdout → `onLog` (run-log) boundary. - **`packages/adapters/pi-local/src/index.ts`** — expose the `stdoutLogMode` config option (documented inline: `string`, optional, default `"compact"`). - **`packages/adapters/pi-local/src/ui/parse-stdout.ts`** — confirm the stdout parser consumes compacted output unchanged. - **Tests** — new `stdout-compaction.test.ts` (359 lines) covering modes, edge cases, and `parsePiJsonl` replay equivalence; updated `execute.remote.test.ts`. ## Verification - **Unit:** `pnpm --filter @paperclipai/adapter-pi-local exec vitest run` → **121/121 passing (12 files)**. - **Typecheck:** `pnpm --filter @paperclipai/adapter-pi-local exec tsc --noEmit` → **clean, 0 errors**. - **Real-data replay:** a real `pi --mode json` stream (`zai/glm-5.2`, thinking=high) compacted line-by-line through this compactor: - Raw **76.96 MB** → Compact **525.5 KB** = **99.33% reduction (≈150×)**. - 4503 lines in, 4503 out (**0 dropped**); **0** `message_update` with a cumulative `.message` remaining; **4489** streaming deltas preserved. - **Parser equivalence:** `parsePiJsonl(raw) ≡ parsePiJsonl(compact)` — **0 errors** on both; identical `finalMessage` (13,369 chars); identical `usage` (`inputTokens 2896, outputTokens 4491, cachedInputTokens 6848`). Result, token accounting, and live-typing are unchanged. ## Risks - **Behavioral shift:** default `compact` changes what is stored in run-logs vs today. Operators who need the verbatim stream can set `stdoutLogMode: "raw"`. Flagging for reviewer preference — happy to flip the default to `raw` for upstream if that's preferred. - **Ordering with secret redaction (#8027 / #9856):** compaction is structural and runs at the stdout boundary **before** redaction; it only removes cumulative duplicates and never alters token content, so it does not change what redaction sees. - **Unknown event types** pass through unchanged (forward-compatible with future `pi` events). - Low risk otherwise — the change is contained to the `pi_local` adapter. ## Model Used OpenAI `gpt-5.6-luna` — via pi's `openai-codex` provider (272K context window). Used for implementation and for the real-data measurement in Verification. Reasoning/thinking: extended thinking enabled at level `high` (pi `--thinking high`; maps to OpenAI reasoning effort). `gpt-5.6-luna` is a reasoning-capable model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com>	2026-10-15 16:24:06 -07:00
3 people	cc549691a1	fix(claude-local): stop sealing max-turns failures as model_not_found (#13885) ... <!-- Written in Simplified Technical English --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The `claude_local` adapter runs the Claude Code CLI and maps each finished process to a terminal result. The `errorCode` in that result decides which recovery path the issue takes. > - `execute.ts` resolves the error code with one long ternary chain. The order of the branches is the precedence rule. > - The model-not-found branch sits before the max-turns branch, and it is driven by `isClaudeModelNotFoundError()`, which regex-scans the run's full stdout for the phrase `model_not_found` (also `model not found`, `unknown model`, `model does not exist`). > - The agent's own tool output is part of that stdout. A run that greps adapter source, tails a log, or echoes an issue body about this defect matches the detector on text the agent printed, not on text the provider returned. > - A real `subtype=error_max_turns` failure therefore never reaches `max_turns_exhausted`. Downstream, `model_not_found` counts as a configuration-incomplete failure, so the issue goes to `blocked` with a wrong "configuration incomplete" notice instead of the normal max-turns continuation. > - This pull request gates the model-not-found branch behind the two structured signals that the `providerQuota` and `transientUpstream` branches above already apply. > - The benefit is that a structured CLI signal outranks a regex scan of untrusted output, so exhausted runs continue instead of blocking the issue. ## Linked Issues or Issue Description No public issue exists for this defect. I searched the open pull request list for `packages/adapters/claude-local` and for the `model_not_found` and `max_turns_exhausted` error codes, and found no duplicate or related pull request. The description follows the bug template. **What happened** A run that fails with `subtype=error_max_turns` is recorded with `error_code=model_not_found`. In `packages/adapters/claude-local/src/server/execute.ts`, the resolved-error-code ternary evaluates the model-not-found branch before the max-turns branch: ```ts : failed && isClaudeModelNotFoundError({ parsed, stdout: proc.stdout, ... }) ? "model_not_found" : failed && clearSessionForMaxTurns ? "max_turns_exhausted" ``` `isClaudeModelNotFoundError()` in `parse.ts` pushes `input.stdout` into its candidate list and tests it against `CLAUDE_MODEL_NOT_FOUND_RE`. The run's stdout holds the agent's own tool output, so any run that prints the phrase poisons its own classification. Because the branch is first, a genuine max-turns failure never reaches `max_turns_exhausted`. This is not cosmetic. Downstream, `model_not_found` is treated as a configuration-incomplete failure. The issue moves to `blocked` with a misleading "configuration incomplete" recovery notice instead of the normal max-turns continuation path. **Steps to reproduce** 1. Run a `claude_local` agent on a task whose tool output contains the literal string `model_not_found`. Grepping the adapter source or reading a bug report about this defect is enough. 2. Let the run exhaust its turn budget, so the CLI emits `subtype=error_max_turns`. 3. Observe that the recorded `error_code` is `model_not_found` and not `max_turns_exhausted`. **Expected behavior** A run whose parsed result carries `subtype=error_max_turns` is classified `max_turns_exhausted`, whatever phrases appear in its stdout. A structured CLI signal outranks a regex scan of untrusted output. **Paperclip version or commit** `ghcr.io/paperclipai/paperclip@sha256:23090eb6`. The branch is rebased onto current `master` (`b41ccf0`). The adapter is `@paperclipai/adapter-claude-local` on the `engine=cli` lane. **Deployment mode** Self-hosted, Docker Compose. ## What Changed - `packages/adapters/claude-local/src/server/execute.ts` - Gate the model-not-found branch with `!clearSessionForMaxTurns && !poisonedPreviousMessageId`, the same guard the `providerQuota` and `transientUpstream` branches ~30 lines above already apply. One line, no new state. - Add a comment that records why the order matters, so a later reorder is a deliberate decision. - `packages/adapters/claude-local/src/server/execute.max-turns-classification.test.ts` (new) - A parsed `error_max_turns` result whose stdout carries the literal `model_not_found` phrase still classifies as `max_turns_exhausted`. - A real provider 404 (`API Error: 404 model not found`) still classifies as `model_not_found`, so the guard does not hide the genuine case. Only the parsed-result path changes. The unparsed fallback path (`if (!parsed)`) is left alone: neither guard variable is in scope there, and a run with no parsed result has no structured max-turns signal to honour. ## Verification - `pnpm exec vitest run src/server/execute.max-turns-classification.test.ts` in `packages/adapters/claude-local`: 2/2 pass. - Negative control: with the one-line guard reverted, the first test fails with `expected "max_turns_exhausted", received "model_not_found"`. The test pins the defect and not the surrounding code. - `pnpm exec vitest run` for the whole adapter: 313 pass, 1 skipped, 3 fail. The 3 failures are in `acp.test.ts` and `acp.auth.test.ts`, all with `claude_acp_node_unsupported` because the local machine runs Node v22.23.1 and the repository requires Node >= 24.11.0. They fail the same way on `master` without this branch, and they do not touch the CLI classification path. - `tsc --noEmit -p tsconfig.json` for the adapter: no errors. ## Risks - Low risk. The change is one added condition on one branch of an existing ternary. - A real model-not-found failure still classifies correctly, because neither `clearSessionForMaxTurns` nor `poisonedPreviousMessageId` is set on that path. The second test covers this. - The narrow case that changes behaviour is a run that is both out of turns and pointed at a missing model. That run now reports `max_turns_exhausted` first. The next run reports the model error against a clean session, which is the same outcome the quota and transient-upstream branches already accept. ## Model Used - Provider: Anthropic Claude. - Exact model ID: `claude-opus-5` (Claude Opus 5). - Reasoning mode: extended thinking. - Capability details: tool use for shell, file editing and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (no user-facing documentation covers adapter error-code precedence) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Builder <builder@nublo.net> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-10-15 16:01:40 -07:00
Michel Tomas	5b15de9568	fix(adapter-utils): accept the bridge token in X-API-Key (#12785) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent can run inside a sandbox. The sandbox callback bridge is the loopback gateway that lets the agent call the Paperclip API from inside that sandbox > - The gateway authenticates each local call with a per-run bridge token. It read that token from the `Authorization: Bearer <token>` header only > - The run hands the agent that token in `PAPERCLIP_API_KEY`, next to `PAPERCLIP_API_URL` (`execution-target.ts`). The token is named an API key, and the public Paperclip API documents that an API key goes in the `X-API-Key` header > - An agent that follows the documentation therefore sends `X-API-Key` first, and its first call fails with 401 `Invalid bridge token.` > - The agent then retries with a bearer header and the same token. The run still succeeds, but it wastes a turn and it can leave a confused agent trying unrelated repairs > - This pull request makes both generated gateways read the bridge token from `X-API-Key` when the request carries no bearer token > - The benefit is that a documented header convention stops producing a needless failure, at no cost to the token check ## Linked Issues or Issue Description No existing issue found. I searched the repository for open pull requests matching `x-api-key` and `bridge token`. Nothing addresses the in-sandbox gateway header convention. The closest neighbours are unrelated: #12632 (Composio `mcp.headers.x-api-key`) and #12120 (the HTTP/2 transport for this bridge, already merged). The problem follows `bug_report.yml`: **What happened?** An agent running in a sandbox calls the Paperclip API through the loopback callback bridge. When it sends the credential as `X-API-Key: <bridge token>`, the gateway answers `401` with `{"error":"Invalid bridge token."}`. The token in that header is the correct per-run bridge token. **Expected behavior** The gateway accepts the correct per-run bridge token in either header convention. The public API documents `X-API-Key`, so an agent that follows the documentation succeeds on its first call. **Steps to reproduce** 1. Start a run whose agent executes in a sandbox execution target, so the callback bridge is active. 2. The run exports `PAPERCLIP_API_URL` (the loopback bridge origin) and `PAPERCLIP_API_KEY` (the bridge token) into the sandbox environment. 3. From inside the sandbox, call the bridge with that key in the documented header: ```bash curl -i -H "X-API-Key: $PAPERCLIP_API_KEY" "$PAPERCLIP_API_URL/api/agents/me" ``` 4. The response is `401 {"error":"Invalid bridge token."}`. 5. Repeat the same call with `-H "Authorization: Bearer $PAPERCLIP_API_KEY"`. The response is `200`. **Paperclip version or commit** Branched from `master` at `06c0e883f`. **Deployment mode** Self-hosted Kubernetes, with the agent in a sandbox execution target and a local model behind an OpenAI-compatible endpoint. **Agent adapter(s) involved** Not adapter-specific (core bug). Observed with a local-model adapter on a sandboxed lane. Any agent that follows the public API documentation sends this header. **Additional context** The variable name is the trigger. The run calls the credential `PAPERCLIP_API_KEY`, and an agent that has read the public API documentation knows an API key goes in `X-API-Key`. Paperclip's own in-prompt guidance uses `Authorization: Bearer $PAPERCLIP_API_KEY`, so the two conventions sit side by side. Across 12 observed runs on that lane, the agent's first bridge call used `X-API-Key` in 6 runs. Every one of those runs recovered by retrying with a bearer header and the same token. In some runs the agent first tried unrelated repairs, such as writing a test comment, before it changed the header. ## What Changed - Add one `readBridgeToken(req)` helper to the generated in-sandbox gateway source. The helper returns the bearer token from `Authorization` when the header carries one. It otherwise returns the `x-api-key` header value, or an empty string. - Use that helper in both generated gateway request handlers: the file-mode queue gateway and the HTTP/2 gateway. Both previously repeated the same two lines of bearer parsing. - The token still goes to the same `timingSafeEqual` comparison. The change does not alter the comparison, the token, or its lifetime. - Extend the file-mode round-trip test to cover an `X-API-Key` request with the right token, an `X-API-Key` request with a wrong token, and the fact that the header never reaches the host. - Add one HTTP/2-path test that covers the same three cases, and that proves a bearer token still wins when a request carries both headers. ## Verification Commands run on this branch: ``` node_modules/.bin/vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts Test Files 1 passed (1) Tests 52 passed (52) cd packages/adapter-utils && ../../node_modules/.bin/tsc --noEmit 0 errors in the two files this pull request touches ``` Both new tests are real regression guards. With the test file in place and the source change reverted, each one fails with `expected 401 to be 200`: the file-mode round-trip test and the HTTP/2 test. Both pass with the source change applied. A reviewer can confirm the security properties by reading two things. First, `tokensMatch` in the generated source is unchanged, so the constant-time comparison against the same per-run token is unchanged. Second, `allowedHeaders` in the same source is a closed allowlist of `accept`, `content-type`, `if-match`, and `if-none-match`. `normalizeHeaders` drops every other header, so neither `authorization` nor `x-api-key` leaves the sandbox on either transport. The independent host-side check in `http2-bridge-server.ts` is untouched, because each gateway sets `authorization: Bearer <token>` on the request it forwards. **ROADMAP.md check:** I grepped `ROADMAP.md` for `bridge`, `x-api-key`, `redaction`, and `sandbox callback`. There were no matches. This is a small bug fix in an existing subsystem, so it does not overlap planned core work. **Duplicate search:** I searched pull requests in this repository for `x-api-key` and `bridge token`. No open or merged pull request changes how the in-sandbox gateway reads the token. ## Risks Low risk. - The change widens which header the gateway reads. It does not widen what the gateway accepts. A request still needs the exact per-run bridge token, and it still passes the same constant-time comparison. - A bearer token keeps priority. A request that carries both headers behaves exactly as it did before, which a test now pins. - A caller that presents a non-bearer `Authorization` header and a valid `X-API-Key` header now succeeds instead of failing. This is intentional: the bridge only ever issues bearer tokens, so no bridge caller relied on that failure. - The header allowlist already excluded `x-api-key`, so no forwarded request changes shape. - No migration, no configuration change, and no behavior change for a caller that already sends a bearer token. ## Model Used Claude Opus 5 (`claude-opus-5`), extended thinking at high effort — verify before merge. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — no documentation describes the bridge's header convention, so there is nothing to update - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- <sub><em>**[@superbiche](https://github.com/superbiche)** · contributor · drafted with Claude Opus 5, reviewed before posting; the voice and decisions are mine.</em></sub>	2026-10-15 15:31:31 -07:00
Nicky LeachandPaperclip	ec3eb1968b	feat(runner): expose credential updates through PRP (#16463) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Runner uses PRP to send harness events to the host. > - A harness can replace its stored OAuth tokens during a run. > - The host has no PRP signal that tells it to read the updated credential. > - This pull request adds a durable credential update event and observers for supported stores. > - The host can react before it acknowledges the event. Token values stay outside PRP. ## Linked Issues or Issue Description **Subsystem affected** `packages/paperclip-runner`: native provider execution, PRP schemas, and host event delivery. **Problem or motivation** A Paperclip host that consumes PRP needs to react when a harness changes its refresh-token credential. Existing credential copy-back paths do not provide a common event during a warm session. **Proposed solution** Add `auth.credentials.updated` as a P0 runner event. Include only the provider, binding ID, revision, and detection method. Observe native Codex and Codex/Grok ACPX credential files. Retain each pending event across runner restart until the durable outbox accepts it. Raw PRP consumers opt in with `observeCredentials: true` before session open. The bundled transport opts in for fresh sessions. **Alternatives considered** `harness.diagnostic` would mix a runner observation with harness diagnostics. Sending token values through PRP would expose credentials to event logs. Both approaches are unnecessary. **Roadmap alignment** I checked `ROADMAP.md`. This change extends the existing Runner protocol and does not duplicate a planned core feature. **Additional context** Refs: #11094, #14056, #12696. These describe credential copy-back and staging work. This PR adds the notification contract. It does not replace those private credential channels or claim to fix their full scope. ## What Changed - Add strict credential update payload validation to PRP v1, v2, and v3. Regenerate validators, the manifest, and the event catalog. - Watch supported credential directories with `notify`. Add a periodic read as a fallback and check at terminal and close boundaries. - Persist a private baseline and pending event together. Notify on account identity changes and retain the new baseline for later refreshes. Ignore incomplete files, unsafe files, and metadata changes. - Forward the event through the host transport. Test commit-before-ACK delivery and replay. - Commit pending runner observations before suspend and shutdown commands complete. Preserve the provider event backlog. - Create the private ACPX runtime directory before configuring observation. Cover fresh session startup with observation enabled. - Fence asynchronous PRP commits to the current authenticated connection and run authority. Ignore replaced callbacks while preserving the maintenance drain of admitted work. - Deliver observations found during final provider cleanup before clearing terminal recovery state. Retain the outbox and recovery state when the host ACK is lost. - Advance the pending Copilot candidate profile to v19 for the changed shared PRP source hashes. Preserve its v18 history and pending qualification status. Include the current fixture in the Docker build context. - Commit credential updates through an awaited host callback before PRP ACK, including during idle and close. Persist them in a separate run-log source namespace with company, agent, session, and runner checks. - Preserve public event metadata through server redaction. Register the event in task run details. - Document opt-in behavior, supported stores, event semantics, and the local run-log path. ## Verification - All 45 durable runner control tests and 24 native executor integration tests pass. - Ten Rust credential observer tests pass. They cover atomic replacement, polling, recovery, account-only changes and later refreshes for Codex and Grok, unsafe files, fresh runtime creation, and the Grok close handoff. Account identity stays private. - The full Runner TypeScript suite passes: 3,565 tests in 221 files. An additional 12 tests are skipped by the suite. - All 214 focused tests pass for the contract, Codex event routing, catalog, and durable host delivery. - All 13 Node protocol contract tests pass. - The exact Docker context integrity command passes: `docker buildx build --file .github/docker-context-checks.Dockerfile .`. - Rust formatting and generated protocol checks pass. - `pnpm -r typecheck` and `pnpm build` pass. The full local `pnpm test:run` was attempted with CI-compatible `init.defaultBranch=master`, then stopped after a reproducible failure outside this change: `git-workspace-expiration.test.ts` expects one re-expired workspace but gets zero. The same case fails in isolation on this macOS checkout. Full repository validation is provided by CI; the focused Runner and host tests pass locally. - All 253 native transport and restart tests pass against the final release binary. - All 3 durable recovery tests and 139 candidate profile and native admission tests pass. - All 41 host redaction and task protocol surface tests pass. `pnpm check:token-gates` passes. - All 467 CI policy tests pass, including the 15 CI profiling tests. The upstream secret-reference test fix is retained from master. No workflow changes are included. - The Apex host-delivery fix passes 381 targeted Runner tests, six host database tests, repository typecheck, and the production build. The Runner tests cover commit-before-ACK, retry after a rejected host commit, warm attachment, and close after the turn queue ends. - Prior head `2f125cfcb` passed CI, standard Greptile 5/5, and Apex 5/5. One CI shard was interrupted by GitHub runner shutdown; its single rerun passed. Both Apex findings are addressed: idle/cleanup host delivery and late callback reconnect fencing. Current head `5db78f082` passes CI (56 successful statuses and four optional skips) and standard Greptile 5/5. Apex review of the account-change update is 5/5 on this exact head, with no actionable issues. - All 121 durable control-plane and credential callback tests pass, including account-only change delivery, subsequent refresh on the new account during close, host commit retry, late success/failure races, forward cursor delivery on a replacement connection, and the existing maintenance retirement contract. All 27 native provider and durable recovery integration tests pass after the account-change update. ## Risks - File changes signal a stored credential update. They do not prove that an OAuth refresh succeeded. Changes between reads can be combined. - Unix Codex and Codex/Grok ACPX stores are supported. Other harnesses and credentials held only in memory need their own integration. - Hosts must update their event schemas before opting in. Older hosts omit the flag and receive no new event. Older runners ignore the flag. - The host must use its existing private credential channel and account checks to read or copy the token. Account switches trigger notification but do not authorize accepting the new account. The event grants no credential access. - Tests use synthetic credentials. No live OAuth refresh has been forced. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`, with high reasoning effort, repository inspection, tool use, and code execution. The session does not expose its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 14:49:00 -07:00
DottaandPaperclip	8843887c24	fix(runner): preserve managed GitHub access in OpenCode (#16488) ... ## Thinking Path > - Paperclip manages AI agents and their authorized connections. > - Native OpenCode agents use managed Git and GitHub launchers for shell work. > - Each operation must obtain credentials from the active run's broker. > - Two OpenCode environment filters dropped that broker binding before the shell started. > - The shell therefore ran anonymously even when connector tools had authorized access. > - This pull request preserves the explicit broker environment through both filters. > - The benefit is working GitHub shell access with current identity and revocation checks. ## Linked Issues or Issue Description **What happened?** Native OpenCode shell commands reported `GitHub capability_missing` despite an authorized GitHub connection. Both runner startup and provider startup dropped the controller's broker settings. Connector MCP calls used a separate path and could still succeed. **Expected behavior** A managed shell operation obtains credentials for the current run. A later operation uses the new identity after rotation and loses access after revocation. Ambient host credentials do not supply authority. **Steps to reproduce** 1. Start the native GitHub broker with an active run binding. 2. Pass its environment through native OpenCode startup. 3. Run `gh api user` from the provider shell. 4. Observe anonymous access before this fix. **Paperclip version or commit** Reproduced on master `6943904bad` before this change. **Deployment mode** Native Paperclip Runner with OpenCode, including remote execution. Related: #16401 fixes the separate oversized OpenCode SSE failure and is already merged. The same live-driver regression was verified locally against its exact pre-fix source and current source. This PR does not duplicate that fix. Searches found no open PR for this native broker propagation defect; #15309 concerns the legacy OpenCode adapter. ## What Changed - Use the existing GitHub credential environment projection at both OpenCode launch boundaries. - Preserve explicit controller authority while excluding an ambient host broker. - Exercise the real launcher and broker from an OpenCode child process, directly and through runner projection. - Check operation-time identity rotation, revocation, and exclusion of controller API credentials and persistent GitHub tokens. - Document the managed OpenCode shell contract. ## Verification - Red: both new `native-github-access.test.ts` cases returned `anonymous` instead of `fixture-a` before production changes. - Green: both cases pass after the fix, including `fixture-a` to `fixture-b` rotation and anonymous operation after revocation. - Full native GitHub broker suite: 18 tests pass. - OpenCode, provider-event, and configured-environment suites: 141 tests pass across eight files. - Separate SSE red/green verification: the existing live-driver regression times out on `fd057f6ac4^` and passes with merged #16401. The full OpenCode group also passes after restoration. - Full repository build and typecheck pass locally. All CI jobs pass for `23d734293f6a5c6d6128cae0c8773661816daf9a`. Final GitHub rollup: 55 successful checks/statuses, two skipped checks, and one neutral check; none pending or failing. Greptile gave that commit 5/5 with no findings or unresolved threads. - Paperclip Review spent over 23 minutes preparing its runtime without a provider assessment. Its run was cancelled for recovery and ended neutral (review execution did not complete). This neutral result is not a code assessment. No check policy or branch protection was changed. - The full local Vitest attempt first hit embedded PostgreSQL startup failures. Running the unchanged `initdb --version` and `postgres --version` independently showed missing native library symlinks. The dependency had been installed with scripts disabled. Running its bundled `hydrate-symlinks.js` restored startup; all 15 affected native-session cases and all 112 AI connection tests then passed without application code changes. The original full-suite attempt was stopped after the setup failure and its log is retained. A complete local suite pass is not claimed. - The broker and executable are local fixtures. No live GitHub account, credential change, or test PR is required by these regressions. ## Risks The change affects credential delivery into OpenCode processes. It reuses the existing bounded GitHub projection and accepts broker authority only from explicit input. Each shell operation still uses the normal broker, scope, identity, and revocation rules. Unrelated controller secrets remain excluded. There are no database or permission-policy changes. ## Model Used OpenAI `gpt-6.1-sol`, through Codex, with xhigh reasoning and repository tools/code execution. The session does not report its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the focused regression and affected suites locally and they pass; the full local attempt and setup failure are disclosed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (Paperclip Review is neutral/incomplete, as disclosed above) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 16:15:38 -05:00
Devin FoleyandPaperclip	0b62c736a0	feat(connections): add generic Daytona API and Toolbox access (#16483) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps connections give agents controlled access to external services. > - Daytona provides sandbox, file, command, Git, and account APIs. > - Operators need to assign this access without exposing a server credential. > - A small task-specific tool list cannot serve general Daytona work. > - This pull request adds the published HTTP actions through the existing connection gateway. > - Operators keep control of identity, agent access, and each action. ## Linked Issues or Issue Description **Problem or motivation** Paperclip has a Daytona sandbox provider for its own execution infrastructure. It lacks a normal Apps connection for agents to manage an operator's Daytona account. Reusing the server's provider credential would bypass normal connection ownership and action permissions. **Proposed solution** Add a Daytona connection with personal or shared vault credentials. Generate its action catalog from the official platform and Toolbox API specifications. Use the existing run-bound gateway, grants, action policies, approvals, revocation, and audit paths. Sandbox IDs come from Daytona; tools need no Paperclip task or lease ID. **Alternatives considered** A four-action diagnostic subset cannot support general provider work. Launching the provider CLI would expose host filesystem behavior. Native OAuth is not included because a supported third-party registration path was not verified. The native method uses an organization API key. Existing Composio setup is a separate possible OAuth route and is not qualified by this change. **Roadmap alignment** This extends the existing Apps and Connections provider coverage in `ROADMAP.md`. It uses that system's credential and permission model. GitHub searches found no existing Daytona Apps connector PR or issue. ## What Changed - Add the Daytona catalog entry, provider source override, official theme logos, permission review, and setup documentation. - Generate 274 HTTP actions: 169 platform and 105 Toolbox operations. Record two WebSocket upgrades separately; do not advertise them as HTTP tools. Record source URLs and hashes. - Add provider-focused execution for sandboxes, snapshots, volumes, files, Git, commands, computer use, and account resources. Provider authority still limits which endpoints accept an organization key. - Validate action arguments. Resolve Toolbox targets through the fixed platform API. Check sandbox identity and organization before using an approved Daytona proxy. Keep normal guarded network requests and vault resolution. - Transfer file bytes without reading or writing host files. Share one caller deadline across prerequisite reads, dispatch, and response reading. Bound requests and responses. Do not automatically retry writes with unknown outcomes. Redact the selected credential from results and errors. - Classify credential access, access changes, interactive input, command execution, file permissions, and deletion (including TTL and auto-delete) as destructive actions. Keep new and changed actions under catalog review. - Upgrade saved catalogs and retire the earlier diagnostic names. Hide retired actions from the UI and agent discovery even when provider refresh fails. Preserve existing action choices and historical records. Reject stale same-name contracts during agent discovery and dispatch before a board refresh. - Add database-backed gateway and catalog-upgrade regressions, schema generator tests, a browser setup check, and both capability map updates. ## Verification - Passed on the review-fix head: 57 focused Vitest cases (18 Daytona database/client cases, five HTTP boundary cases, and 34 shared catalog cases), four generator tests, `pnpm -r typecheck`, and `pnpm build`. The five HTTP cases use a real local server through the pinned production transport. The normal browser setup check and 17 generator/artwork cases passed on the initial PR head; those UI and artwork paths are unchanged. - Passed on this branch: brand asset validation, token gates, map structure checks, and `git diff --check`. All latest-head GitHub checks pass on `ea979ccb3a64dc9508807823540f935c4316ea35`. Apex is 5/5, with all five findings addressed and zero unresolved threads. The branch has no merge conflicts. - The initial catalog-generation command could not find the external research corpus. The supported `--definitions-only` mode regenerated the definitions successfully. Unrelated generated drift was excluded. - An earlier full local Vitest attempt on the preview branch failed when bundled PostgreSQL could not start on macOS. After replay onto current master, all three unchanged event-sequencing cases pass with bundled PostgreSQL; the earlier startup cause remains uncertain. - The first full-suite command on this branch used my external test database override. Chat fixtures require a current migrated schema and failed because `agents.lifecycle_state` was absent in that disposable database. A focused comparison reproduces failure with the override and passes with normal setup. That attempt was stopped, and the full suite restarted without the override. Focused Daytona cases use their own migrated PostgreSQL 16 databases. - The clean local full-suite attempt recorded embedded PostgreSQL startup failures in two unchanged native-runtime cases and two failures in unchanged Git fixtures. It was stopped after these failures were isolated; no full local suite pass is claimed. The 15 matching native-runtime cases pass separately. The Git cases fail with the local `init.defaultBranch=main` setting and pass with `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master` limited to the test command. No global Git setting was changed. Full Linux CI is green. - Fixtures use real database persistence and run-bound gateway sessions with provider responses replaced by fixtures. They cover personal/shared vault ownership, reads and writes, file bytes, Off policies, revocation, organization mismatch, untrusted proxy rejection, output bounds, saved catalog upgrades, failed refresh, and permission preservation. - The first Apex review found five valid issues: multipart transport compatibility, destructive classification, caller deadlines, audit filters, and stale saved contracts. All five were fixed with regressions before the second review. A typecheck then found the nullable saved schema hash; admission now accepts that stored type and rejects null as stale. - The browser check covers setup controls without submitting a provider key. No real Daytona account, provider write, native OAuth flow, or agent model run is qualified. Live provider read/write proof remains pending a test account. ## Risks - Daytona keys have broad Toolbox authority, including keys with no selected resource scopes. Only assign this connection to trusted agents and review its action permissions. - Remote writes or commands can continue after a timeout. Calls do not automatically retry an uncertain outcome. - The bridge targets Daytona Cloud. Proxy hosts are restricted; an unrecognized future regional host fails closed. - Files and responses have an 8 MiB bound. Each action has one caller deadline, capped at 30 seconds across all provider requests and response reading. Interactive WebSocket sessions are not exposed as HTTP actions. - The generated JSON snapshot exceeds 1,000 lines because it holds the provider input contracts. Generator logic and execution code remain in separate small modules. Updates require explicit regeneration and catalog review. - No database migration or lockfile change is required. ## Model Used OpenAI Codex, GPT-6. The runtime identifies the model family but does not expose the exact backend model ID or context window. Used reasoning, repository inspection, code editing, shell tools, database verification, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 14:05:26 -07:00
DottaandPaperclip	61831ba7b8	refactor: defer adapter launcher imports during workspace snapshots (#16492) ... Defer adapter launcher imports until a directory merge resolves its lock root. Snapshot-only workers retain the same hashes, modes, exclusions, and ownership checks. Verification: 63 focused cases; full typecheck/build; exact-head full CI/E2E/security; Apex 5/5. Local native Daytona replies: 66.939s cold and 26.485s warm. Hosted staging and the cold latency target remain unverified. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 16:03:47 -05:00
DottaandPaperclip	6943904bad	perf: stage independent remote GitHub launchers concurrently (#16484) ... Stage at most three independent remote GitHub launcher files at once only when the existing effective concurrent-sync capability permits it. Preserve serial fallback, per-file locks and integrity checks, and settle every started upload before failure returns. Verification: 119 affected tests pass (one existing Linux-only oracle skipped on macOS); full repository typecheck/build pass; same-sandbox real-Daytona component comparison preserves all bytes/modes; current-head full CI/E2E/Superagent/Snyk and Apex 5/5 pass. The full native timing target remains separate and is not claimed by this change. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 15:36:43 -05:00
DottaandPaperclip	69dded3230	fix: allow normal Daytona command status response latency (#16480) ... Allow a normal completed Daytona status response up to 500ms within the unchanged caller deadline. Keep final separated logs, stream fallback and single dispatch. Verified: 396 provider cases, full repository and standalone provider typecheck/build, matched real-Daytona component controls, and exact-head full CI/E2E, security and Apex 5/5. Whole-agent latency target remains open. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 14:42:36 -05:00
DottaandPaperclip	c53bdd9eb3	perf: capture sparse checkpoint files in bounded batches (#16473) ... Capture sparse checkpoint files in bounded batches while retaining content and mutation checks. Prevent inserts after SQLite automatic rollback and drain active readers before database close. Verified: 209 affected cases, final six capture controls, repository typecheck/build, real Daytona helper validation, and exact-head full CI/E2E, security and Apex 5/5. The matched component median saves 1.05s; the whole-agent latency target remains open. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 14:32:44 -05:00
DottaandPaperclip	b1566ca494	perf(daytona): avoid log-follow delay for completed session commands (#16468) ... A completed Daytona session command now reads final separated logs without waiting for a new log-follow socket. Running commands retain streaming, recovery, and the existing total deadline. Verified with full repository typecheck/build, standalone provider typecheck/build, 395 passing provider cases, real immutable-image Daytona command controls, and fresh exact-head full CI/E2E and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 13:35:10 -05:00
DottaandPaperclip	68a09dcb5f	perf(ci): shorten the test execution critical path (#16434) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors wait for the full PR workflow before they can deliver fixes. > - Server shards still execute independent files in series, and a single Runner file serializes two long real-clock waits. > - Faster work outside the critical path does not prove a faster workflow. > - Overlap independent files while preserving isolation, test cases and real production deadlines. > - Compare complete baseline and candidate workflows, separating queue delay from execution cost. ## Linked Issues or Issue Description Refs #16429. Related duration refresh: #12577. **What existing behavior does this improve?** The elapsed time from PR CI creation to the last required check. **Current behavior** General server shards use one worker. One Runner transport file takes about 333 seconds on Linux, including over 100 seconds in two real-clock regressions. These are the longest verification lanes after #16429. **Proposed behavior** Run independent general-server files in at most two isolated forks. Move the two intact Runner deadline regressions into a separate file to overlap their waits in the existing pool. Run two chat partitions per existing job, each with its own temporary home and embedded PostgreSQL. Preserve serial execution within files and every assertion and timing boundary. ## What Changed - Qualify both serial and parallel forks with real worker processes, isolated homes/configuration, timers, globals and environment. - Require both parallel workers to arrive before either exits. - Use two isolated forks for sharded general-server tests; retain one worker with an explicit shared test database. Other test groups keep their existing concurrency. - Extract the lease-expiry and Pi cold-admission regressions verbatim into a separate Runner test file; retain real waits and process/lease assertions. - Split each sharded chat job into two independent invocations; preserve serial shared-database execution. Await both children before failing the parent. Prove disjoint coverage and failure propagation. - Update the paired Linux benchmark to compare a complete general-server shard with identical case inventories. - Document concurrency and deadline-test placement. No production behavior, workflow topology or runner count changes. - Allow 15 seconds for snapshot integration cases that cold-start multiple real TypeScript workers. On the repeat, one-worker cases took 2.5–3 seconds, so queued or sequential starts could exceed the default five seconds. Preserve all cleanup, queue-release and filesystem assertions; these cases do not enforce a production latency deadline. ## Verification All 48 jobs passed on scheduling head `5d718e18` in [attempt 1](https://github.com/paperclipai/paperclip/actions/runs/38763885288/attempts/1). The [bounded repeat](https://github.com/paperclipai/paperclip/actions/runs/38763885288/attempts/2) failed two existing snapshot-worker tests at their default five-second timeout. Its shorter duration is **not** counted as a successful performance sample. The latest head `3c894462` adds a targeted test-budget repair and [passes all 48 CI jobs](https://github.com/paperclipai/paperclip/actions/runs/38768433532) in **8m06s** (workflow creation to final required workflow job). This is **70 seconds / 12.6% shorter** than the 9m16s intermediate run, but the earlier 9m23s result remains part of the evidence: a stable average improvement is not established. Both branch-protection checks (`ci / verify` and `ci / e2e`) pass. The non-required Superagent Security Scan is still running; external PR scans are separate from this workflow duration. | Measurement | Before | After | What it establishes | | --- | ---: | ---: | --- | | General-server shard, paired on one Linux host | 430.94s | 292.99s | 32.0% faster; identical 1,047 case inventory | | Chat job 1 | 460s | 211s | Same full chat inventory spread across isolated partitions | | Chat job 2 | 408s | 274s | Serial cases within each child, separate PostgreSQL instances | | Chat job 3 | 408s | 216s | Total chat runner time fell from 1,276s to 701s | | Full PR workflow, first chat candidate | 9m16s | 9m23s | Not an end-to-end win in this pair: unchanged Build rose from 365s to 465s | | Full PR workflow, final repaired head | 9m16s | 8m06s | All 48 jobs pass; 70 seconds faster observed, not a controlled causal estimate | - Latest [paired benchmark](https://github.com/paperclipai/paperclip/actions/runs/38763885223) passes. Earlier paired server measurements were 33.3% and 37.4% faster. Unchanged fixture timing noise is not attributed to this PR. - All **1,337 chat case identities and outcomes match exactly** across [before-chat](https://github.com/paperclipai/paperclip/actions/runs/38760937988) and [after-chat](https://github.com/paperclipai/paperclip/actions/runs/38763885288/attempts/1), without duplicates or removals. Whole-suite records rose from 44,563 to 44,574: three new launcher checks and eight upstream warm-owner tests. Remaining name differences were generated fixture IDs/timestamps/PIDs, not removed tests. - The final run used `m7i-flex.xlarge` for Build (408s), with all chat jobs completing in 238s / 318s / 280s. It used the same actual merge base `afe5b341` as the first chat candidate. Optional profiling was disabled after the bounded measurement experiment; the full 48-job correctness workflow still ran. - The fleet used `m7i.xlarge` for the earlier Build and `m6a.xlarge` for the first chat candidate Build. Several unchanged compile/install/probe phases slowed together. This does not prove hardware alone caused the difference, and full-run attribution must account for runner/cache variance and the upstream base change. - Original full baseline: [all 48 jobs passed](https://github.com/paperclipai/paperclip/actions/runs/38753547876/attempts/2), 12m07s, including a server job that queued over five minutes. It is not a clean basis for attributing the whole elapsed-time reduction to this PR. An interrupted baseline and two misrouted candidate attempts were excluded and preserved; routing/trust rules were not weakened. - Local: 32 sharding/launcher checks pass, including disjoint coverage, shared-database fallback and a failed child that waits for its sibling. Real paired chat partitions passed 416 cases in 217.63s on independent embedded databases. - Both extracted Runner deadline regressions pass unchanged with real waits. An initial local attempt rejected a Node executable under the controller home; the identical binary in an owned non-home path passed without weakening that guard. - Real worker-isolation proof, actionlint and diff checks pass. Latest-head Greptile is **5/5**, with no review threads or actionable findings. - The eight snapshot-worker cases pass locally with a private `PAPERCLIP_HOME`. The first direct local invocation failed to open the default-home lock database under the sandbox; no production guard was changed. ## Risks Parallel files and paired chat databases may expose shared-state assumptions or increase peak memory. Cases stay serial within each invocation, and explicit shared databases keep the serial path. Complete server coverage and the full 48-job CI workflow pass. Check external required PR scans before merge; the full-workflow measurements are observational and do not establish a steady 12.6% speedup. Queueing, cache state and runner variance must be distinguished from execution improvements. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code execution and GitHub tools. The exact serving model suffix and context capacity are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (48 jobs; both required branch-protection checks pass; non-required external security scan pending) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 12:22:59 -05:00
DottaandPaperclip	25b16e65fc	fix(observability): detail workspace and connector preparation (#16454) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server prepares a repository, sandbox workspace, and connector access before an agent can reply. > - Existing traces measure large preparation phases but leave important work inside those phases unclear. > - Repository source materialization and Browser Use discovery can do more work than their callers suggest. > - New detail must preserve access checks, run ownership, and the existing work order. > - This pull request adds fixed OpenTelemetry spans inside those operations. > - Operators can use one deployed trace to select the next performance fix. ## Linked Issues or Issue Description Refs: #16419, #16425, #16428. **What existing behavior does this improve?** Startup traces for server workspace preparation, native sandbox synchronization, and connector assignment resolution. **Current behavior** Workspace and connector aggregate spans can take many seconds. The trace does not separate private source cloning, seed reuse, archive work, progress persistence, or each connector resolver. Browser Use resource selection builds the full authorized tool catalog. The configuration and native backend can each resolve connector assignments. **Proposed behavior** Show the awaited work as fixed child spans. Separate discovery queue wait from catalog construction. Preserve the initiating run context when another run releases a scheduler slot. Add no new lifecycle event writes for workspace detail callbacks. **Reason and benefit** Operators need evidence about the hosted critical path before another performance change. These spans distinguish filesystem work, provider transfer, authorization, transaction entry, locks, and reporting overhead. **Breaking changes** None. Authorization, query order, transaction ownership, synchronization, existing progress events, and task results retain their contracts. The change adds opt-in Observability spans under the existing OTLP endpoint gate. ## What Changed - Trace private source directory setup, Git credentials, clone, requested ref, source identity, and checkout publication. - Trace sandbox source capture, seed cache/replay, shallow clone and archive, overlay copy/archive, verification, upload, warm stamps, and descriptor persistence. - Separate existing awaited progress reporting from useful workspace work. Keep new detail callbacks trace-only. - Trace each connector resolver and Browser Use catalog/grant work. Split discovery queueing, descriptor sources, and policy evaluation. - Trace Slack authority, credential reads, catalog synchronization, transaction entry/settlement, and advisory locks. - Update the real filesystem staging trace expectations and the Honeycomb investigation guide. - Add a real scheduler regression for a queued job released by another run. Assert run ownership, parent spans, and exactly one completed queue span. ## Verification - Passed: server and adapter-utils TypeScript checks (`pnpm --filter @paperclipai/server exec tsc --noEmit` and `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`). - Passed: 252 checks across 9 focused files. These cover real filesystem staging, native workspace synchronization, trace ownership, endpoint gating, and workspace preparation. - Passed after review: 26 trace-context, tracing-failure, and discovery-scheduler checks across 3 files (these overlap the first run). The new queued-discovery regression fails when context binding is removed and passes after restoration. An initial submillisecond timestamp assertion was replaced with explicit span-completion order to avoid SDK clock precision assumptions. - Passed on implementation commit `0e68968a4bd61001893d833f281f12bb110edbe1`: `pnpm -r typecheck` and `pnpm build`. The later commit updates only an engine test expectation. - The first CI workspace lane found an old engine trace-tree expectation. It expected the earlier span set. The updated assertion now checks the new detail spans, their parents, and single completion. Its targeted executor check passes. - The full test suite, build, release registry, and end-to-end gates pass in CI on final head `40e37d89ef679109d60a96bb5731a0616efeaa43`: 55 passing checks and 2 optional Storybook skips. The full suite was not duplicated locally after the focused run. - Greptile reviewed the final head at 5/5. No unresolved review threads remain. - After deployment, filter Honeycomb by the hashed run ID and verify `service.version`. Inspect `heartbeat.prepare.workspace`, `runtime_selection.workspace_sync`, and the connector assignment parents. The guide in `doc/observability.md` explains each new family and the nested intervals. - No hosted run has used this new instrumentation yet. Local checks cannot establish hosted latency improvement. ## Risks - Tracing adds spans when an operator configures an OTLP endpoint. Detail remains capped at 16 spans per phase per execution attempt. - Parent and child durations overlap. The documentation explains how to avoid counting them twice. - A discovery job can start in another run's scheduler continuation. The bound callback preserves the initiating async context. - New spans export fixed names and the existing hashed run ID. They add no paths, repository URLs, connector resource IDs, credentials, tool schemas, or exception text. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 12:21:32 -05:00
Leia MarshallandLeia Marshall	e4ce989099	Add opt-in Gmail archive capability (#16331) ... ## Thinking Path > - Paperclip separates Gmail capabilities by reviewed OAuth scope and tool policy. > - Existing Gmail connections support reading and draft creation but not bounded archiving. > - Gmail archive requires `gmail.modify`, so it must not widen existing read or draft grants. > - This change adds an opt-in archive profile and limits it to removing only the `INBOX` label. > - A final pre-dispatch check preserves that boundary for approved and replayed calls. ## Linked Issues or Issue Description **What existing behavior does this improve?** This improves the existing Gmail connection and tool-governance behavior. **Subsystem affected** Cross-cutting: `packages/shared` and `server`. **Current behavior** Paperclip can read Gmail and create drafts. It does not provide an opt-in, bounded action for archiving a thread. **Proposed behavior** Paperclip offers a separate Read & archive connection. It exposes `unlabel_thread` only for that connection and accepts only `threadId` with `labelIds: ["INBOX"]`. **Reason and benefit** Users can archive processed Gmail threads while sending, trashing, and other label changes remain disabled. **Breaking changes** None. Existing read and draft methods keep their current scopes and defaults. Related work: #15545 adds a unified Google Workspace connector and keeps Gmail draft-only. #15412 adds a separate Gmail REST server without label mutations. Neither PR adds this bounded archive capability. ## What Changed - Added the `gmail.archive` connector profile. - Added managed and customer-owned archive connection methods. - Added an Inbox-only schema and final pre-dispatch argument guard. - Classified archive as destructive and added Ask-first defaults for the archive methods. - Added permission-review records and focused tests. ## Verification - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/shared typecheck` - Focused Vitest suites: 42 tests passed. - App-definition ingestion: 103 definitions and 0 review flags. - `git diff --check` ## Risks The main risk is accidental scope or operation expansion. The new profile is opt-in. Existing profiles retain their scopes. The gateway validates the exact Inbox-only payload immediately before dispatch. Production rollout requires separate OAuth configuration and fresh consent. ## Model Used Unverified — reliable runtime model metadata was not available. Confirm the model attribution before publishing. ## Checklist - [x] I have included a concise design rationale - [x] I have specified that model attribution is unverified - [x] I have checked ROADMAP.md and found no Gmail archive entry - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR - [x] I have not referenced internal or instance-local issues - [x] My branch name describes the change - [x] Local tests and builds pass - [x] I have added or updated tests - [x] I have updated relevant documentation and permission records - [x] I have documented risks - [ ] All hosted CI gates are green - [ ] Greptile and human review are complete --------- Co-authored-by: Leia Marshall <leia@paperclip.ing>	2026-10-15 09:24:31 -07:00
DottaandPaperclip	cccde7122b	Trace execution preparation and durable response persistence (#16419) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server prepares an execution environment before it starts an agent. > - Operators need to distinguish preparation, provider wait, and response persistence. > - Existing preparation phases start separate traces, and the native session trace ends before the final response is saved. > - This pull request adds an execution-attempt trace and detailed spans at the existing work boundaries. > - The benefit is a measurable startup and settlement path without changing execution policy. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operator-configured OpenTelemetry tracing for agent preparation, startup, and durable response storage. **Subsystem affected** Server orchestration, native Runner startup, and the Honeycomb run link. **Current behavior** Preparation phases start independent roots. Workspace lock waits, authorization groups, runtime asset work, provider-pack fallback, and the gap after provider completion are hard to attribute. **Proposed behavior** Use a server-owned async context for each execution attempt. Parent nested work to that attempt. Link the separately sampled native session because its ingress and queue history can begin earlier. Record transaction entry, lock waits, grouped authorization, asset preparation, artifact fallback, copyback, and response commit. **Reason and benefit** Operators can identify expensive work from recorded boundaries instead of inferring the cause from elapsed time. Nested and concurrent durations remain visible and must not be added together. **Breaking changes** No API or execution-policy change. New Observability span names and closed count attributes are documented. The existing local run-performance schema also records the additional provider-pack and harness measurements. Refs: #16178, #16205, #16330. ## What Changed - Add `task.execution_attempt` with same-run async ownership, independent root sampling, and native-session links. - Measure workspace transaction entry and settlement, advisory and row waits, source scans, reader groups, notifications, and writes. - Measure runtime selection and each concurrent instructions, skills, and MCP branch. Separate manifest, cache, materialization, and verification work. - Measure provider-pack discovery, verified reuse or upload, final verification, and remote harness state checks. - Measure workspace settlement, accounting, liveness, response authorization, comment commit, and publication enqueue. - Preserve the silent endpoint-off path, including when another library has registered a tracer. Bound repeated detail spans to 16 per name per attempt. - Extend the Honeycomb run link and document the actual coverage and remaining gaps. ## Verification - After resolving the conflict with retained-reader batching on `b98cdaea531c8447637cd4dcc19c61b3f5f41844`, all 61 targeted query, authorization, workspace-preparation and trace-boundary tests passed. The server typecheck passed. Reader tests still bound query count, avoid loading retained run payloads and enforce fresh project grants on every publication attempt. - Before that base integration, the tracing/exporter group passed 855 tests with 2 optional-SDK skips across 10 files and no inherited OTLP endpoint. It covers real PostgreSQL waits/rollback, durable response commit, assembled heartbeat preparation, concurrent parentage, sampling and tracer failure isolation. Native workspace/history tests passed 23 cases. These results remain supporting evidence for the unchanged tracing paths. - Full workspace typecheck and build passed during implementation. Full sharded CI, release validation, E2E and Apex 5/5 passed on prior head `8cebad80f3856dbcff6f3a6d5d34880ff77a20f2`. Fresh CI, security and Apex review are required on the current head before merge. - `pnpm check:token-gates` passed. No styles changed. - Earlier CI runs exposed exact-source-expression assertions changed by wrappers, missing run hashes on transaction-boundary spans, and sampling fixtures without an enabled endpoint. The fixes preserve the calls, add the declared hash and span cap, and enable tracing explicitly in the sampling fixtures. Focused repairs passed; the full prior-head CI passed after those repairs. Earlier failed run: https://github.com/paperclipai/paperclip/actions/runs/38729131366. - The full local `pnpm test:run` was stopped after the base advanced during that process. Its mixed-version results are not final-head evidence. A full local-suite pass is not claimed. - The old-head Superagent scan stalled at 35%. The current head has a fresh scan; no security pass is inferred from the old pending result. - No deployed Honeycomb acceptance is claimed by this code change. ## Risks - Enabled tracing creates more spans. Repeated detail spans are bounded, and inventories use group spans. - The attempt and native session remain linked traces. Historical queue time can precede the attempt, and nested intervals overlap. - A committed comment proves durable server storage. Publication enqueue and realtime dispatch do not prove external delivery or browser paint. - New custom attributes use only the existing run hash and validated file/byte counts. No SQL, paths, permissions, content, or exception text is added. ## Model Used OpenAI GPT-6 through Codex. The serving model ID and context window size are not exposed in this session. The agent used reasoning, tools, code editing, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 08:13:53 -05:00
DottaandPaperclip	6acec4266e	feat(ci): profile full CODEOWNER PR suites in Honeycomb (#16378) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its PR workflow runs dozens of parallel build and test jobs. > - Individual job durations do not show the elapsed cost of the whole suite. > - Maintainers need one shared timeline to find slow work and repeated setup. > - This pull request adds full-suite profiles for selected CI CODEOWNER PRs. > - Honeycomb shows each job, step, test file, and supported test on that timeline. > - The benefit is evidence for changes that reduce total CI time. ## Linked Issues or Issue Description **What existing behavior does this improve?** The main PR workflow has about 48 jobs. Maintainers need to compare their overlapping work and find which jobs delay completion. **Current behavior** GitHub shows job and step durations separately. Test files and tests are not joined to a full workflow trace. **Proposed behavior** A CI CODEOWNER adds `profile-ci` to a PR. The tested code stays on that PR branch. All main PR lanes run, including lanes normally skipped for stack position or release notes. A trusted post-run exporter sends one wall-clock trace to Honeycomb. **Reason and benefit** The trace supports decisions about which CI work to speed up or remove. Ordinary PRs do not export traces. The roadmap and related-PR search found no duplicate CI profiling work. ## What Changed - Add label selection and a full-suite override. Verify the PR author, original actor, and rerun actors against the trusted `.github/**` CODEOWNERS rule. - Record Vitest, Playwright, and Node test file/test timings in local artifacts. Keep ordinary reporter behavior and failing exit codes. - Add a trusted `workflow_run` exporter. Use actual GitHub timestamps for all jobs, setup, steps, waiting, and teardown. Mark missing test data and partial reruns. - Keep the ingest key out of PR jobs. Store it only in the `ci-profiling` environment, restricted to `master`. The exporter reads bounded JSON and never executes artifact code. - Record expected test invocations outside reporters so a missing invocation cannot appear complete. Preserve `node --test` on older PR checkouts. Flush large shard inventories before exit. - Add security, timing, routing, and real-reporter checks. Document operation and activation. Protect the exporter with CODEOWNERS. ## Verification - 509 focused workflow/reporter/shard checks passed. They include outside authors, outside rerun actors, bots, stale revisions, malicious ZIP paths, incomplete reports, and partial ingestion failures. - Final revision `3bf5d9017d` adds multi-file qualification and fixes short-file step parenting and duplicate Node file-wrapper events. Greptile reviewed this exact revision at 5/5 with no unresolved threads. [Main PR CI run](https://github.com/paperclipai/paperclip/actions/runs/38728040302): the initial attempt had 54 successful checks, 3 intentional skips, and one serialized-shard failure from an unchanged workspace-authorization test exceeding its 15-second timeout. The single failed-shard retry passed: the final result is 55 successful checks and 3 intentional skips. The same timeout reproduces locally with profiling disabled; both assertions pass with a diagnostic 60-second timeout override (about 23 seconds total). No source or CI timeout was changed. - A dry run against real GitHub attempt 2 of run `38677445871` produced exactly 3 executed jobs, 40 spans, and 510 seconds, excluding 45 carried-over jobs. The partial trace remains explicitly incomplete; this validation did not upload data. - `actionlint` passed for all four changed workflows. `git diff --check` passed. - Full Node 24 `pnpm -r typecheck` and `pnpm build` passed. The local full `pnpm test:run` was stopped after it reported a setup-token timeout and a distribution-plugin-catalog fixture failure. The 30 setup-token tests passed on focused rerun. The plugin fixture fails at `rmSync` of a directory symlink on this macOS/Node 24.11.0 host; the same failure reproduces with the unchanged base Vitest configuration and unchanged test/service files (10 other tests pass). The local full test suite is therefore not claimed green. - A real root-config Vitest probe passed 13 tests with profiling both off and on. Multi-file Vitest, Playwright, and Node probes each verify two distinct filename rows and four correctly parented tests, including repeated test names across files. Separate passing probes exercise the full exporter hierarchy `workflow → job → step → filename → tests`. Intentional failing probes preserve failing exit status. - Verified all 48 measured jobs in latest-head run `38728040302` used AWS: one `paperclip-ci-routing-x64` gate and 47 `paperclip-public-pr-x64` jobs. Profiling preserves AWS authorization and its GitHub-hosted fallback. The separate privileged exporter runs on `ubuntu-latest` outside the measured suite. - [Live Honeycomb baseline](https://ui.honeycomb.io/paperclip/environments/test/datasets/paperclip-ci/trace/oaPMzw8zGwG): 48 jobs, 731 spans, 479 seconds. It matches [GitHub run 38671274294](https://github.com/paperclipai/paperclip/actions/runs/38671274294). This historical trace has job/step data only; it is not proof of the future label-triggered test-reporting journey. - Verified the dedicated Honeycomb key is ingest-only with dataset creation disabled. Verified the GitHub environment allows only the `master` branch. No repository-level secret of that name exists. Both credential values were scanned absent from source and branch history. After merge, add `profile-ci` to a CI CODEOWNER PR, wait for the entire PR run and exporter, then follow the run-summary link. Collapse job children to compare parallel lanes; expand a job to inspect its steps and tests. ## Risks - Activation requires merging the trusted workflow definitions to `master`. This PR continues using the current trusted reusable workflows until then. - Selected runs produce many spans and artifacts. Oversized artifacts fail export instead of truncating data. Missing or cancelled reporters are marked incomplete. - GitHub timings have second precision. Node test times use reporter receipt time. Rust retains job/step timing but lacks per-test wall timestamps. - CI CODEOWNERS are individual users in the explicit `.github/**` rule. Unsupported team rules and identity lookup failures disable profiling. - An interrupted upload can leave partial data. Retrying uses the same span IDs, but the Events API does not promise deduplication. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, repository tools, and browser verification. The runtime does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 06:59:50 -05:00
DottaandPaperclip	efcc8a7d89	test: bound command snapshot forwarding checks (#16421) ... Keep the command snapshot option test focused on forwarding and real file copyback. Use inline delegates in that test, and retain the separate real-worker suites and original deadlines. Validation: 124 affected tests, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. Negative controls reject lost command and restore forwarding. Failed earlier merge-readiness evidence remains in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 06:57:38 -05:00
DottaandPaperclip	7fc2dc8b32	fix: isolate workspace snapshots from controller work (#16414) ... Move native sandbox snapshot captures off the controller callback queue while preserving the existing capture and restore contract. Keep legacy adapters inline, bound captures to two workers, and verify cleanup and reordered worker completion. Validation: 461 affected cases, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. The full local canonical suite remains running; no full local pass or under-30-second staging response is claimed. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 06:31:59 -05:00
Devin Foley	8365872743	fix(codex-local): retry ACP model capacity failures as transient upstream (#16407) ... Classify typed Codex ACP `service` failures about model capacity or server overload as `codex_transient_upstream` / `transient_upstream`, so the existing bounded transient retry and continuation recovery apply. In `unexpected` Sentry mode, skip only runs carrying that full classified evidence; `all` mode still reports them.	2026-10-15 04:28:02 -07:00
3 people	fd057f6ac4	fix(runner): keep OpenCode turns alive through oversized SSE events ... Retain bounded OpenCode SSE events without restarting the event stream. Preserve canonical tool identity, status, output tails, original UTF-8 sizes, and truthful truncation markers. Bound id-less event deduplication with hashes. Cover fragmented and complete large frames, exact-limit split separators, bulk input retention pressure, truncation-record exhaustion, and unpaired surrogate byte counts. Full current-head CI and Apex 5/5 passed. The unrelated local post-tool answer assertion was reproduced on unchanged baseline source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 01:02:28 -07:00
DottaandPaperclip	074c67fea8	fix(grok): avoid routine wakes for accepted work (#16394) ... ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - Grok Bot receives assignments through routine webhooks and uses agent MCP tools. > - Each accepted assignment belongs to the current external interaction. > - A pending tool result or comment could start another routine while that work was still active. > - Quiet MCP traffic cannot prove that Grok has stopped working. > - This pull request keeps receipts and feedback durable for the current interaction to read. > - New task routines start only for offers that the Bot has not accepted. ## Linked Issues or Issue Description Refs #16219 and #16305. The duplicate search also found #12991, which covers instruction files for local Grok runs. **What happened?** The dispatcher could start another routine for an unread result after a period without a routine reply. It could also wake accepted work for a task comment. A queued delivery could still start after the Bot had claimed the offer. **Expected behavior** Accepted work stays in its current interaction. The Bot polls the original operation receipt and reads queued feedback at a safe boundary. Acknowledgements of a sent wake remain valid after acceptance. **Steps to reproduce** 1. Accept a Grok assignment and leave it active. 2. Add an unread comment, or complete a tool operation whose call returned pending. 3. Run the dispatcher after its quiet period. 4. Observe another webhook call. Also queue an offer wake, accept the offer before sending, and observe that delivery still starts. **Paperclip version or commit** Observed during real-client qualification on Core ff0f976de1. The three new regression cases also failed on current master before the correction. ## What Changed - Stop automatic result and feedback wakes for accepted work. Keep both available through their existing read paths. - Require an unaccepted offer before dispatch or an explicit delivery retry. Serialize the final HTTP handoff with acceptance receipts. Cancel obsolete unsent wakes. Preserve late acknowledgements and immediate Stop authority. - Return current Grok polling instructions on assignment reads. Clarify pending and unknown outcomes in setup guidance. - Add protocol, receipt-polling, queued-delivery and late-acknowledgement regressions. Update the operation guide and readiness plan. ## Verification - Three new regression cases fail before the fix because a webhook starts. - The latest 31 Grok protocol checks and 47 existing Dot Runner checks pass. New tests force acceptance during secret lookup and the final send check. They also prove that acknowledgements and Stop remain free during pending delivery. - The final-head Grok, Dot, MCP and question-delivery suite passes all 212 checks across six files. Dedicated onboarding UI checks passed six. - Dedicated Runner driver checks use the real local Rust daemon. Built the debug daemon required by that package; all 17 checks then passed. - Repository typecheck and build pass. Token gates, the 2,731-story structural catalog and the Storybook build pass. - Public synthetic marker for the post-deployment native source-read check: `grok-native-read-b45be4eff22cb15a31d0af512f437b8d`. - Final-head CI passes all 55 checks, with two optional Storybook skips. Greptile is 5/5 with no open threads. The first chat-shard attempt failed one Telegram SQL-quiet timing assertion; all five related cases pass locally and the single targeted CI rerun passes. - After deployment, repeat actual Grok work through an answered question, receipt polling, a saved report and file, canonical completion and cleanup. This correction is not production qualification by itself. ## Risks A Bot that does not poll a pending receipt can wait without another routine wake. Current assignment reads now supply polling instructions to existing connections. Task comments stay queued until the Bot reads them or Runner admits the next continuation. The real client must verify that behavior before production enablement. The observed second file version does not prove which provider interaction created it. Acceptance can wait for the bounded webhook response. Stop and authenticated acknowledgements use independent row locks and remain available. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository editing, shell tools and browser tools. The exact serving model ID, snapshot and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 00:54:15 -05:00
DottaandPaperclip	854ab294cc	fix: restore repository admission and Claude startup on Linux ARM64 (#16372) ... Batch retained workspace admission reads while preserving individual privacy and trust decisions. Preserve bounded saved-policy projections when integrating current master, and qualify the pinned Claude executable on Linux ARM64. Verified exact head 4d468f998b with full CI/E2E, Superagent, Snyk and Apex 5/5, with no unresolved findings. Local typecheck/build and 209 server plus 74 integrity cases pass; full local suite remains pending. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:33:09 -05:00
DottaandPaperclip	444d5854b0	test(adapter-utils): consolidate outbound restore span coverage (#16115) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:23:28 -05:00
DottaandPaperclip	86816ff965	test(adapters): consolidate Grok stale-auth promotion coverage (#16112) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:09:21 -05:00
DottaandPaperclip	982ff62949	test(shared): consolidate ZIP conversion observations (#16118) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:06:50 -05:00
DottaandPaperclip	5e62927a8f	test(adapters): consolidate Codex staging permission guards (#16111) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:06:35 -05:00
DottaandPaperclip	662a20281d	test(cli): retire duplicate ZIP codec owner tests (#16116) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:57:43 -05:00
DottaandPaperclip	0de90e25bf	test(adapter-utils): share pre-READY work-bound bridge execution (#16098) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:56:07 -05:00
DottaandPaperclip	1ff08a53cb	test(tailscale): retire duplicate substituted-listener withdrawal (#16106) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:51:52 -05:00
DottaandPaperclip	3b6d1452f2	test(daytona): retire duplicate pack span execution (#16099) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:45:50 -05:00
DottaandPaperclip	3ea39f75c1	test: consolidate duplicate owner proofs (#16075) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:17:13 -05:00
DottaandPaperclip	9076329149	test: consolidate duplicate runtime credential proofs (#16079) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:13:45 -05:00
DottaandPaperclip	0e1bb3748b	test(db): retain owner nullability in the active-slot keeper (#16076) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:13:15 -05:00
DottaandPaperclip	f64d05adf4	test: consolidate repeated ACPX startup metadata scenarios (#16068) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:02:09 -05:00
DottaandPaperclip	422728fc36	test: retire seven repeated ACPX trace executions (#16061) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 22:50:25 -05:00
DottaandPaperclip	3dffd2197d	test: consolidate owner fixtures and retain real regression proof (#16047) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tests protect sandbox execution, run recovery, credentials and the board UI. > - Several owner suites repeat the same setup and prove the same result more than once. > - These copies add maintenance work and can hide weak assertions. > - Each cut needs a retained test that proves every distinct contract. > - This pull request consolidates repeated setup and removes twelve redundant executions. > - It also replaces two weak assertions with checks of real outputs. > - The benefit is 2,831 fewer test lines with clearer regression proof. ## Linked Issues or Issue Description **What existing behavior does this improve?** The maintenance and regression strength of the existing Vitest owner suites. **Subsystem affected** Sandbox adapter utilities, server heartbeat recovery and environment capabilities, and the board UI. **Current behavior** Tests copy database rows, query builders, render calls and runtime inputs. Some cases repeat an existing contract. One token test omits a real bridge credential. A PostHog host comparison can compare absent controls. **Proposed behavior** Use small local helpers with fresh values and the same action order. Keep each distinct contract in an owner test. Check real bridge credentials and fixed PostHog method states. Carry the exact persisted first-run success assertion into the stronger cancellation keeper. **Reason and benefit** The ten test files shrink by 2,831 net lines. Twelve redundant executions retire. All other case tables, inputs, security boundaries, timeouts and cleanup rules remain. **Breaking changes** None. This round changes tests and audit evidence only. **Additional context** Refs: #15978, the merged first cleanup. Refs: #16022, the merged prior cleanup. Its squash merge is `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Related open work: #14634 repairs feedback fixture completion; #14477 supplies explicit sandbox fixture tools; #14198 changes cleanup retry behavior in another heartbeat file. This audit preserves those separate action and cleanup contracts. It does not apply their changes. This PR was first stacked on `codex/test-audit-followup`. After #16022 was squash merged, only the two Round 6 commits were replayed onto merged master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. The original stack head is preserved in a local snapshot branch. The current diff contains only ten test files and two evidence files. No merge is requested. ## What Changed - Consolidate repeated setup in three sandbox adapter suites. Remove seven redundant executions and retain the real protocol, cleanup and scratch-file checks. Save 875 lines. - Consolidate heartbeat database fixtures and read queries. Retain exact wake cancellation, total run count and the actual first-run success receipt. Remove one duplicate execution. Save 1,372 lines. - Remove two repeated rejected-capability cases. Carry the absent snapshot assertion into the retained runner case. Save 22 lines. - Consolidate UI fixtures and render calls. Remove two repeated connection cases. Replace the weak PostHog comparison with fixed method and required-key checks in both real hosts. Save 562 lines. - Record audit scope, decision and expansion counts, all twelve keeper mappings and deliberate mutation results in the [audit report](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6.md) and [controls evidence](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6-controls.json). ## Verification Source baseline: `a090035cafb29c06f6fca2dbe43561ad866b7c1b`. All ten original test programs are byte-identical at the stacked base `58c5c4aa241c85f4665a3159c192888bbb2a5cfa`. - Baseline: 1,841/1,841 cases passed in the main thirty-file owner/sibling run. Another 219/219 cases passed in nine extra files. - Preservation: all four proposal packages passed independent review. Whole-program helper expansion retains ordered inputs, callbacks, awaits, queries, fresh allocations and hooks outside the declared assertion migrations and repairs. - Candidate: all 2,048 cases passed across 39 complete owner/sibling files. The main run passed 1,831 cases. The extra run passed 217 cases. Both runs had zero failures and zero skips. - PostHog control: the repaired positive case passed. The original weak case passed with authentication choices removed. The repaired case failed at its fixed OAuth aria-checked assertion under the same mutation. - Eight accepted mutations failed at their intended assertions. They cover bridge-token exclusion, fallback cleanup, READY bytes, real scratch-file preservation, exact wake cancellation, the persisted first-run success receipt, rejected capability snapshots and reachable PostHog method controls. Two original weak cases passed with the same defects. All production source bytes were restored. - Rejected control: changing run status before legacy release timed out before wake cancellation. This result is not counted as assertion proof. Its replacement changed only the persisted receipt after real drain in the existing transaction. It failed at the direct fixed succeeded-row assertion. - No new code comments or 1,000-line threshold crossings were introduced. No production, schema, lockfile or workflow changes are included. On the exact final test sources, later committed as `d8de9422f0703d4508a9dc880b67a07f13e0f110`, `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build` passed. The required full `pnpm test:run` completed with **exit 1** after **52.14 minutes** on clean, immutable `d8de9422f0703d4508a9dc880b67a07f13e0f110`. HEAD and tracked sources remained unchanged. The first general-server group ran 889 files: **882 passed, one failed and six skipped**. Its 18,032 cases were **17,925 passed, two failed and 105 skipped**. The only failing file was `server/src/__tests__/remove-test-home.test.ts`. On macOS, the read-only runtime bundle and symlink cleanup cases returned `ENOTEMPTY`. The permission-error case expects `EACCES` or `EPERM`. Four reported failure entries cover these two cases, including their cleanup errors. The test and helper are byte-identical to previous master `4b766b872993045a2e787404f0475503683e5ded` and audit master `e364b92c82`. The default wrapper stopped after that group failed. **Later local groups did not run.** The earlier public-MCP, HTTP log redaction and workspace snapshot failures did not recur. The 39 focused owner/sibling files and all 48 remote CI jobs completed independently. This result is an unresolved macOS cleanup limitation, not full local success. No test, skip, retry, timeout or cleanup rule was changed to hide it. The three adapter files received a whitespace tidy after focused execution. Full runtime AST and string/template/regex byte equality link those tested sources to the final sources. All 134 adapter helper calls were checked again. Replayed code commit `3d3defd2a60595a623045d9e2c6c586f4acb3452` has the exact final test bytes. Evidence commit `e868e6645ce0d598d756d3458f0b3d71afd6db4a` adds the report and controls JSON, with one historical ancestry clarification. All ten accepted test hashes and all seven production-owner source guards match. No package, lockfile, workspace, TypeScript, Rust or Vitest dependency manifest changed since the audit snapshot. The [original stack CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38312876910) passed all 48 jobs on head `c171d6c0d603d22bb997c1f9a9bdcd5bcc5d22aa`. Those results are historical stack proof. The [fresh master-context CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38318223856) completed successfully with **all 48 jobs passed** on current head `e868e6645ce0d598d756d3458f0b3d71afd6db4a`. Its tested merge commit was `41292af60c6eb60d477637d23822bb17b08dc07e`, combining workflow base `88bf4c6c1d222f818faa56257efe79207c892a6c` with that head. The delivery commits remain based on pinned master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Both required master checks, `ci / verify` and `ci / e2e`, passed. Two separate optional Storybook checks were skipped by their scope gate. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/16047#issuecomment-6133323165) gives the current head **5/5**, with no findings, inline comments or unresolved review threads. The [current-head commitperclip review](https://github.com/paperclipai/paperclip/actions/runs/38318223666) also completed successfully. This PR remains open and draft; no merge is requested. The local macOS failure above remains unresolved. ## Risks - A helper could share a value or capture the wrong binding. Independent expansion and lexical binding checks guard the actual helper calls. - A retired case could contain a distinct assertion. Each cut has an exact keeper map and carried observations. - A mutation could fail before the intended assertion. Controls reject timeouts and earlier failures. The rejected receipt mutation remains in the evidence. - The delivery ancestry changed after the parent squash merge. Exact source hashes preserve the audited tests and producer receipts. Fresh CI and review passed in the current master context; the broad local run remains evidence from its immutable audit snapshot. - The complete local default failed in two unchanged macOS cleanup cases. Later local groups were not run. This limitation remains visible even though focused checks and remote CI passed. ## Model Used OpenAI Codex, based on GPT-6, assisted with this work. The exact served model ID and context window are not exposed in this session. It used repository tools, code execution, AST analysis and parallel audit agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 21:43:12 -05:00
3 people	3935e300dd	fix(acpx): bound Claude MCP tool calls when runtime MCP servers attach (#16360) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted `claude_local` runs use the ACPX engine. Paperclip gives each run its runtime MCP servers (for example the `paperclip-assigned` tool gateway) as ACP `mcpServers` entries. > - Claude's default MCP tool timeout is far longer than a run. ACP `mcpServers` entries have no per-server timeout field. So the agent waits on an MCP call until the call answers. > - Hosted error reports show runs that end with "Run exceeded the adapter execution timeout (sandbox)". The stop-time diagnostics from #15962 show the problem: the agent went silent about 50 seconds into the run, three tool calls of ACP kind `other` stayed pending, and nothing ended them until the 4h sandbox backstop. > - The gateway bounds the remote provider call (10s default, 60s max). But other steps on the gateway `tools/call` path have no deadline: DB queries, row locks and pool acquisition, the credential steps before the provider timer starts, and the stdio child exit wait. One stuck step holds the agent turn for hours. > - This pull request gives the agent's MCP client a deadline: `MCP_TOOL_TIMEOUT` for ACP Claude runs that receive runtime MCP servers. > - The benefit is that a stalled gateway call becomes an MCP error that the model can see. The turn continues and does not spend the full 4h execution budget. ## Linked Issues or Issue Description - Refs #15962 (diagnostics that identified the silent turn with pending tools) - Refs #15448, #15456, #15850 (related gateway timeout and connection-health work; this PR does not overlap with them) **What happened?** A hosted `claude_local` run sent parallel MCP tool calls. No call answered. The run stayed silent for about 4 hours. Then the sandbox execution timeout ended it with "Run exceeded the adapter execution timeout (sandbox)". **Expected behavior** A stalled MCP call ends with an error after a bounded time. The agent sees the error and continues. **Steps to reproduce** 1. Start an ACP `claude_local` run that has runtime MCP servers. 2. Make the gateway `tools/call` request stall (for example, hold a lock that the request needs). 3. See that the run stays silent until the adapter execution timeout. **Paperclip version or commit** `master` at 976cdc7. ## What Changed - Add `ACPX_CLAUDE_MCP_TOOL_TIMEOUT_MS` (15 minutes) in `acpx-engine/constants.ts`. The value has wide headroom over the gateway's longest legitimate wait (60s provider calls, 2 minutes of approval preparation, and the 10-minute connection cap proposed in #15850). - The ACPX engine sets `MCP_TOOL_TIMEOUT` in the agent env for Claude when the run has runtime MCP servers. If the adapter `env` already sets `MCP_TOOL_TIMEOUT`, the engine keeps that value. Other agents and runs without MCP servers do not change. - Add an engine test for the bound, the no-server case, the user override, and a non-Claude agent. - Document the client-side deadline in `doc/MCP-RUNTIME-OPERATIONS.md`. ## Verification - `npx vitest run src/acpx-engine/` in `packages/adapter-utils`: 21 files, 423 tests pass. - The new test fails without the env assignment (`expected undefined to be '900000'`), and passes with it. - `npx tsc --noEmit -p packages/adapter-utils`: passes. - The env reaches Claude through the same per-session env path as `ANTHROPIC_MODEL`. `claude-agent-acp` merges this env into the Claude SDK env. ## Risks - A legitimate MCP call that takes longer than 15 minutes now fails. No gateway path is designed to wait that long. Operators can set `MCP_TOOL_TIMEOUT` in the adapter `env` to change the value. - The value also applies to other MCP servers in the same Claude session, for example servers in user settings. The same override is available. - Warm sessions get the env when the agent process starts again. The env is not part of the session fingerprint, so this change does not reset resumable sessions. - This PR does not fix the server-side step that stalls. It bounds the effect on the run. Server-side deadlines on the gateway `tools/call` path (DB lock and statement timeouts, a request-level deadline, SIGKILL escalation for stdio children) are possible follow-ups. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use and code execution. It read the repository, traced the gateway path, edited the code, and ran the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 19:01:42 -07:00
3 people	1b58ac6ec9	build(deps): bump smol-toml from 1.8.0 to 1.9.1 (#16020) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Codex runner must set trust for the exact startup project. > - smol-toml 1.9.1 returns tables with null prototypes. > - Object spread replaces those prototypes and causes strict validation to reject valid trust edits. > - This PR keeps the parsed tables and adds tests for the dependency update. > - Codex can start while unrelated settings and trust boundaries stay intact. ## Linked Issues or Issue Description Refs #16028. That merged PR contains the same table-preservation fix and tests for both prototype styles. This branch retains that work and adds more regression cases. Related dependency updates: #15293 and #15364. **What happened?** Eight startup trust tests fail after the update to smol-toml 1.9.1. The error is `codex_startup_trust_cannot_preserve_configuration`. **Expected behavior** Set trust for the exact startup project. Keep unrelated configuration bytes and sibling project trust unchanged. **Steps to reproduce** 1. Check out the original PR head, `90b133dfd963af4f19b4df11fdd6c11b4eeb2a2d`. 2. Install dependencies with `pnpm install --frozen-lockfile`. 3. In `packages/paperclip-runner`, run `pnpm exec vitest run src/drivers/codex/codex-startup-trust.test.ts`. **Paperclip version or commit** The verified head is `1a8e8ad8b2e1049ebb68595d85c92e3c0621559d`. **Deployment mode** Built from source on Linux. The affected code runs before Codex starts. ## What Changed - Keep Dependabot's update from smol-toml 1.8.0 to 1.9.1 in the runner and server. - Keep parsed project tables. Use `parse("")` to create missing tables with the parser's prototype. - Keep strict candidate validation and the upstream tests for both prototype styles. - Add tests for nested values, dates, times, special numbers, prototype-named keys, sibling trust, and repeated calls. - Preserve the original Dependabot commit and lockfile. The two follow-up commits change only the trust source and tests. ## Verification - All 28 startup trust cases pass at the verified head. Run the command above. - The focused strict TypeScript check passes. Use `tsc --ignoreConfig --noEmit --module nodenext --moduleResolution nodenext --target es2023 --types node --esModuleInterop --strict --skipLibCheck src/drivers/codex/codex-startup-trust.ts src/drivers/codex/codex-startup-trust.test.ts`. - The runner TypeScript build passes. The full local build stops because the test workspace has no Cargo executable. - The broader Codex suite passed 271 of 272 cases before the source conflict resolution. One unrelated fixture expects the runtime's workspace path to be absent. - The repository type check exited 137 in the DB package. The full local test command was stopped after resource exhaustion. - The final files merge cleanly with master `976cdc77ed564dc3e014621505890b647e554a7f`. `git diff --check` passes. - [CI for the verified head](https://github.com/paperclipai/paperclip/actions/runs/38664681165) passed. All 55 active checks passed; two Storybook checks were skipped. Snyk also passed. [Greptile cycle 3](https://github.com/paperclipai/paperclip/pull/16020#issuecomment-6182103775) scored this head 5/5 with no actionable findings and no open review threads. ## Risks - This update changes the parser used by the runner and server. CI must check both packages. - The trust edit must preserve unrelated TOML values and bytes. Regression tests cover this behavior. - Local resource and tool limits prevent a complete local validation result. The full remote CI run passed at the verified head. ## Model Used OpenAI GPT-6 through Codex helped with the compatibility fix, tests, and review. The runtime does not expose the exact model snapshot or context window size. The agent used repository tools, code execution, and GitHub tools. Dependabot produced the original dependency and lockfile update. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — the focused 28-case suite and strict type check pass; broader limits are listed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — this PR records the compatibility behavior and verification - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- ## Original Dependabot Update Bumps [smol-toml](https://github.com/squirrelchat/smol-toml) from 1.8.0 to 1.9.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/squirrelchat/smol-toml/releases">smol-toml's releases</a>.</em></p> <blockquote> <h2>v1.9.1</h2> <h2>What's Changed</h2> <ul> <li>fix: reject invalid extra exponent sign by <a href="https://github.com/cyyynthia"><code>@cyyynthia</code></a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/squirrelchat/smol-toml/compare/v1.9.0...v1.9.1">https://github.com/squirrelchat/smol-toml/compare/v1.9.0...v1.9.1</a></p> <h2>v1.9.0</h2> <p><strong>Huge update</strong>!!! This is most likely the largest update the library received since its release, with lots of new features and improvements.</p> <h3>Performance improvements</h3> <p>Significant parts of the internal parse logic have been rewritten, improving performance by <strong>1.5x-2x</strong>. The library was already comfortably ahead of the others, but it is now faster than ever, sitting at <strong>4x faster parse performance</strong> than the closest maintained implementation.</p> <p>Problematic code paths have also been replaced by safer implementations, solving potential DoS vectors. See GHSA-r4xh-jqrq-34v2.</p> <p>Note: the objects returned by the library now have a <em>null prototype</em>. This is a transparent change for 99.9% of users, and is one of the most significant contributors to the major performance gains in this version.</p> <h3>Full Temporal support</h3> <p>Version 1.8.0 brought support for Temporal in <code>stringify</code>; now the library is also able to emit Temporal objects instead of its own ad-hoc <code>TomlDate</code> object. It is not enabled by default, but it will become the default in v2. Enable by setting <code>useLegacyDate: false</code> in the parser's options.</p> <h4>Better Temporal support in stringify</h4> <p>Temporal support has been improved since it released: Temporal objects that cannot be represented (such as <code>Temporal.PlainMonthDay</code>) now throw an error (instead of silently emitting a bogus object).</p> <p>A new option has been added to <code>stringify</code> to disallow Temporal objects that cannot be fully represented in TOML. This includes <code>ZonedDateTime</code> objects with a IANA timezone attached instead of a plain offset, and dates with a specific calendar value set. Enable by setting <code>strictTemporal: true</code> in the options.</p> <h3>Handling of unsafe keys</h3> <p>Since its release the library has been protected against prototype pollution attacks, setting properties like <code>__proto__</code> using safe mechanisms that do not trigger prototype pollution. However, while the returned objects are safe on their own, they may become problematic if used carelessly.</p> <p>Inspired by <code>secure-json-parse</code>, the library now offers a way to either drop unsafe properties from the returned object, or to throw an error and reject documents altogether. By default, these potentially unsafe keys are preserved and returned.</p> <h3>Miscellaneous updates</h3> <ul> <li>Unicode BOM is now gracefully accepted and ignored.</li> <li>Table array headers are now properly checked again. Reported in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/65">#65</a>.</li> <li>Closed certain gaps where invalid whitespace would be accepted. Reported in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/61">#61</a>.</li> <li>Bogus local date and local time values with a UTC offset are no longer accepted.</li> <li>Certain error messages are more accurate and handle errors at line boundaries better.</li> <li>The default export of the lib is now formally deprecated; use a <code>import *</code> instead. Proposed in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/50">#50</a>.</li> <li>On Node 20+, strings that contain lone surrogates are now normalised to well-formed strings.</li> <li>On Node 20+, keys that contain lone surrogates are now rejected.</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.0">https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/squirrelchat/smol-toml/commit/69aa050fef018b71799519bb2cbb54f158a0c5f9"><code>69aa050</code></a> fix: actually properly fix exponent issues</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/7be7084d24c85909cec84ecaafe39737ef4e4472"><code>7be7084</code></a> chore: version bump</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/be3290da3309502447e020ce106bb8a11d99e580"><code>be3290d</code></a> chore: upgrade deps</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/86c76134ad7a50e8e101ef7fd82cb0f155b278d8"><code>86c7613</code></a> fix: reject invalid extra exponent sign</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/6f9739a5ab481f57c3112024bc4947534b580b58"><code>6f9739a</code></a> fix: gate <code>[is|to]WellFormed</code> (Node 18 compat)</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/a73ca32c7c9237f744c880d900ad15307a0cef56"><code>a73ca32</code></a> fix: no Temporal with toml-test when Node &lt; 26</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/7727890604580bd3fc2d7d2f79ad6e67b745ebe7"><code>7727890</code></a> chore: version bump</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/641903d60d47900080602b6a83e919d2a526547f"><code>641903d</code></a> chore: rewrite README.md</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/2df14c57945fec5a3737bc0cdce06a436bf9a77e"><code>2df14c5</code></a> fix(types): make it work if Temporal doesn't exist</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/3eaa44ef54b859990969935664b12023b26d0eae"><code>3eaa44e</code></a> chore: update benchmark harness</li> <li>Additional commits viewable in <a href="https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Devin Foley <devin@devinfoley.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 18:46:09 -07:00