PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-21 00:05:49 +02:00

Author	SHA1	Message	Date
Nicky LeachandPaperclip	c3ffa3f0dd	test(adapter-utils): fix feedback request index in bridge round trip (#16507 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox agents use the callback bridge to call the Paperclip API. > - The bridge round-trip test checks request delivery and credential removal. > - #12785 added an accepted API-key request before the feedback request. > - The feedback assertions still read request 2, which is now the GitHub credentials request. > - This PR checks request 3 and verifies that all four accepted requests reached the host. > - This restores the shared test gate while retaining the Unicode payload and token checks. ## Linked Issues or Issue Description Related PR: #12785 introduced the stale feedback index. The reported failure appears on #16486 and master. Searches for `workspaces-b`, `sandbox-callback-bridge`, `surrogate`, and `EncoderFallbackException` found no duplicate fix. What happened? The `General tests (workspaces-b)` job fails on PRs and master. The hosted PR job and AWS master job both report `System.Text.EncoderFallbackException` for an unpaired `\uD83D`. The runner exits during the test step and does not upload that step's log. The unchanged master test reproduces a deterministic assertion failure at the feedback request. It compares `/runtime-tools/github/credentials` with `/api/companies/company-1/agent-commentary` and prints a diff containing 5,000 crying emoji. The assertion output is the likely trigger for the runner's Unicode logging crash. - [Reported PR job](https://github.com/paperclipai/paperclip/actions/runs/38818474563/job/116719880033) - [Master job with the same runner exception](https://github.com/paperclipai/paperclip/actions/runs/38821128038/job/116717690951) Expected behavior The test checks the actual feedback request, verifies its full payload, and confirms that its authorization header is removed. Both shared CI lanes complete normally. Steps to reproduce 1. Check out master at `c4a16cbe981f96e2756c7014103c88da67e5514f`. 2. Install dependencies with pnpm 9.15.4. 3. Run `node_modules/.bin/vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts -t 'round-trips localhost bridge requests'`. 4. Observe the feedback assertion reading the preceding GitHub credentials request. Paperclip version or commit Master at `c4a16cbe981f96e2756c7014103c88da67e5514f`. Deployment mode Built from source. GitHub-hosted PR CI and AWS post-merge CI. ## What Changed - Assert that four accepted requests reached the host. - Check feedback delivery and authorization stripping on request 3. - Retain the large Unicode payload and all existing authentication checks. ## Verification - Unchanged master: the focused round-trip test fails at the stale feedback index. - Fixed branch: all 68 tests in `sandbox-callback-bridge.test.ts` pass with real localhost gateways. - `pnpm --filter @paperclipai/adapter-utils typecheck` passes. - `git diff --check` passes. - Exact CI group: `node scripts/run-vitest-stable.mjs --mode general --group general-workspaces-b` passes (4,800 tests passed, 28 skipped across 17 projects). - `pnpm --filter @paperclipai/adapter-utils build` passes. - Greptile reviewed current head `3de15d3374354809ea84ca4df5f93c2473715f02`: 5/5, success, no comments or actionable findings. - All GitHub checks pass on current head `3de15d3374354809ea84ca4df5f93c2473715f02`: 55 successful checks and 2 intentional skips, including `workspaces-b`, build, typecheck, Runner verification, and e2e. - Full local `pnpm test:run` was started and stopped after the CI test lanes passed; it did not complete locally. No worktree-owned test processes remained afterward. - Full local typecheck and build were attempted. Both stop at the Runner Rust commands because `cargo` is absent from the shell PATH. The changed package typecheck passes. The complete repository typecheck and build pass in CI. - The first sandboxed local reproduction could not bind localhost (`EPERM`). It was repeated with listener permission and reached the deterministic assertion failure described above. ## Risks Low risk. This changes three test lines. Runtime behavior, credentials, workflow settings, and payload coverage stay the same. The GitHub runner Unicode logging defect remains upstream; this fix removes the incorrect assertion that triggers it in this test. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository inspection, and command execution. The exact runtime model identifier and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 16:32:20 -07:00
Serhi Paperclip commitperclip[bot]	8aaef59339	fix(pi-local): add stdout compaction to stop quadratic run-log growth (#10249 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `pi_local` adapter runs the `pi` CLI in `--mode json` and records its NDJSON stream as a run-log per turn > - In JSON mode `pi` emits one `message_update` event per streaming token, each carrying the full cumulative assistant message, so a single turn's run-log grows quadratically with output length (measured ~77 MB for one thinking-model turn) > - The root cause is in `pi`, but the Pi project declined to add a compact mode (earendil-works/pi#3905, closed the same day), so Paperclip has to filter on the consumer side > - This PR adds a dedicated, config-gated compactor at the `pi_local` stdout boundary (`stdoutLogMode`: `raw` \| `compact`, default `compact`) that strips the cumulative snapshots while preserving streaming deltas, terminal events, and parser/UI output > - The benefit is linear run-log growth (~150× smaller in practice) with no change to `parsePiJsonl` results, token usage, or live typing — keeping storage and replay bounded ## Linked Issues or Issue Description - Refs #5699 — same root cause ("full accumulated partial state" → 46 MB log, per that PR). This PR takes a different, single-concern approach: a dedicated `stdout-compaction.ts` module with an explicit `stdoutLogMode` config (default `compact`, fail-safe fallback to `raw`), scoped to `pi_local` only. #5699 instead filters inline in `buffered-on-log.ts`, bundles an unrelated `claude-local` change, and is currently conflicting + stale (last updated 2026-05-14). Happy to coordinate with @ukint-vs on a merged direction. - Refs earendil-works/pi#3905 — proposed adding a compact JSON mode upstream in `pi`; closed by the maintainer, which is why this filters on the Paperclip side rather than the producer. - Related #1846 (run-log/retention TTL) — complementary: this PR reduces what's stored per run; #1846 addresses how long rows live. - No public GitHub issue exists that is an exact duplicate of the `pi_local` quadratic-growth symptom; the problem is tracked via #5699 above. (No internal/instance-local references used.) ## What Changed - `packages/adapters/pi-local/src/server/stdout-compaction.ts` (new, 123 lines) — `createPiStdoutLogCompactor(mode)`. `raw` passes lines through unchanged; `compact` strips the cumulative `message`/`assistantMessageEvent.partial` from `message_update` events while preserving streaming deltas, terminal events, and passthrough of unknown event types. Default mode `compact`; an invalid explicit mode fails safely to `raw`. - `packages/adapters/pi-local/src/server/execute.ts` — read the mode via `resolvePiStdoutLogMode(config)` and apply the compactor at the stdout → `onLog` (run-log) boundary. - `packages/adapters/pi-local/src/index.ts` — expose the `stdoutLogMode` config option (documented inline: `string`, optional, default `"compact"`). - `packages/adapters/pi-local/src/ui/parse-stdout.ts` — confirm the stdout parser consumes compacted output unchanged. - Tests — new `stdout-compaction.test.ts` (359 lines) covering modes, edge cases, and `parsePiJsonl` replay equivalence; updated `execute.remote.test.ts`. ## Verification - Unit: `pnpm --filter @paperclipai/adapter-pi-local exec vitest run` → 121/121 passing (12 files). - Typecheck: `pnpm --filter @paperclipai/adapter-pi-local exec tsc --noEmit` → clean, 0 errors. - Real-data replay: a real `pi --mode json` stream (`zai/glm-5.2`, thinking=high) compacted line-by-line through this compactor: - Raw 76.96 MB → Compact 525.5 KB = 99.33% reduction (≈150×). - 4503 lines in, 4503 out (0 dropped); 0 `message_update` with a cumulative `.message` remaining; 4489 streaming deltas preserved. - Parser equivalence: `parsePiJsonl(raw) ≡ parsePiJsonl(compact)` — 0 errors on both; identical `finalMessage` (13,369 chars); identical `usage` (`inputTokens 2896, outputTokens 4491, cachedInputTokens 6848`). Result, token accounting, and live-typing are unchanged. ## Risks - Behavioral shift: default `compact` changes what is stored in run-logs vs today. Operators who need the verbatim stream can set `stdoutLogMode: "raw"`. Flagging for reviewer preference — happy to flip the default to `raw` for upstream if that's preferred. - Ordering with secret redaction (#8027 / #9856): compaction is structural and runs at the stdout boundary before redaction; it only removes cumulative duplicates and never alters token content, so it does not change what redaction sees. - Unknown event types pass through unchanged (forward-compatible with future `pi` events). - Low risk otherwise — the change is contained to the `pi_local` adapter. ## Model Used OpenAI `gpt-5.6-luna` — via pi's `openai-codex` provider (272K context window). Used for implementation and for the real-data measurement in Verification. Reasoning/thinking: extended thinking enabled at level `high` (pi `--thinking high`; maps to OpenAI reasoning effort). `gpt-5.6-luna` is a reasoning-capable model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com>	2026-10-15 16:24:06 -07:00
Sergio-LPA Builder Claude Opus 5	cc549691a1	fix(claude-local): stop sealing max-turns failures as model_not_found (#13885 ) <!-- Written in Simplified Technical English --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The `claude_local` adapter runs the Claude Code CLI and maps each finished process to a terminal result. The `errorCode` in that result decides which recovery path the issue takes. > - `execute.ts` resolves the error code with one long ternary chain. The order of the branches is the precedence rule. > - The model-not-found branch sits before the max-turns branch, and it is driven by `isClaudeModelNotFoundError()`, which regex-scans the run's full stdout for the phrase `model_not_found` (also `model not found`, `unknown model`, `model does not exist`). > - The agent's own tool output is part of that stdout. A run that greps adapter source, tails a log, or echoes an issue body about this defect matches the detector on text the agent printed, not on text the provider returned. > - A real `subtype=error_max_turns` failure therefore never reaches `max_turns_exhausted`. Downstream, `model_not_found` counts as a configuration-incomplete failure, so the issue goes to `blocked` with a wrong "configuration incomplete" notice instead of the normal max-turns continuation. > - This pull request gates the model-not-found branch behind the two structured signals that the `providerQuota` and `transientUpstream` branches above already apply. > - The benefit is that a structured CLI signal outranks a regex scan of untrusted output, so exhausted runs continue instead of blocking the issue. ## Linked Issues or Issue Description No public issue exists for this defect. I searched the open pull request list for `packages/adapters/claude-local` and for the `model_not_found` and `max_turns_exhausted` error codes, and found no duplicate or related pull request. The description follows the bug template. What happened A run that fails with `subtype=error_max_turns` is recorded with `error_code=model_not_found`. In `packages/adapters/claude-local/src/server/execute.ts`, the resolved-error-code ternary evaluates the model-not-found branch before the max-turns branch: ```ts : failed && isClaudeModelNotFoundError({ parsed, stdout: proc.stdout, ... }) ? "model_not_found" : failed && clearSessionForMaxTurns ? "max_turns_exhausted" ``` `isClaudeModelNotFoundError()` in `parse.ts` pushes `input.stdout` into its candidate list and tests it against `CLAUDE_MODEL_NOT_FOUND_RE`. The run's stdout holds the agent's own tool output, so any run that prints the phrase poisons its own classification. Because the branch is first, a genuine max-turns failure never reaches `max_turns_exhausted`. This is not cosmetic. Downstream, `model_not_found` is treated as a configuration-incomplete failure. The issue moves to `blocked` with a misleading "configuration incomplete" recovery notice instead of the normal max-turns continuation path. Steps to reproduce 1. Run a `claude_local` agent on a task whose tool output contains the literal string `model_not_found`. Grepping the adapter source or reading a bug report about this defect is enough. 2. Let the run exhaust its turn budget, so the CLI emits `subtype=error_max_turns`. 3. Observe that the recorded `error_code` is `model_not_found` and not `max_turns_exhausted`. Expected behavior A run whose parsed result carries `subtype=error_max_turns` is classified `max_turns_exhausted`, whatever phrases appear in its stdout. A structured CLI signal outranks a regex scan of untrusted output. Paperclip version or commit `ghcr.io/paperclipai/paperclip@sha256:23090eb6`. The branch is rebased onto current `master` (`b41ccf0`). The adapter is `@paperclipai/adapter-claude-local` on the `engine=cli` lane. Deployment mode Self-hosted, Docker Compose. ## What Changed - `packages/adapters/claude-local/src/server/execute.ts` - Gate the model-not-found branch with `!clearSessionForMaxTurns && !poisonedPreviousMessageId`, the same guard the `providerQuota` and `transientUpstream` branches ~30 lines above already apply. One line, no new state. - Add a comment that records why the order matters, so a later reorder is a deliberate decision. - `packages/adapters/claude-local/src/server/execute.max-turns-classification.test.ts` (new) - A parsed `error_max_turns` result whose stdout carries the literal `model_not_found` phrase still classifies as `max_turns_exhausted`. - A real provider 404 (`API Error: 404 model not found`) still classifies as `model_not_found`, so the guard does not hide the genuine case. Only the parsed-result path changes. The unparsed fallback path (`if (!parsed)`) is left alone: neither guard variable is in scope there, and a run with no parsed result has no structured max-turns signal to honour. ## Verification - `pnpm exec vitest run src/server/execute.max-turns-classification.test.ts` in `packages/adapters/claude-local`: 2/2 pass. - Negative control: with the one-line guard reverted, the first test fails with `expected "max_turns_exhausted", received "model_not_found"`. The test pins the defect and not the surrounding code. - `pnpm exec vitest run` for the whole adapter: 313 pass, 1 skipped, 3 fail. The 3 failures are in `acp.test.ts` and `acp.auth.test.ts`, all with `claude_acp_node_unsupported` because the local machine runs Node v22.23.1 and the repository requires Node >= 24.11.0. They fail the same way on `master` without this branch, and they do not touch the CLI classification path. - `tsc --noEmit -p tsconfig.json` for the adapter: no errors. ## Risks - Low risk. The change is one added condition on one branch of an existing ternary. - A real model-not-found failure still classifies correctly, because neither `clearSessionForMaxTurns` nor `poisonedPreviousMessageId` is set on that path. The second test covers this. - The narrow case that changes behaviour is a run that is both out of turns and pointed at a missing model. That run now reports `max_turns_exhausted` first. The next run reports the model error against a clean session, which is the same outcome the quota and transient-upstream branches already accept. ## Model Used - Provider: Anthropic Claude. - Exact model ID: `claude-opus-5` (Claude Opus 5). - Reasoning mode: extended thinking. - Capability details: tool use for shell, file editing and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (no user-facing documentation covers adapter error-code precedence) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Builder <builder@nublo.net> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-10-15 16:01:40 -07:00
Michel Tomas	5b15de9568	fix(adapter-utils): accept the bridge token in X-API-Key (#12785 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent can run inside a sandbox. The sandbox callback bridge is the loopback gateway that lets the agent call the Paperclip API from inside that sandbox > - The gateway authenticates each local call with a per-run bridge token. It read that token from the `Authorization: Bearer <token>` header only > - The run hands the agent that token in `PAPERCLIP_API_KEY`, next to `PAPERCLIP_API_URL` (`execution-target.ts`). The token is named an API key, and the public Paperclip API documents that an API key goes in the `X-API-Key` header > - An agent that follows the documentation therefore sends `X-API-Key` first, and its first call fails with 401 `Invalid bridge token.` > - The agent then retries with a bearer header and the same token. The run still succeeds, but it wastes a turn and it can leave a confused agent trying unrelated repairs > - This pull request makes both generated gateways read the bridge token from `X-API-Key` when the request carries no bearer token > - The benefit is that a documented header convention stops producing a needless failure, at no cost to the token check ## Linked Issues or Issue Description No existing issue found. I searched the repository for open pull requests matching `x-api-key` and `bridge token`. Nothing addresses the in-sandbox gateway header convention. The closest neighbours are unrelated: #12632 (Composio `mcp.headers.x-api-key`) and #12120 (the HTTP/2 transport for this bridge, already merged). The problem follows `bug_report.yml`: What happened? An agent running in a sandbox calls the Paperclip API through the loopback callback bridge. When it sends the credential as `X-API-Key: <bridge token>`, the gateway answers `401` with `{"error":"Invalid bridge token."}`. The token in that header is the correct per-run bridge token. Expected behavior The gateway accepts the correct per-run bridge token in either header convention. The public API documents `X-API-Key`, so an agent that follows the documentation succeeds on its first call. Steps to reproduce 1. Start a run whose agent executes in a sandbox execution target, so the callback bridge is active. 2. The run exports `PAPERCLIP_API_URL` (the loopback bridge origin) and `PAPERCLIP_API_KEY` (the bridge token) into the sandbox environment. 3. From inside the sandbox, call the bridge with that key in the documented header: ```bash curl -i -H "X-API-Key: $PAPERCLIP_API_KEY" "$PAPERCLIP_API_URL/api/agents/me" ``` 4. The response is `401 {"error":"Invalid bridge token."}`. 5. Repeat the same call with `-H "Authorization: Bearer $PAPERCLIP_API_KEY"`. The response is `200`. Paperclip version or commit Branched from `master` at `06c0e883f`. Deployment mode Self-hosted Kubernetes, with the agent in a sandbox execution target and a local model behind an OpenAI-compatible endpoint. Agent adapter(s) involved Not adapter-specific (core bug). Observed with a local-model adapter on a sandboxed lane. Any agent that follows the public API documentation sends this header. Additional context The variable name is the trigger. The run calls the credential `PAPERCLIP_API_KEY`, and an agent that has read the public API documentation knows an API key goes in `X-API-Key`. Paperclip's own in-prompt guidance uses `Authorization: Bearer $PAPERCLIP_API_KEY`, so the two conventions sit side by side. Across 12 observed runs on that lane, the agent's first bridge call used `X-API-Key` in 6 runs. Every one of those runs recovered by retrying with a bearer header and the same token. In some runs the agent first tried unrelated repairs, such as writing a test comment, before it changed the header. ## What Changed - Add one `readBridgeToken(req)` helper to the generated in-sandbox gateway source. The helper returns the bearer token from `Authorization` when the header carries one. It otherwise returns the `x-api-key` header value, or an empty string. - Use that helper in both generated gateway request handlers: the file-mode queue gateway and the HTTP/2 gateway. Both previously repeated the same two lines of bearer parsing. - The token still goes to the same `timingSafeEqual` comparison. The change does not alter the comparison, the token, or its lifetime. - Extend the file-mode round-trip test to cover an `X-API-Key` request with the right token, an `X-API-Key` request with a wrong token, and the fact that the header never reaches the host. - Add one HTTP/2-path test that covers the same three cases, and that proves a bearer token still wins when a request carries both headers. ## Verification Commands run on this branch: ``` node_modules/.bin/vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts Test Files 1 passed (1) Tests 52 passed (52) cd packages/adapter-utils && ../../node_modules/.bin/tsc --noEmit 0 errors in the two files this pull request touches ``` Both new tests are real regression guards. With the test file in place and the source change reverted, each one fails with `expected 401 to be 200`: the file-mode round-trip test and the HTTP/2 test. Both pass with the source change applied. A reviewer can confirm the security properties by reading two things. First, `tokensMatch` in the generated source is unchanged, so the constant-time comparison against the same per-run token is unchanged. Second, `allowedHeaders` in the same source is a closed allowlist of `accept`, `content-type`, `if-match`, and `if-none-match`. `normalizeHeaders` drops every other header, so neither `authorization` nor `x-api-key` leaves the sandbox on either transport. The independent host-side check in `http2-bridge-server.ts` is untouched, because each gateway sets `authorization: Bearer <token>` on the request it forwards. ROADMAP.md check: I grepped `ROADMAP.md` for `bridge`, `x-api-key`, `redaction`, and `sandbox callback`. There were no matches. This is a small bug fix in an existing subsystem, so it does not overlap planned core work. Duplicate search: I searched pull requests in this repository for `x-api-key` and `bridge token`. No open or merged pull request changes how the in-sandbox gateway reads the token. ## Risks Low risk. - The change widens which header the gateway reads. It does not widen what the gateway accepts. A request still needs the exact per-run bridge token, and it still passes the same constant-time comparison. - A bearer token keeps priority. A request that carries both headers behaves exactly as it did before, which a test now pins. - A caller that presents a non-bearer `Authorization` header and a valid `X-API-Key` header now succeeds instead of failing. This is intentional: the bridge only ever issues bearer tokens, so no bridge caller relied on that failure. - The header allowlist already excluded `x-api-key`, so no forwarded request changes shape. - No migration, no configuration change, and no behavior change for a caller that already sends a bearer token. ## Model Used Claude Opus 5 (`claude-opus-5`), extended thinking at high effort — verify before merge. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — no documentation describes the bridge's header convention, so there is nothing to update - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- <sub><em>[@superbiche](https://github.com/superbiche) · contributor · drafted with Claude Opus 5, reviewed before posting; the voice and decisions are mine.</em></sub>	2026-10-15 15:31:31 -07:00
Nicky LeachandPaperclip	ec3eb1968b	feat(runner): expose credential updates through PRP (#16463 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Runner uses PRP to send harness events to the host. > - A harness can replace its stored OAuth tokens during a run. > - The host has no PRP signal that tells it to read the updated credential. > - This pull request adds a durable credential update event and observers for supported stores. > - The host can react before it acknowledges the event. Token values stay outside PRP. ## Linked Issues or Issue Description Subsystem affected `packages/paperclip-runner`: native provider execution, PRP schemas, and host event delivery. Problem or motivation A Paperclip host that consumes PRP needs to react when a harness changes its refresh-token credential. Existing credential copy-back paths do not provide a common event during a warm session. Proposed solution Add `auth.credentials.updated` as a P0 runner event. Include only the provider, binding ID, revision, and detection method. Observe native Codex and Codex/Grok ACPX credential files. Retain each pending event across runner restart until the durable outbox accepts it. Raw PRP consumers opt in with `observeCredentials: true` before session open. The bundled transport opts in for fresh sessions. Alternatives considered `harness.diagnostic` would mix a runner observation with harness diagnostics. Sending token values through PRP would expose credentials to event logs. Both approaches are unnecessary. Roadmap alignment I checked `ROADMAP.md`. This change extends the existing Runner protocol and does not duplicate a planned core feature. Additional context Refs: #11094, #14056, #12696. These describe credential copy-back and staging work. This PR adds the notification contract. It does not replace those private credential channels or claim to fix their full scope. ## What Changed - Add strict credential update payload validation to PRP v1, v2, and v3. Regenerate validators, the manifest, and the event catalog. - Watch supported credential directories with `notify`. Add a periodic read as a fallback and check at terminal and close boundaries. - Persist a private baseline and pending event together. Notify on account identity changes and retain the new baseline for later refreshes. Ignore incomplete files, unsafe files, and metadata changes. - Forward the event through the host transport. Test commit-before-ACK delivery and replay. - Commit pending runner observations before suspend and shutdown commands complete. Preserve the provider event backlog. - Create the private ACPX runtime directory before configuring observation. Cover fresh session startup with observation enabled. - Fence asynchronous PRP commits to the current authenticated connection and run authority. Ignore replaced callbacks while preserving the maintenance drain of admitted work. - Deliver observations found during final provider cleanup before clearing terminal recovery state. Retain the outbox and recovery state when the host ACK is lost. - Advance the pending Copilot candidate profile to v19 for the changed shared PRP source hashes. Preserve its v18 history and pending qualification status. Include the current fixture in the Docker build context. - Commit credential updates through an awaited host callback before PRP ACK, including during idle and close. Persist them in a separate run-log source namespace with company, agent, session, and runner checks. - Preserve public event metadata through server redaction. Register the event in task run details. - Document opt-in behavior, supported stores, event semantics, and the local run-log path. ## Verification - All 45 durable runner control tests and 24 native executor integration tests pass. - Ten Rust credential observer tests pass. They cover atomic replacement, polling, recovery, account-only changes and later refreshes for Codex and Grok, unsafe files, fresh runtime creation, and the Grok close handoff. Account identity stays private. - The full Runner TypeScript suite passes: 3,565 tests in 221 files. An additional 12 tests are skipped by the suite. - All 214 focused tests pass for the contract, Codex event routing, catalog, and durable host delivery. - All 13 Node protocol contract tests pass. - The exact Docker context integrity command passes: `docker buildx build --file .github/docker-context-checks.Dockerfile .`. - Rust formatting and generated protocol checks pass. - `pnpm -r typecheck` and `pnpm build` pass. The full local `pnpm test:run` was attempted with CI-compatible `init.defaultBranch=master`, then stopped after a reproducible failure outside this change: `git-workspace-expiration.test.ts` expects one re-expired workspace but gets zero. The same case fails in isolation on this macOS checkout. Full repository validation is provided by CI; the focused Runner and host tests pass locally. - All 253 native transport and restart tests pass against the final release binary. - All 3 durable recovery tests and 139 candidate profile and native admission tests pass. - All 41 host redaction and task protocol surface tests pass. `pnpm check:token-gates` passes. - All 467 CI policy tests pass, including the 15 CI profiling tests. The upstream secret-reference test fix is retained from master. No workflow changes are included. - The Apex host-delivery fix passes 381 targeted Runner tests, six host database tests, repository typecheck, and the production build. The Runner tests cover commit-before-ACK, retry after a rejected host commit, warm attachment, and close after the turn queue ends. - Prior head `2f125cfcb` passed CI, standard Greptile 5/5, and Apex 5/5. One CI shard was interrupted by GitHub runner shutdown; its single rerun passed. Both Apex findings are addressed: idle/cleanup host delivery and late callback reconnect fencing. Current head `5db78f082` passes CI (56 successful statuses and four optional skips) and standard Greptile 5/5. Apex review of the account-change update is 5/5 on this exact head, with no actionable issues. - All 121 durable control-plane and credential callback tests pass, including account-only change delivery, subsequent refresh on the new account during close, host commit retry, late success/failure races, forward cursor delivery on a replacement connection, and the existing maintenance retirement contract. All 27 native provider and durable recovery integration tests pass after the account-change update. ## Risks - File changes signal a stored credential update. They do not prove that an OAuth refresh succeeded. Changes between reads can be combined. - Unix Codex and Codex/Grok ACPX stores are supported. Other harnesses and credentials held only in memory need their own integration. - Hosts must update their event schemas before opting in. Older hosts omit the flag and receive no new event. Older runners ignore the flag. - The host must use its existing private credential channel and account checks to read or copy the token. Account switches trigger notification but do not authorize accepting the new account. The event grants no credential access. - Tests use synthetic credentials. No live OAuth refresh has been forced. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`, with high reasoning effort, repository inspection, tool use, and code execution. The session does not expose its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 14:49:00 -07:00
DottaandPaperclip	8843887c24	fix(runner): preserve managed GitHub access in OpenCode (#16488 ) ## Thinking Path > - Paperclip manages AI agents and their authorized connections. > - Native OpenCode agents use managed Git and GitHub launchers for shell work. > - Each operation must obtain credentials from the active run's broker. > - Two OpenCode environment filters dropped that broker binding before the shell started. > - The shell therefore ran anonymously even when connector tools had authorized access. > - This pull request preserves the explicit broker environment through both filters. > - The benefit is working GitHub shell access with current identity and revocation checks. ## Linked Issues or Issue Description What happened? Native OpenCode shell commands reported `GitHub capability_missing` despite an authorized GitHub connection. Both runner startup and provider startup dropped the controller's broker settings. Connector MCP calls used a separate path and could still succeed. Expected behavior A managed shell operation obtains credentials for the current run. A later operation uses the new identity after rotation and loses access after revocation. Ambient host credentials do not supply authority. Steps to reproduce 1. Start the native GitHub broker with an active run binding. 2. Pass its environment through native OpenCode startup. 3. Run `gh api user` from the provider shell. 4. Observe anonymous access before this fix. Paperclip version or commit Reproduced on master `6943904bad` before this change. Deployment mode Native Paperclip Runner with OpenCode, including remote execution. Related: #16401 fixes the separate oversized OpenCode SSE failure and is already merged. The same live-driver regression was verified locally against its exact pre-fix source and current source. This PR does not duplicate that fix. Searches found no open PR for this native broker propagation defect; #15309 concerns the legacy OpenCode adapter. ## What Changed - Use the existing GitHub credential environment projection at both OpenCode launch boundaries. - Preserve explicit controller authority while excluding an ambient host broker. - Exercise the real launcher and broker from an OpenCode child process, directly and through runner projection. - Check operation-time identity rotation, revocation, and exclusion of controller API credentials and persistent GitHub tokens. - Document the managed OpenCode shell contract. ## Verification - Red: both new `native-github-access.test.ts` cases returned `anonymous` instead of `fixture-a` before production changes. - Green: both cases pass after the fix, including `fixture-a` to `fixture-b` rotation and anonymous operation after revocation. - Full native GitHub broker suite: 18 tests pass. - OpenCode, provider-event, and configured-environment suites: 141 tests pass across eight files. - Separate SSE red/green verification: the existing live-driver regression times out on `fd057f6ac4^` and passes with merged #16401. The full OpenCode group also passes after restoration. - Full repository build and typecheck pass locally. All CI jobs pass for `23d734293f6a5c6d6128cae0c8773661816daf9a`. Final GitHub rollup: 55 successful checks/statuses, two skipped checks, and one neutral check; none pending or failing. Greptile gave that commit 5/5 with no findings or unresolved threads. - Paperclip Review spent over 23 minutes preparing its runtime without a provider assessment. Its run was cancelled for recovery and ended neutral (review execution did not complete). This neutral result is not a code assessment. No check policy or branch protection was changed. - The full local Vitest attempt first hit embedded PostgreSQL startup failures. Running the unchanged `initdb --version` and `postgres --version` independently showed missing native library symlinks. The dependency had been installed with scripts disabled. Running its bundled `hydrate-symlinks.js` restored startup; all 15 affected native-session cases and all 112 AI connection tests then passed without application code changes. The original full-suite attempt was stopped after the setup failure and its log is retained. A complete local suite pass is not claimed. - The broker and executable are local fixtures. No live GitHub account, credential change, or test PR is required by these regressions. ## Risks The change affects credential delivery into OpenCode processes. It reuses the existing bounded GitHub projection and accepts broker authority only from explicit input. Each shell operation still uses the normal broker, scope, identity, and revocation rules. Unrelated controller secrets remain excluded. There are no database or permission-policy changes. ## Model Used OpenAI `gpt-6.1-sol`, through Codex, with xhigh reasoning and repository tools/code execution. The session does not report its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the focused regression and affected suites locally and they pass; the full local attempt and setup failure are disclosed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (Paperclip Review is neutral/incomplete, as disclosed above) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 16:15:38 -05:00
Devin FoleyandPaperclip	0b62c736a0	feat(connections): add generic Daytona API and Toolbox access (#16483 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps connections give agents controlled access to external services. > - Daytona provides sandbox, file, command, Git, and account APIs. > - Operators need to assign this access without exposing a server credential. > - A small task-specific tool list cannot serve general Daytona work. > - This pull request adds the published HTTP actions through the existing connection gateway. > - Operators keep control of identity, agent access, and each action. ## Linked Issues or Issue Description Problem or motivation Paperclip has a Daytona sandbox provider for its own execution infrastructure. It lacks a normal Apps connection for agents to manage an operator's Daytona account. Reusing the server's provider credential would bypass normal connection ownership and action permissions. Proposed solution Add a Daytona connection with personal or shared vault credentials. Generate its action catalog from the official platform and Toolbox API specifications. Use the existing run-bound gateway, grants, action policies, approvals, revocation, and audit paths. Sandbox IDs come from Daytona; tools need no Paperclip task or lease ID. Alternatives considered A four-action diagnostic subset cannot support general provider work. Launching the provider CLI would expose host filesystem behavior. Native OAuth is not included because a supported third-party registration path was not verified. The native method uses an organization API key. Existing Composio setup is a separate possible OAuth route and is not qualified by this change. Roadmap alignment This extends the existing Apps and Connections provider coverage in `ROADMAP.md`. It uses that system's credential and permission model. GitHub searches found no existing Daytona Apps connector PR or issue. ## What Changed - Add the Daytona catalog entry, provider source override, official theme logos, permission review, and setup documentation. - Generate 274 HTTP actions: 169 platform and 105 Toolbox operations. Record two WebSocket upgrades separately; do not advertise them as HTTP tools. Record source URLs and hashes. - Add provider-focused execution for sandboxes, snapshots, volumes, files, Git, commands, computer use, and account resources. Provider authority still limits which endpoints accept an organization key. - Validate action arguments. Resolve Toolbox targets through the fixed platform API. Check sandbox identity and organization before using an approved Daytona proxy. Keep normal guarded network requests and vault resolution. - Transfer file bytes without reading or writing host files. Share one caller deadline across prerequisite reads, dispatch, and response reading. Bound requests and responses. Do not automatically retry writes with unknown outcomes. Redact the selected credential from results and errors. - Classify credential access, access changes, interactive input, command execution, file permissions, and deletion (including TTL and auto-delete) as destructive actions. Keep new and changed actions under catalog review. - Upgrade saved catalogs and retire the earlier diagnostic names. Hide retired actions from the UI and agent discovery even when provider refresh fails. Preserve existing action choices and historical records. Reject stale same-name contracts during agent discovery and dispatch before a board refresh. - Add database-backed gateway and catalog-upgrade regressions, schema generator tests, a browser setup check, and both capability map updates. ## Verification - Passed on the review-fix head: 57 focused Vitest cases (18 Daytona database/client cases, five HTTP boundary cases, and 34 shared catalog cases), four generator tests, `pnpm -r typecheck`, and `pnpm build`. The five HTTP cases use a real local server through the pinned production transport. The normal browser setup check and 17 generator/artwork cases passed on the initial PR head; those UI and artwork paths are unchanged. - Passed on this branch: brand asset validation, token gates, map structure checks, and `git diff --check`. All latest-head GitHub checks pass on `ea979ccb3a64dc9508807823540f935c4316ea35`. Apex is 5/5, with all five findings addressed and zero unresolved threads. The branch has no merge conflicts. - The initial catalog-generation command could not find the external research corpus. The supported `--definitions-only` mode regenerated the definitions successfully. Unrelated generated drift was excluded. - An earlier full local Vitest attempt on the preview branch failed when bundled PostgreSQL could not start on macOS. After replay onto current master, all three unchanged event-sequencing cases pass with bundled PostgreSQL; the earlier startup cause remains uncertain. - The first full-suite command on this branch used my external test database override. Chat fixtures require a current migrated schema and failed because `agents.lifecycle_state` was absent in that disposable database. A focused comparison reproduces failure with the override and passes with normal setup. That attempt was stopped, and the full suite restarted without the override. Focused Daytona cases use their own migrated PostgreSQL 16 databases. - The clean local full-suite attempt recorded embedded PostgreSQL startup failures in two unchanged native-runtime cases and two failures in unchanged Git fixtures. It was stopped after these failures were isolated; no full local suite pass is claimed. The 15 matching native-runtime cases pass separately. The Git cases fail with the local `init.defaultBranch=main` setting and pass with `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master` limited to the test command. No global Git setting was changed. Full Linux CI is green. - Fixtures use real database persistence and run-bound gateway sessions with provider responses replaced by fixtures. They cover personal/shared vault ownership, reads and writes, file bytes, Off policies, revocation, organization mismatch, untrusted proxy rejection, output bounds, saved catalog upgrades, failed refresh, and permission preservation. - The first Apex review found five valid issues: multipart transport compatibility, destructive classification, caller deadlines, audit filters, and stale saved contracts. All five were fixed with regressions before the second review. A typecheck then found the nullable saved schema hash; admission now accepts that stored type and rejects null as stale. - The browser check covers setup controls without submitting a provider key. No real Daytona account, provider write, native OAuth flow, or agent model run is qualified. Live provider read/write proof remains pending a test account. ## Risks - Daytona keys have broad Toolbox authority, including keys with no selected resource scopes. Only assign this connection to trusted agents and review its action permissions. - Remote writes or commands can continue after a timeout. Calls do not automatically retry an uncertain outcome. - The bridge targets Daytona Cloud. Proxy hosts are restricted; an unrecognized future regional host fails closed. - Files and responses have an 8 MiB bound. Each action has one caller deadline, capped at 30 seconds across all provider requests and response reading. Interactive WebSocket sessions are not exposed as HTTP actions. - The generated JSON snapshot exceeds 1,000 lines because it holds the provider input contracts. Generator logic and execution code remain in separate small modules. Updates require explicit regeneration and catalog review. - No database migration or lockfile change is required. ## Model Used OpenAI Codex, GPT-6. The runtime identifies the model family but does not expose the exact backend model ID or context window. Used reasoning, repository inspection, code editing, shell tools, database verification, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 14:05:26 -07:00
DottaandPaperclip	61831ba7b8	refactor: defer adapter launcher imports during workspace snapshots (#16492 ) Defer adapter launcher imports until a directory merge resolves its lock root. Snapshot-only workers retain the same hashes, modes, exclusions, and ownership checks. Verification: 63 focused cases; full typecheck/build; exact-head full CI/E2E/security; Apex 5/5. Local native Daytona replies: 66.939s cold and 26.485s warm. Hosted staging and the cold latency target remain unverified. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 16:03:47 -05:00
DottaandPaperclip	6943904bad	perf: stage independent remote GitHub launchers concurrently (#16484 ) Stage at most three independent remote GitHub launcher files at once only when the existing effective concurrent-sync capability permits it. Preserve serial fallback, per-file locks and integrity checks, and settle every started upload before failure returns. Verification: 119 affected tests pass (one existing Linux-only oracle skipped on macOS); full repository typecheck/build pass; same-sandbox real-Daytona component comparison preserves all bytes/modes; current-head full CI/E2E/Superagent/Snyk and Apex 5/5 pass. The full native timing target remains separate and is not claimed by this change. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 15:36:43 -05:00
DottaandPaperclip	69dded3230	fix: allow normal Daytona command status response latency (#16480 ) Allow a normal completed Daytona status response up to 500ms within the unchanged caller deadline. Keep final separated logs, stream fallback and single dispatch. Verified: 396 provider cases, full repository and standalone provider typecheck/build, matched real-Daytona component controls, and exact-head full CI/E2E, security and Apex 5/5. Whole-agent latency target remains open. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 14:42:36 -05:00
DottaandPaperclip	c53bdd9eb3	perf: capture sparse checkpoint files in bounded batches (#16473 ) Capture sparse checkpoint files in bounded batches while retaining content and mutation checks. Prevent inserts after SQLite automatic rollback and drain active readers before database close. Verified: 209 affected cases, final six capture controls, repository typecheck/build, real Daytona helper validation, and exact-head full CI/E2E, security and Apex 5/5. The matched component median saves 1.05s; the whole-agent latency target remains open. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 14:32:44 -05:00
DottaandPaperclip	b1566ca494	perf(daytona): avoid log-follow delay for completed session commands (#16468 ) A completed Daytona session command now reads final separated logs without waiting for a new log-follow socket. Running commands retain streaming, recovery, and the existing total deadline. Verified with full repository typecheck/build, standalone provider typecheck/build, 395 passing provider cases, real immutable-image Daytona command controls, and fresh exact-head full CI/E2E and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 13:35:10 -05:00
DottaandPaperclip	68a09dcb5f	perf(ci): shorten the test execution critical path (#16434 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Contributors wait for the full PR workflow before they can deliver fixes. > - Server shards still execute independent files in series, and a single Runner file serializes two long real-clock waits. > - Faster work outside the critical path does not prove a faster workflow. > - Overlap independent files while preserving isolation, test cases and real production deadlines. > - Compare complete baseline and candidate workflows, separating queue delay from execution cost. ## Linked Issues or Issue Description Refs #16429. Related duration refresh: #12577. What existing behavior does this improve? The elapsed time from PR CI creation to the last required check. Current behavior General server shards use one worker. One Runner transport file takes about 333 seconds on Linux, including over 100 seconds in two real-clock regressions. These are the longest verification lanes after #16429. Proposed behavior Run independent general-server files in at most two isolated forks. Move the two intact Runner deadline regressions into a separate file to overlap their waits in the existing pool. Run two chat partitions per existing job, each with its own temporary home and embedded PostgreSQL. Preserve serial execution within files and every assertion and timing boundary. ## What Changed - Qualify both serial and parallel forks with real worker processes, isolated homes/configuration, timers, globals and environment. - Require both parallel workers to arrive before either exits. - Use two isolated forks for sharded general-server tests; retain one worker with an explicit shared test database. Other test groups keep their existing concurrency. - Extract the lease-expiry and Pi cold-admission regressions verbatim into a separate Runner test file; retain real waits and process/lease assertions. - Split each sharded chat job into two independent invocations; preserve serial shared-database execution. Await both children before failing the parent. Prove disjoint coverage and failure propagation. - Update the paired Linux benchmark to compare a complete general-server shard with identical case inventories. - Document concurrency and deadline-test placement. No production behavior, workflow topology or runner count changes. - Allow 15 seconds for snapshot integration cases that cold-start multiple real TypeScript workers. On the repeat, one-worker cases took 2.5–3 seconds, so queued or sequential starts could exceed the default five seconds. Preserve all cleanup, queue-release and filesystem assertions; these cases do not enforce a production latency deadline. ## Verification All 48 jobs passed on scheduling head `5d718e18` in [attempt 1](https://github.com/paperclipai/paperclip/actions/runs/38763885288/attempts/1). The [bounded repeat](https://github.com/paperclipai/paperclip/actions/runs/38763885288/attempts/2) failed two existing snapshot-worker tests at their default five-second timeout. Its shorter duration is not counted as a successful performance sample. The latest head `3c894462` adds a targeted test-budget repair and [passes all 48 CI jobs](https://github.com/paperclipai/paperclip/actions/runs/38768433532) in 8m06s (workflow creation to final required workflow job). This is 70 seconds / 12.6% shorter than the 9m16s intermediate run, but the earlier 9m23s result remains part of the evidence: a stable average improvement is not established. Both branch-protection checks (`ci / verify` and `ci / e2e`) pass. The non-required Superagent Security Scan is still running; external PR scans are separate from this workflow duration. \| Measurement \| Before \| After \| What it establishes \| \| --- \| ---: \| ---: \| --- \| \| General-server shard, paired on one Linux host \| 430.94s \| 292.99s \| 32.0% faster; identical 1,047 case inventory \| \| Chat job 1 \| 460s \| 211s \| Same full chat inventory spread across isolated partitions \| \| Chat job 2 \| 408s \| 274s \| Serial cases within each child, separate PostgreSQL instances \| \| Chat job 3 \| 408s \| 216s \| Total chat runner time fell from 1,276s to 701s \| \| Full PR workflow, first chat candidate \| 9m16s \| 9m23s \| Not an end-to-end win in this pair: unchanged Build rose from 365s to 465s \| \| Full PR workflow, final repaired head \| 9m16s \| 8m06s \| All 48 jobs pass; 70 seconds faster observed, not a controlled causal estimate \| - Latest [paired benchmark](https://github.com/paperclipai/paperclip/actions/runs/38763885223) passes. Earlier paired server measurements were 33.3% and 37.4% faster. Unchanged fixture timing noise is not attributed to this PR. - All 1,337 chat case identities and outcomes match exactly across [before-chat](https://github.com/paperclipai/paperclip/actions/runs/38760937988) and [after-chat](https://github.com/paperclipai/paperclip/actions/runs/38763885288/attempts/1), without duplicates or removals. Whole-suite records rose from 44,563 to 44,574: three new launcher checks and eight upstream warm-owner tests. Remaining name differences were generated fixture IDs/timestamps/PIDs, not removed tests. - The final run used `m7i-flex.xlarge` for Build (408s), with all chat jobs completing in 238s / 318s / 280s. It used the same actual merge base `afe5b341` as the first chat candidate. Optional profiling was disabled after the bounded measurement experiment; the full 48-job correctness workflow still ran. - The fleet used `m7i.xlarge` for the earlier Build and `m6a.xlarge` for the first chat candidate Build. Several unchanged compile/install/probe phases slowed together. This does not prove hardware alone caused the difference, and full-run attribution must account for runner/cache variance and the upstream base change. - Original full baseline: [all 48 jobs passed](https://github.com/paperclipai/paperclip/actions/runs/38753547876/attempts/2), 12m07s, including a server job that queued over five minutes. It is not a clean basis for attributing the whole elapsed-time reduction to this PR. An interrupted baseline and two misrouted candidate attempts were excluded and preserved; routing/trust rules were not weakened. - Local: 32 sharding/launcher checks pass, including disjoint coverage, shared-database fallback and a failed child that waits for its sibling. Real paired chat partitions passed 416 cases in 217.63s on independent embedded databases. - Both extracted Runner deadline regressions pass unchanged with real waits. An initial local attempt rejected a Node executable under the controller home; the identical binary in an owned non-home path passed without weakening that guard. - Real worker-isolation proof, actionlint and diff checks pass. Latest-head Greptile is 5/5, with no review threads or actionable findings. - The eight snapshot-worker cases pass locally with a private `PAPERCLIP_HOME`. The first direct local invocation failed to open the default-home lock database under the sandbox; no production guard was changed. ## Risks Parallel files and paired chat databases may expose shared-state assumptions or increase peak memory. Cases stay serial within each invocation, and explicit shared databases keep the serial path. Complete server coverage and the full 48-job CI workflow pass. Check external required PR scans before merge; the full-workflow measurements are observational and do not establish a steady 12.6% speedup. Queueing, cache state and runner variance must be distinguished from execution improvements. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code execution and GitHub tools. The exact serving model suffix and context capacity are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (48 jobs; both required branch-protection checks pass; non-required external security scan pending) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 12:22:59 -05:00
DottaandPaperclip	25b16e65fc	fix(observability): detail workspace and connector preparation (#16454 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server prepares a repository, sandbox workspace, and connector access before an agent can reply. > - Existing traces measure large preparation phases but leave important work inside those phases unclear. > - Repository source materialization and Browser Use discovery can do more work than their callers suggest. > - New detail must preserve access checks, run ownership, and the existing work order. > - This pull request adds fixed OpenTelemetry spans inside those operations. > - Operators can use one deployed trace to select the next performance fix. ## Linked Issues or Issue Description Refs: #16419, #16425, #16428. What existing behavior does this improve? Startup traces for server workspace preparation, native sandbox synchronization, and connector assignment resolution. Current behavior Workspace and connector aggregate spans can take many seconds. The trace does not separate private source cloning, seed reuse, archive work, progress persistence, or each connector resolver. Browser Use resource selection builds the full authorized tool catalog. The configuration and native backend can each resolve connector assignments. Proposed behavior Show the awaited work as fixed child spans. Separate discovery queue wait from catalog construction. Preserve the initiating run context when another run releases a scheduler slot. Add no new lifecycle event writes for workspace detail callbacks. Reason and benefit Operators need evidence about the hosted critical path before another performance change. These spans distinguish filesystem work, provider transfer, authorization, transaction entry, locks, and reporting overhead. Breaking changes None. Authorization, query order, transaction ownership, synchronization, existing progress events, and task results retain their contracts. The change adds opt-in Observability spans under the existing OTLP endpoint gate. ## What Changed - Trace private source directory setup, Git credentials, clone, requested ref, source identity, and checkout publication. - Trace sandbox source capture, seed cache/replay, shallow clone and archive, overlay copy/archive, verification, upload, warm stamps, and descriptor persistence. - Separate existing awaited progress reporting from useful workspace work. Keep new detail callbacks trace-only. - Trace each connector resolver and Browser Use catalog/grant work. Split discovery queueing, descriptor sources, and policy evaluation. - Trace Slack authority, credential reads, catalog synchronization, transaction entry/settlement, and advisory locks. - Update the real filesystem staging trace expectations and the Honeycomb investigation guide. - Add a real scheduler regression for a queued job released by another run. Assert run ownership, parent spans, and exactly one completed queue span. ## Verification - Passed: server and adapter-utils TypeScript checks (`pnpm --filter @paperclipai/server exec tsc --noEmit` and `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`). - Passed: 252 checks across 9 focused files. These cover real filesystem staging, native workspace synchronization, trace ownership, endpoint gating, and workspace preparation. - Passed after review: 26 trace-context, tracing-failure, and discovery-scheduler checks across 3 files (these overlap the first run). The new queued-discovery regression fails when context binding is removed and passes after restoration. An initial submillisecond timestamp assertion was replaced with explicit span-completion order to avoid SDK clock precision assumptions. - Passed on implementation commit `0e68968a4bd61001893d833f281f12bb110edbe1`: `pnpm -r typecheck` and `pnpm build`. The later commit updates only an engine test expectation. - The first CI workspace lane found an old engine trace-tree expectation. It expected the earlier span set. The updated assertion now checks the new detail spans, their parents, and single completion. Its targeted executor check passes. - The full test suite, build, release registry, and end-to-end gates pass in CI on final head `40e37d89ef679109d60a96bb5731a0616efeaa43`: 55 passing checks and 2 optional Storybook skips. The full suite was not duplicated locally after the focused run. - Greptile reviewed the final head at 5/5. No unresolved review threads remain. - After deployment, filter Honeycomb by the hashed run ID and verify `service.version`. Inspect `heartbeat.prepare.workspace`, `runtime_selection.workspace_sync`, and the connector assignment parents. The guide in `doc/observability.md` explains each new family and the nested intervals. - No hosted run has used this new instrumentation yet. Local checks cannot establish hosted latency improvement. ## Risks - Tracing adds spans when an operator configures an OTLP endpoint. Detail remains capped at 16 spans per phase per execution attempt. - Parent and child durations overlap. The documentation explains how to avoid counting them twice. - A discovery job can start in another run's scheduler continuation. The bound callback preserves the initiating async context. - New spans export fixed names and the existing hashed run ID. They add no paths, repository URLs, connector resource IDs, credentials, tool schemas, or exception text. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 12:21:32 -05:00
Leia MarshallandLeia Marshall	e4ce989099	Add opt-in Gmail archive capability (#16331 ) ## Thinking Path > - Paperclip separates Gmail capabilities by reviewed OAuth scope and tool policy. > - Existing Gmail connections support reading and draft creation but not bounded archiving. > - Gmail archive requires `gmail.modify`, so it must not widen existing read or draft grants. > - This change adds an opt-in archive profile and limits it to removing only the `INBOX` label. > - A final pre-dispatch check preserves that boundary for approved and replayed calls. ## Linked Issues or Issue Description What existing behavior does this improve? This improves the existing Gmail connection and tool-governance behavior. Subsystem affected Cross-cutting: `packages/shared` and `server`. Current behavior Paperclip can read Gmail and create drafts. It does not provide an opt-in, bounded action for archiving a thread. Proposed behavior Paperclip offers a separate Read & archive connection. It exposes `unlabel_thread` only for that connection and accepts only `threadId` with `labelIds: ["INBOX"]`. Reason and benefit Users can archive processed Gmail threads while sending, trashing, and other label changes remain disabled. Breaking changes None. Existing read and draft methods keep their current scopes and defaults. Related work: #15545 adds a unified Google Workspace connector and keeps Gmail draft-only. #15412 adds a separate Gmail REST server without label mutations. Neither PR adds this bounded archive capability. ## What Changed - Added the `gmail.archive` connector profile. - Added managed and customer-owned archive connection methods. - Added an Inbox-only schema and final pre-dispatch argument guard. - Classified archive as destructive and added Ask-first defaults for the archive methods. - Added permission-review records and focused tests. ## Verification - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/shared typecheck` - Focused Vitest suites: 42 tests passed. - App-definition ingestion: 103 definitions and 0 review flags. - `git diff --check` ## Risks The main risk is accidental scope or operation expansion. The new profile is opt-in. Existing profiles retain their scopes. The gateway validates the exact Inbox-only payload immediately before dispatch. Production rollout requires separate OAuth configuration and fresh consent. ## Model Used Unverified — reliable runtime model metadata was not available. Confirm the model attribution before publishing. ## Checklist - [x] I have included a concise design rationale - [x] I have specified that model attribution is unverified - [x] I have checked ROADMAP.md and found no Gmail archive entry - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR - [x] I have not referenced internal or instance-local issues - [x] My branch name describes the change - [x] Local tests and builds pass - [x] I have added or updated tests - [x] I have updated relevant documentation and permission records - [x] I have documented risks - [ ] All hosted CI gates are green - [ ] Greptile and human review are complete --------- Co-authored-by: Leia Marshall <leia@paperclip.ing>	2026-10-15 09:24:31 -07:00
DottaandPaperclip	cccde7122b	Trace execution preparation and durable response persistence (#16419 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server prepares an execution environment before it starts an agent. > - Operators need to distinguish preparation, provider wait, and response persistence. > - Existing preparation phases start separate traces, and the native session trace ends before the final response is saved. > - This pull request adds an execution-attempt trace and detailed spans at the existing work boundaries. > - The benefit is a measurable startup and settlement path without changing execution policy. ## Linked Issues or Issue Description What existing behavior does this improve? Operator-configured OpenTelemetry tracing for agent preparation, startup, and durable response storage. Subsystem affected Server orchestration, native Runner startup, and the Honeycomb run link. Current behavior Preparation phases start independent roots. Workspace lock waits, authorization groups, runtime asset work, provider-pack fallback, and the gap after provider completion are hard to attribute. Proposed behavior Use a server-owned async context for each execution attempt. Parent nested work to that attempt. Link the separately sampled native session because its ingress and queue history can begin earlier. Record transaction entry, lock waits, grouped authorization, asset preparation, artifact fallback, copyback, and response commit. Reason and benefit Operators can identify expensive work from recorded boundaries instead of inferring the cause from elapsed time. Nested and concurrent durations remain visible and must not be added together. Breaking changes No API or execution-policy change. New Observability span names and closed count attributes are documented. The existing local run-performance schema also records the additional provider-pack and harness measurements. Refs: #16178, #16205, #16330. ## What Changed - Add `task.execution_attempt` with same-run async ownership, independent root sampling, and native-session links. - Measure workspace transaction entry and settlement, advisory and row waits, source scans, reader groups, notifications, and writes. - Measure runtime selection and each concurrent instructions, skills, and MCP branch. Separate manifest, cache, materialization, and verification work. - Measure provider-pack discovery, verified reuse or upload, final verification, and remote harness state checks. - Measure workspace settlement, accounting, liveness, response authorization, comment commit, and publication enqueue. - Preserve the silent endpoint-off path, including when another library has registered a tracer. Bound repeated detail spans to 16 per name per attempt. - Extend the Honeycomb run link and document the actual coverage and remaining gaps. ## Verification - After resolving the conflict with retained-reader batching on `b98cdaea531c8447637cd4dcc19c61b3f5f41844`, all 61 targeted query, authorization, workspace-preparation and trace-boundary tests passed. The server typecheck passed. Reader tests still bound query count, avoid loading retained run payloads and enforce fresh project grants on every publication attempt. - Before that base integration, the tracing/exporter group passed 855 tests with 2 optional-SDK skips across 10 files and no inherited OTLP endpoint. It covers real PostgreSQL waits/rollback, durable response commit, assembled heartbeat preparation, concurrent parentage, sampling and tracer failure isolation. Native workspace/history tests passed 23 cases. These results remain supporting evidence for the unchanged tracing paths. - Full workspace typecheck and build passed during implementation. Full sharded CI, release validation, E2E and Apex 5/5 passed on prior head `8cebad80f3856dbcff6f3a6d5d34880ff77a20f2`. Fresh CI, security and Apex review are required on the current head before merge. - `pnpm check:token-gates` passed. No styles changed. - Earlier CI runs exposed exact-source-expression assertions changed by wrappers, missing run hashes on transaction-boundary spans, and sampling fixtures without an enabled endpoint. The fixes preserve the calls, add the declared hash and span cap, and enable tracing explicitly in the sampling fixtures. Focused repairs passed; the full prior-head CI passed after those repairs. Earlier failed run: https://github.com/paperclipai/paperclip/actions/runs/38729131366. - The full local `pnpm test:run` was stopped after the base advanced during that process. Its mixed-version results are not final-head evidence. A full local-suite pass is not claimed. - The old-head Superagent scan stalled at 35%. The current head has a fresh scan; no security pass is inferred from the old pending result. - No deployed Honeycomb acceptance is claimed by this code change. ## Risks - Enabled tracing creates more spans. Repeated detail spans are bounded, and inventories use group spans. - The attempt and native session remain linked traces. Historical queue time can precede the attempt, and nested intervals overlap. - A committed comment proves durable server storage. Publication enqueue and realtime dispatch do not prove external delivery or browser paint. - New custom attributes use only the existing run hash and validated file/byte counts. No SQL, paths, permissions, content, or exception text is added. ## Model Used OpenAI GPT-6 through Codex. The serving model ID and context window size are not exposed in this session. The agent used reasoning, tools, code editing, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 08:13:53 -05:00
DottaandPaperclip	6acec4266e	feat(ci): profile full CODEOWNER PR suites in Honeycomb (#16378 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its PR workflow runs dozens of parallel build and test jobs. > - Individual job durations do not show the elapsed cost of the whole suite. > - Maintainers need one shared timeline to find slow work and repeated setup. > - This pull request adds full-suite profiles for selected CI CODEOWNER PRs. > - Honeycomb shows each job, step, test file, and supported test on that timeline. > - The benefit is evidence for changes that reduce total CI time. ## Linked Issues or Issue Description What existing behavior does this improve? The main PR workflow has about 48 jobs. Maintainers need to compare their overlapping work and find which jobs delay completion. Current behavior GitHub shows job and step durations separately. Test files and tests are not joined to a full workflow trace. Proposed behavior A CI CODEOWNER adds `profile-ci` to a PR. The tested code stays on that PR branch. All main PR lanes run, including lanes normally skipped for stack position or release notes. A trusted post-run exporter sends one wall-clock trace to Honeycomb. Reason and benefit The trace supports decisions about which CI work to speed up or remove. Ordinary PRs do not export traces. The roadmap and related-PR search found no duplicate CI profiling work. ## What Changed - Add label selection and a full-suite override. Verify the PR author, original actor, and rerun actors against the trusted `.github/` CODEOWNERS rule. - Record Vitest, Playwright, and Node test file/test timings in local artifacts. Keep ordinary reporter behavior and failing exit codes. - Add a trusted `workflow_run` exporter. Use actual GitHub timestamps for all jobs, setup, steps, waiting, and teardown. Mark missing test data and partial reruns. - Keep the ingest key out of PR jobs. Store it only in the `ci-profiling` environment, restricted to `master`. The exporter reads bounded JSON and never executes artifact code. - Record expected test invocations outside reporters so a missing invocation cannot appear complete. Preserve `node --test` on older PR checkouts. Flush large shard inventories before exit. - Add security, timing, routing, and real-reporter checks. Document operation and activation. Protect the exporter with CODEOWNERS. ## Verification - 509 focused workflow/reporter/shard checks passed. They include outside authors, outside rerun actors, bots, stale revisions, malicious ZIP paths, incomplete reports, and partial ingestion failures. - Final revision `3bf5d9017d` adds multi-file qualification and fixes short-file step parenting and duplicate Node file-wrapper events. Greptile reviewed this exact revision at 5/5 with no unresolved threads. [Main PR CI run](https://github.com/paperclipai/paperclip/actions/runs/38728040302): the initial attempt had 54 successful checks, 3 intentional skips, and one serialized-shard failure from an unchanged workspace-authorization test exceeding its 15-second timeout. The single failed-shard retry passed: the final result is 55 successful checks and 3 intentional skips. The same timeout reproduces locally with profiling disabled; both assertions pass with a diagnostic 60-second timeout override (about 23 seconds total). No source or CI timeout was changed. - A dry run against real GitHub attempt 2 of run `38677445871` produced exactly 3 executed jobs, 40 spans, and 510 seconds, excluding 45 carried-over jobs. The partial trace remains explicitly incomplete; this validation did not upload data. - `actionlint` passed for all four changed workflows. `git diff --check` passed. - Full Node 24 `pnpm -r typecheck` and `pnpm build` passed. The local full `pnpm test:run` was stopped after it reported a setup-token timeout and a distribution-plugin-catalog fixture failure. The 30 setup-token tests passed on focused rerun. The plugin fixture fails at `rmSync` of a directory symlink on this macOS/Node 24.11.0 host; the same failure reproduces with the unchanged base Vitest configuration and unchanged test/service files (10 other tests pass). The local full test suite is therefore not claimed green. - A real root-config Vitest probe passed 13 tests with profiling both off and on. Multi-file Vitest, Playwright, and Node probes each verify two distinct filename rows and four correctly parented tests, including repeated test names across files. Separate passing probes exercise the full exporter hierarchy `workflow → job → step → filename → tests`. Intentional failing probes preserve failing exit status. - Verified all 48 measured jobs in latest-head run `38728040302` used AWS: one `paperclip-ci-routing-x64` gate and 47 `paperclip-public-pr-x64` jobs. Profiling preserves AWS authorization and its GitHub-hosted fallback. The separate privileged exporter runs on `ubuntu-latest` outside the measured suite. - [Live Honeycomb baseline](https://ui.honeycomb.io/paperclip/environments/test/datasets/paperclip-ci/trace/oaPMzw8zGwG): 48 jobs, 731 spans, 479 seconds. It matches [GitHub run 38671274294](https://github.com/paperclipai/paperclip/actions/runs/38671274294). This historical trace has job/step data only; it is not proof of the future label-triggered test-reporting journey. - Verified the dedicated Honeycomb key is ingest-only with dataset creation disabled. Verified the GitHub environment allows only the `master` branch. No repository-level secret of that name exists. Both credential values were scanned absent from source and branch history. After merge, add `profile-ci` to a CI CODEOWNER PR, wait for the entire PR run and exporter, then follow the run-summary link. Collapse job children to compare parallel lanes; expand a job to inspect its steps and tests. ## Risks - Activation requires merging the trusted workflow definitions to `master`. This PR continues using the current trusted reusable workflows until then. - Selected runs produce many spans and artifacts. Oversized artifacts fail export instead of truncating data. Missing or cancelled reporters are marked incomplete. - GitHub timings have second precision. Node test times use reporter receipt time. Rust retains job/step timing but lacks per-test wall timestamps. - CI CODEOWNERS are individual users in the explicit `.github/` rule. Unsupported team rules and identity lookup failures disable profiling. - An interrupted upload can leave partial data. Retrying uses the same span IDs, but the Events API does not promise deduplication. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, repository tools, and browser verification. The runtime does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 06:59:50 -05:00
DottaandPaperclip	efcc8a7d89	test: bound command snapshot forwarding checks (#16421 ) Keep the command snapshot option test focused on forwarding and real file copyback. Use inline delegates in that test, and retain the separate real-worker suites and original deadlines. Validation: 124 affected tests, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. Negative controls reject lost command and restore forwarding. Failed earlier merge-readiness evidence remains in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 06:57:38 -05:00
DottaandPaperclip	7fc2dc8b32	fix: isolate workspace snapshots from controller work (#16414 ) Move native sandbox snapshot captures off the controller callback queue while preserving the existing capture and restore contract. Keep legacy adapters inline, bound captures to two workers, and verify cleanup and reordered worker completion. Validation: 461 affected cases, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. The full local canonical suite remains running; no full local pass or under-30-second staging response is claimed. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 06:31:59 -05:00
Devin Foley	8365872743	fix(codex-local): retry ACP model capacity failures as transient upstream (#16407 ) Classify typed Codex ACP `service` failures about model capacity or server overload as `codex_transient_upstream` / `transient_upstream`, so the existing bounded transient retry and continuation recovery apply. In `unexpected` Sentry mode, skip only runs carrying that full classified evidence; `all` mode still reports them.	2026-10-15 04:28:02 -07:00
Devin Foley Claude Opus 5.5 Paperclip	fd057f6ac4	fix(runner): keep OpenCode turns alive through oversized SSE events Retain bounded OpenCode SSE events without restarting the event stream. Preserve canonical tool identity, status, output tails, original UTF-8 sizes, and truthful truncation markers. Bound id-less event deduplication with hashes. Cover fragmented and complete large frames, exact-limit split separators, bulk input retention pressure, truncation-record exhaustion, and unpaired surrogate byte counts. Full current-head CI and Apex 5/5 passed. The unrelated local post-tool answer assertion was reproduced on unchanged baseline source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 01:02:28 -07:00
DottaandPaperclip	074c67fea8	fix(grok): avoid routine wakes for accepted work (#16394 ) ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - Grok Bot receives assignments through routine webhooks and uses agent MCP tools. > - Each accepted assignment belongs to the current external interaction. > - A pending tool result or comment could start another routine while that work was still active. > - Quiet MCP traffic cannot prove that Grok has stopped working. > - This pull request keeps receipts and feedback durable for the current interaction to read. > - New task routines start only for offers that the Bot has not accepted. ## Linked Issues or Issue Description Refs #16219 and #16305. The duplicate search also found #12991, which covers instruction files for local Grok runs. What happened? The dispatcher could start another routine for an unread result after a period without a routine reply. It could also wake accepted work for a task comment. A queued delivery could still start after the Bot had claimed the offer. Expected behavior Accepted work stays in its current interaction. The Bot polls the original operation receipt and reads queued feedback at a safe boundary. Acknowledgements of a sent wake remain valid after acceptance. Steps to reproduce 1. Accept a Grok assignment and leave it active. 2. Add an unread comment, or complete a tool operation whose call returned pending. 3. Run the dispatcher after its quiet period. 4. Observe another webhook call. Also queue an offer wake, accept the offer before sending, and observe that delivery still starts. Paperclip version or commit Observed during real-client qualification on Core `ff0f976de1`. The three new regression cases also failed on current master before the correction. ## What Changed - Stop automatic result and feedback wakes for accepted work. Keep both available through their existing read paths. - Require an unaccepted offer before dispatch or an explicit delivery retry. Serialize the final HTTP handoff with acceptance receipts. Cancel obsolete unsent wakes. Preserve late acknowledgements and immediate Stop authority. - Return current Grok polling instructions on assignment reads. Clarify pending and unknown outcomes in setup guidance. - Add protocol, receipt-polling, queued-delivery and late-acknowledgement regressions. Update the operation guide and readiness plan. ## Verification - Three new regression cases fail before the fix because a webhook starts. - The latest 31 Grok protocol checks and 47 existing Dot Runner checks pass. New tests force acceptance during secret lookup and the final send check. They also prove that acknowledgements and Stop remain free during pending delivery. - The final-head Grok, Dot, MCP and question-delivery suite passes all 212 checks across six files. Dedicated onboarding UI checks passed six. - Dedicated Runner driver checks use the real local Rust daemon. Built the debug daemon required by that package; all 17 checks then passed. - Repository typecheck and build pass. Token gates, the 2,731-story structural catalog and the Storybook build pass. - Public synthetic marker for the post-deployment native source-read check: `grok-native-read-b45be4eff22cb15a31d0af512f437b8d`. - Final-head CI passes all 55 checks, with two optional Storybook skips. Greptile is 5/5 with no open threads. The first chat-shard attempt failed one Telegram SQL-quiet timing assertion; all five related cases pass locally and the single targeted CI rerun passes. - After deployment, repeat actual Grok work through an answered question, receipt polling, a saved report and file, canonical completion and cleanup. This correction is not production qualification by itself. ## Risks A Bot that does not poll a pending receipt can wait without another routine wake. Current assignment reads now supply polling instructions to existing connections. Task comments stay queued until the Bot reads them or Runner admits the next continuation. The real client must verify that behavior before production enablement. The observed second file version does not prove which provider interaction created it. Acceptance can wait for the bounded webhook response. Stop and authenticated acknowledgements use independent row locks and remain available. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository editing, shell tools and browser tools. The exact serving model ID, snapshot and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 00:54:15 -05:00
DottaandPaperclip	854ab294cc	fix: restore repository admission and Claude startup on Linux ARM64 (#16372 ) Batch retained workspace admission reads while preserving individual privacy and trust decisions. Preserve bounded saved-policy projections when integrating current master, and qualify the pinned Claude executable on Linux ARM64. Verified exact head `4d468f998b` with full CI/E2E, Superagent, Snyk and Apex 5/5, with no unresolved findings. Local typecheck/build and 209 server plus 74 integrity cases pass; full local suite remains pending. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:33:09 -05:00
DottaandPaperclip	444d5854b0	test(adapter-utils): consolidate outbound restore span coverage (#16115 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:23:28 -05:00
DottaandPaperclip	86816ff965	test(adapters): consolidate Grok stale-auth promotion coverage (#16112 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:09:21 -05:00
DottaandPaperclip	982ff62949	test(shared): consolidate ZIP conversion observations (#16118 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:06:50 -05:00
DottaandPaperclip	5e62927a8f	test(adapters): consolidate Codex staging permission guards (#16111 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-15 00:06:35 -05:00
DottaandPaperclip	662a20281d	test(cli): retire duplicate ZIP codec owner tests (#16116 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:57:43 -05:00
DottaandPaperclip	0de90e25bf	test(adapter-utils): share pre-READY work-bound bridge execution (#16098 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:56:07 -05:00
DottaandPaperclip	1ff08a53cb	test(tailscale): retire duplicate substituted-listener withdrawal (#16106 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:51:52 -05:00
DottaandPaperclip	3b6d1452f2	test(daytona): retire duplicate pack span execution (#16099 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:45:50 -05:00
DottaandPaperclip	3ea39f75c1	test: consolidate duplicate owner proofs (#16075 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:17:13 -05:00
DottaandPaperclip	9076329149	test: consolidate duplicate runtime credential proofs (#16079 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:13:45 -05:00
DottaandPaperclip	0e1bb3748b	test(db): retain owner nullability in the active-slot keeper (#16076 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:13:15 -05:00
DottaandPaperclip	f64d05adf4	test: consolidate repeated ACPX startup metadata scenarios (#16068 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 23:02:09 -05:00
DottaandPaperclip	422728fc36	test: retire seven repeated ACPX trace executions (#16061 ) Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 22:50:25 -05:00
DottaandPaperclip	3dffd2197d	test: consolidate owner fixtures and retain real regression proof (#16047 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tests protect sandbox execution, run recovery, credentials and the board UI. > - Several owner suites repeat the same setup and prove the same result more than once. > - These copies add maintenance work and can hide weak assertions. > - Each cut needs a retained test that proves every distinct contract. > - This pull request consolidates repeated setup and removes twelve redundant executions. > - It also replaces two weak assertions with checks of real outputs. > - The benefit is 2,831 fewer test lines with clearer regression proof. ## Linked Issues or Issue Description What existing behavior does this improve? The maintenance and regression strength of the existing Vitest owner suites. Subsystem affected Sandbox adapter utilities, server heartbeat recovery and environment capabilities, and the board UI. Current behavior Tests copy database rows, query builders, render calls and runtime inputs. Some cases repeat an existing contract. One token test omits a real bridge credential. A PostHog host comparison can compare absent controls. Proposed behavior Use small local helpers with fresh values and the same action order. Keep each distinct contract in an owner test. Check real bridge credentials and fixed PostHog method states. Carry the exact persisted first-run success assertion into the stronger cancellation keeper. Reason and benefit The ten test files shrink by 2,831 net lines. Twelve redundant executions retire. All other case tables, inputs, security boundaries, timeouts and cleanup rules remain. Breaking changes None. This round changes tests and audit evidence only. Additional context Refs: #15978, the merged first cleanup. Refs: #16022, the merged prior cleanup. Its squash merge is `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Related open work: #14634 repairs feedback fixture completion; #14477 supplies explicit sandbox fixture tools; #14198 changes cleanup retry behavior in another heartbeat file. This audit preserves those separate action and cleanup contracts. It does not apply their changes. This PR was first stacked on `codex/test-audit-followup`. After #16022 was squash merged, only the two Round 6 commits were replayed onto merged master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. The original stack head is preserved in a local snapshot branch. The current diff contains only ten test files and two evidence files. No merge is requested. ## What Changed - Consolidate repeated setup in three sandbox adapter suites. Remove seven redundant executions and retain the real protocol, cleanup and scratch-file checks. Save 875 lines. - Consolidate heartbeat database fixtures and read queries. Retain exact wake cancellation, total run count and the actual first-run success receipt. Remove one duplicate execution. Save 1,372 lines. - Remove two repeated rejected-capability cases. Carry the absent snapshot assertion into the retained runner case. Save 22 lines. - Consolidate UI fixtures and render calls. Remove two repeated connection cases. Replace the weak PostHog comparison with fixed method and required-key checks in both real hosts. Save 562 lines. - Record audit scope, decision and expansion counts, all twelve keeper mappings and deliberate mutation results in the [audit report](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6.md) and [controls evidence](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6-controls.json). ## Verification Source baseline: `a090035cafb29c06f6fca2dbe43561ad866b7c1b`. All ten original test programs are byte-identical at the stacked base `58c5c4aa241c85f4665a3159c192888bbb2a5cfa`. - Baseline: 1,841/1,841 cases passed in the main thirty-file owner/sibling run. Another 219/219 cases passed in nine extra files. - Preservation: all four proposal packages passed independent review. Whole-program helper expansion retains ordered inputs, callbacks, awaits, queries, fresh allocations and hooks outside the declared assertion migrations and repairs. - Candidate: all 2,048 cases passed across 39 complete owner/sibling files. The main run passed 1,831 cases. The extra run passed 217 cases. Both runs had zero failures and zero skips. - PostHog control: the repaired positive case passed. The original weak case passed with authentication choices removed. The repaired case failed at its fixed OAuth aria-checked assertion under the same mutation. - Eight accepted mutations failed at their intended assertions. They cover bridge-token exclusion, fallback cleanup, READY bytes, real scratch-file preservation, exact wake cancellation, the persisted first-run success receipt, rejected capability snapshots and reachable PostHog method controls. Two original weak cases passed with the same defects. All production source bytes were restored. - Rejected control: changing run status before legacy release timed out before wake cancellation. This result is not counted as assertion proof. Its replacement changed only the persisted receipt after real drain in the existing transaction. It failed at the direct fixed succeeded-row assertion. - No new code comments or 1,000-line threshold crossings were introduced. No production, schema, lockfile or workflow changes are included. On the exact final test sources, later committed as `d8de9422f0703d4508a9dc880b67a07f13e0f110`, `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build` passed. The required full `pnpm test:run` completed with exit 1 after 52.14 minutes on clean, immutable `d8de9422f0703d4508a9dc880b67a07f13e0f110`. HEAD and tracked sources remained unchanged. The first general-server group ran 889 files: 882 passed, one failed and six skipped. Its 18,032 cases were 17,925 passed, two failed and 105 skipped. The only failing file was `server/src/__tests__/remove-test-home.test.ts`. On macOS, the read-only runtime bundle and symlink cleanup cases returned `ENOTEMPTY`. The permission-error case expects `EACCES` or `EPERM`. Four reported failure entries cover these two cases, including their cleanup errors. The test and helper are byte-identical to previous master `4b766b872993045a2e787404f0475503683e5ded` and audit master `e364b92c82`. The default wrapper stopped after that group failed. Later local groups did not run. The earlier public-MCP, HTTP log redaction and workspace snapshot failures did not recur. The 39 focused owner/sibling files and all 48 remote CI jobs completed independently. This result is an unresolved macOS cleanup limitation, not full local success. No test, skip, retry, timeout or cleanup rule was changed to hide it. The three adapter files received a whitespace tidy after focused execution. Full runtime AST and string/template/regex byte equality link those tested sources to the final sources. All 134 adapter helper calls were checked again. Replayed code commit `3d3defd2a60595a623045d9e2c6c586f4acb3452` has the exact final test bytes. Evidence commit `e868e6645ce0d598d756d3458f0b3d71afd6db4a` adds the report and controls JSON, with one historical ancestry clarification. All ten accepted test hashes and all seven production-owner source guards match. No package, lockfile, workspace, TypeScript, Rust or Vitest dependency manifest changed since the audit snapshot. The [original stack CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38312876910) passed all 48 jobs on head `c171d6c0d603d22bb997c1f9a9bdcd5bcc5d22aa`. Those results are historical stack proof. The [fresh master-context CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38318223856) completed successfully with all 48 jobs passed on current head `e868e6645ce0d598d756d3458f0b3d71afd6db4a`. Its tested merge commit was `41292af60c6eb60d477637d23822bb17b08dc07e`, combining workflow base `88bf4c6c1d222f818faa56257efe79207c892a6c` with that head. The delivery commits remain based on pinned master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Both required master checks, `ci / verify` and `ci / e2e`, passed. Two separate optional Storybook checks were skipped by their scope gate. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/16047#issuecomment-6133323165) gives the current head 5/5, with no findings, inline comments or unresolved review threads. The [current-head commitperclip review](https://github.com/paperclipai/paperclip/actions/runs/38318223666) also completed successfully. This PR remains open and draft; no merge is requested. The local macOS failure above remains unresolved. ## Risks - A helper could share a value or capture the wrong binding. Independent expansion and lexical binding checks guard the actual helper calls. - A retired case could contain a distinct assertion. Each cut has an exact keeper map and carried observations. - A mutation could fail before the intended assertion. Controls reject timeouts and earlier failures. The rejected receipt mutation remains in the evidence. - The delivery ancestry changed after the parent squash merge. Exact source hashes preserve the audited tests and producer receipts. Fresh CI and review passed in the current master context; the broad local run remains evidence from its immutable audit snapshot. - The complete local default failed in two unchanged macOS cleanup cases. Later local groups were not run. This limitation remains visible even though focused checks and remote CI passed. ## Model Used OpenAI Codex, based on GPT-6, assisted with this work. The exact served model ID and context window are not exposed in this session. It used repository tools, code execution, AST analysis and parallel audit agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 21:43:12 -05:00
Devin Foley devinfoley Claude Opus 5.5	3935e300dd	fix(acpx): bound Claude MCP tool calls when runtime MCP servers attach (#16360 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted `claude_local` runs use the ACPX engine. Paperclip gives each run its runtime MCP servers (for example the `paperclip-assigned` tool gateway) as ACP `mcpServers` entries. > - Claude's default MCP tool timeout is far longer than a run. ACP `mcpServers` entries have no per-server timeout field. So the agent waits on an MCP call until the call answers. > - Hosted error reports show runs that end with "Run exceeded the adapter execution timeout (sandbox)". The stop-time diagnostics from #15962 show the problem: the agent went silent about 50 seconds into the run, three tool calls of ACP kind `other` stayed pending, and nothing ended them until the 4h sandbox backstop. > - The gateway bounds the remote provider call (10s default, 60s max). But other steps on the gateway `tools/call` path have no deadline: DB queries, row locks and pool acquisition, the credential steps before the provider timer starts, and the stdio child exit wait. One stuck step holds the agent turn for hours. > - This pull request gives the agent's MCP client a deadline: `MCP_TOOL_TIMEOUT` for ACP Claude runs that receive runtime MCP servers. > - The benefit is that a stalled gateway call becomes an MCP error that the model can see. The turn continues and does not spend the full 4h execution budget. ## Linked Issues or Issue Description - Refs #15962 (diagnostics that identified the silent turn with pending tools) - Refs #15448, #15456, #15850 (related gateway timeout and connection-health work; this PR does not overlap with them) What happened? A hosted `claude_local` run sent parallel MCP tool calls. No call answered. The run stayed silent for about 4 hours. Then the sandbox execution timeout ended it with "Run exceeded the adapter execution timeout (sandbox)". Expected behavior A stalled MCP call ends with an error after a bounded time. The agent sees the error and continues. Steps to reproduce 1. Start an ACP `claude_local` run that has runtime MCP servers. 2. Make the gateway `tools/call` request stall (for example, hold a lock that the request needs). 3. See that the run stays silent until the adapter execution timeout. Paperclip version or commit `master` at `976cdc7`. ## What Changed - Add `ACPX_CLAUDE_MCP_TOOL_TIMEOUT_MS` (15 minutes) in `acpx-engine/constants.ts`. The value has wide headroom over the gateway's longest legitimate wait (60s provider calls, 2 minutes of approval preparation, and the 10-minute connection cap proposed in #15850). - The ACPX engine sets `MCP_TOOL_TIMEOUT` in the agent env for Claude when the run has runtime MCP servers. If the adapter `env` already sets `MCP_TOOL_TIMEOUT`, the engine keeps that value. Other agents and runs without MCP servers do not change. - Add an engine test for the bound, the no-server case, the user override, and a non-Claude agent. - Document the client-side deadline in `doc/MCP-RUNTIME-OPERATIONS.md`. ## Verification - `npx vitest run src/acpx-engine/` in `packages/adapter-utils`: 21 files, 423 tests pass. - The new test fails without the env assignment (`expected undefined to be '900000'`), and passes with it. - `npx tsc --noEmit -p packages/adapter-utils`: passes. - The env reaches Claude through the same per-session env path as `ANTHROPIC_MODEL`. `claude-agent-acp` merges this env into the Claude SDK env. ## Risks - A legitimate MCP call that takes longer than 15 minutes now fails. No gateway path is designed to wait that long. Operators can set `MCP_TOOL_TIMEOUT` in the adapter `env` to change the value. - The value also applies to other MCP servers in the same Claude session, for example servers in user settings. The same override is available. - Warm sessions get the env when the agent process starts again. The env is not part of the session fingerprint, so this change does not reset resumable sessions. - This PR does not fix the server-side step that stalls. It bounds the effect on the run. Server-side deadlines on the gateway `tools/call` path (DB lock and statement timeouts, a request-level deadline, SIGKILL escalation for stdio children) are possible follow-ups. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use and code execution. It read the repository, traced the gateway path, edited the code, and ran the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 19:01:42 -07:00
dependabot[bot]Devin Foley Paperclip	1b58ac6ec9	build(deps): bump smol-toml from 1.8.0 to 1.9.1 (#16020 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Codex runner must set trust for the exact startup project. > - smol-toml 1.9.1 returns tables with null prototypes. > - Object spread replaces those prototypes and causes strict validation to reject valid trust edits. > - This PR keeps the parsed tables and adds tests for the dependency update. > - Codex can start while unrelated settings and trust boundaries stay intact. ## Linked Issues or Issue Description Refs #16028. That merged PR contains the same table-preservation fix and tests for both prototype styles. This branch retains that work and adds more regression cases. Related dependency updates: #15293 and #15364. What happened? Eight startup trust tests fail after the update to smol-toml 1.9.1. The error is `codex_startup_trust_cannot_preserve_configuration`. Expected behavior Set trust for the exact startup project. Keep unrelated configuration bytes and sibling project trust unchanged. Steps to reproduce 1. Check out the original PR head, `90b133dfd963af4f19b4df11fdd6c11b4eeb2a2d`. 2. Install dependencies with `pnpm install --frozen-lockfile`. 3. In `packages/paperclip-runner`, run `pnpm exec vitest run src/drivers/codex/codex-startup-trust.test.ts`. Paperclip version or commit The verified head is `1a8e8ad8b2e1049ebb68595d85c92e3c0621559d`. Deployment mode Built from source on Linux. The affected code runs before Codex starts. ## What Changed - Keep Dependabot's update from smol-toml 1.8.0 to 1.9.1 in the runner and server. - Keep parsed project tables. Use `parse("")` to create missing tables with the parser's prototype. - Keep strict candidate validation and the upstream tests for both prototype styles. - Add tests for nested values, dates, times, special numbers, prototype-named keys, sibling trust, and repeated calls. - Preserve the original Dependabot commit and lockfile. The two follow-up commits change only the trust source and tests. ## Verification - All 28 startup trust cases pass at the verified head. Run the command above. - The focused strict TypeScript check passes. Use `tsc --ignoreConfig --noEmit --module nodenext --moduleResolution nodenext --target es2023 --types node --esModuleInterop --strict --skipLibCheck src/drivers/codex/codex-startup-trust.ts src/drivers/codex/codex-startup-trust.test.ts`. - The runner TypeScript build passes. The full local build stops because the test workspace has no Cargo executable. - The broader Codex suite passed 271 of 272 cases before the source conflict resolution. One unrelated fixture expects the runtime's workspace path to be absent. - The repository type check exited 137 in the DB package. The full local test command was stopped after resource exhaustion. - The final files merge cleanly with master `976cdc77ed564dc3e014621505890b647e554a7f`. `git diff --check` passes. - [CI for the verified head](https://github.com/paperclipai/paperclip/actions/runs/38664681165) passed. All 55 active checks passed; two Storybook checks were skipped. Snyk also passed. [Greptile cycle 3](https://github.com/paperclipai/paperclip/pull/16020#issuecomment-6182103775) scored this head 5/5 with no actionable findings and no open review threads. ## Risks - This update changes the parser used by the runner and server. CI must check both packages. - The trust edit must preserve unrelated TOML values and bytes. Regression tests cover this behavior. - Local resource and tool limits prevent a complete local validation result. The full remote CI run passed at the verified head. ## Model Used OpenAI GPT-6 through Codex helped with the compatibility fix, tests, and review. The runtime does not expose the exact model snapshot or context window size. The agent used repository tools, code execution, and GitHub tools. Dependabot produced the original dependency and lockfile update. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — the focused 28-case suite and strict type check pass; broader limits are listed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — this PR records the compatibility behavior and verification - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- ## Original Dependabot Update Bumps [smol-toml](https://github.com/squirrelchat/smol-toml) from 1.8.0 to 1.9.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/squirrelchat/smol-toml/releases">smol-toml's releases</a>.</em></p> <blockquote> <h2>v1.9.1</h2> <h2>What's Changed</h2> <ul> <li>fix: reject invalid extra exponent sign by <a href="https://github.com/cyyynthia"><code>@cyyynthia</code></a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/squirrelchat/smol-toml/compare/v1.9.0...v1.9.1">https://github.com/squirrelchat/smol-toml/compare/v1.9.0...v1.9.1</a></p> <h2>v1.9.0</h2> <p><strong>Huge update</strong>!!! This is most likely the largest update the library received since its release, with lots of new features and improvements.</p> <h3>Performance improvements</h3> <p>Significant parts of the internal parse logic have been rewritten, improving performance by <strong>1.5x-2x</strong>. The library was already comfortably ahead of the others, but it is now faster than ever, sitting at <strong>4x faster parse performance</strong> than the closest maintained implementation.</p> <p>Problematic code paths have also been replaced by safer implementations, solving potential DoS vectors. See GHSA-r4xh-jqrq-34v2.</p> <p>Note: the objects returned by the library now have a <em>null prototype</em>. This is a transparent change for 99.9% of users, and is one of the most significant contributors to the major performance gains in this version.</p> <h3>Full Temporal support</h3> <p>Version 1.8.0 brought support for Temporal in <code>stringify</code>; now the library is also able to emit Temporal objects instead of its own ad-hoc <code>TomlDate</code> object. It is not enabled by default, but it will become the default in v2. Enable by setting <code>useLegacyDate: false</code> in the parser's options.</p> <h4>Better Temporal support in stringify</h4> <p>Temporal support has been improved since it released: Temporal objects that cannot be represented (such as <code>Temporal.PlainMonthDay</code>) now throw an error (instead of silently emitting a bogus object).</p> <p>A new option has been added to <code>stringify</code> to disallow Temporal objects that cannot be fully represented in TOML. This includes <code>ZonedDateTime</code> objects with a IANA timezone attached instead of a plain offset, and dates with a specific calendar value set. Enable by setting <code>strictTemporal: true</code> in the options.</p> <h3>Handling of unsafe keys</h3> <p>Since its release the library has been protected against prototype pollution attacks, setting properties like <code>__proto__</code> using safe mechanisms that do not trigger prototype pollution. However, while the returned objects are safe on their own, they may become problematic if used carelessly.</p> <p>Inspired by <code>secure-json-parse</code>, the library now offers a way to either drop unsafe properties from the returned object, or to throw an error and reject documents altogether. By default, these potentially unsafe keys are preserved and returned.</p> <h3>Miscellaneous updates</h3> <ul> <li>Unicode BOM is now gracefully accepted and ignored.</li> <li>Table array headers are now properly checked again. Reported in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/65">#65</a>.</li> <li>Closed certain gaps where invalid whitespace would be accepted. Reported in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/61">#61</a>.</li> <li>Bogus local date and local time values with a UTC offset are no longer accepted.</li> <li>Certain error messages are more accurate and handle errors at line boundaries better.</li> <li>The default export of the lib is now formally deprecated; use a <code>import *</code> instead. Proposed in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/50">#50</a>.</li> <li>On Node 20+, strings that contain lone surrogates are now normalised to well-formed strings.</li> <li>On Node 20+, keys that contain lone surrogates are now rejected.</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.0">https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/squirrelchat/smol-toml/commit/69aa050fef018b71799519bb2cbb54f158a0c5f9"><code>69aa050</code></a> fix: actually properly fix exponent issues</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/7be7084d24c85909cec84ecaafe39737ef4e4472"><code>7be7084</code></a> chore: version bump</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/be3290da3309502447e020ce106bb8a11d99e580"><code>be3290d</code></a> chore: upgrade deps</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/86c76134ad7a50e8e101ef7fd82cb0f155b278d8"><code>86c7613</code></a> fix: reject invalid extra exponent sign</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/6f9739a5ab481f57c3112024bc4947534b580b58"><code>6f9739a</code></a> fix: gate <code>[is\|to]WellFormed</code> (Node 18 compat)</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/a73ca32c7c9237f744c880d900ad15307a0cef56"><code>a73ca32</code></a> fix: no Temporal with toml-test when Node < 26</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/7727890604580bd3fc2d7d2f79ad6e67b745ebe7"><code>7727890</code></a> chore: version bump</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/641903d60d47900080602b6a83e919d2a526547f"><code>641903d</code></a> chore: rewrite README.md</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/2df14c57945fec5a3737bc0cdce06a436bf9a77e"><code>2df14c5</code></a> fix(types): make it work if Temporal doesn't exist</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/3eaa44ef54b859990969935664b12023b26d0eae"><code>3eaa44e</code></a> chore: update benchmark harness</li> <li>Additional commits viewable in <a href="https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.1">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=smol-toml&package-manager=npm_and_yarn&previous-version=1.8.0&new-version=1.9.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Devin Foley <devin@devinfoley.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 18:46:09 -07:00
Huanyi Xie Codex Andrew Aymeloglu	082423000e	fix(claude-local): retain probe budget for unlimited runs (#15329 ) ## Thinking Path > - Paperclip starts Claude agents through the local adapter. > - The CLI engine checks its version and effort flag before a run. > - An unset run timeout resolves to zero and means unlimited. > - The probe code turns that zero into a one-second timeout. > - A valid CLI that starts slowly then fails the version gate. > - This pull request gives unlimited runs the existing 20-second probe cap. > - The probe can confirm a valid installed CLI before the run starts. ## Linked Issues or Issue Description Fixes: #15258. Related PRs #13927 and #14954 change version/model gates. This change fixes the timeout for the probes used by those gates. ## What Changed - Use a 20-second process timeout for version and effort probes when the run timeout is zero or negative. - Keep the existing cap and one-second floor for positive timeouts. - Add real slow-CLI regressions. ## Verification - A real fixture CLI prints its version and help after 1.5 seconds. The unlimited-timeout case fails on the original source and passes after the fix. - An explicit one-second timeout still stops both probes. - The complete Claude adapter source suite passes: 346 tests, 1 skipped. The command excludes generated `dist/` tests, as the repository runner does. - Full workspace typecheck and build pass in the verification checkout. - The full `pnpm test:run` attempt was stopped after it reported the unchanged rapid-Slack ordering assertion. A clean checkout at `f858207` reproduces it (3 comments, expected 8). Native V8 coverage shows that the functions changed by these three fixes are not called in that control. Additional broad checks reported timeouts and assertions in unmodified files; those extra failures were not all reproduced as clean-base controls; this is not a claim that the full repository suite is green. - Repository-wide typecheck and build were run with three independent fixes together. This PR includes only the Claude probe change. The affected source suites listed above pass. ## Risks Unlimited run settings now use a 20-second process timeout for capability probes. A probe for a CLI that hangs can therefore fail later. The real executable fixture runs on POSIX hosts and skips Windows; no live Claude account or remote provider was used. ## Model Used OpenAI Codex, a GPT-6-based coding agent, with reasoning, repository inspection, editing and local test execution. This session did not expose the exact deployed model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge CI and Greptile must finish on the updated head after removal of the documentation change. --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>	2026-10-14 16:24:41 -05:00
Nicky LeachandPaperclip	c30fb7d2b7	refactor(core): share Runner execution lifecycle (#16318 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core starts Runners in local, SSH, and sandbox environments. > - Coding runs, Dot runs, and setup probes repeat connection setup code. > - These paths must keep the same preparation, authentication, and cleanup order. > - This pull request gives them one execution lifecycle contract and one server connection driver. > - Core keeps the existing PRP control plane after connection. > - The benefit is one place to maintain environment routing before adding another execution provider. ## Linked Issues or Issue Description What existing behavior does this improve? Runner environment preparation and connection setup across local, SSH, and sandbox execution. Current behavior The coding transport owns the environment hook type. Dot borrows that type. The server repeats connection selection, registration, ingress activation, and release in task execution and setup probes. Changes to one path can miss another path. Proposed behavior Use a shared `RunnerExecutionDriver` contract. Route built-in task execution and remote setup probes through `createNativeRunnerExecutionDriver`. Keep provider session setup, process supervision, PRP commands, and recovery logic on their existing paths. Reason and benefit Environment code can prepare resources and provide a connection without adding another PRP execution path. The same driver preserves local loopback, remote outbound WSS, and authorized sandbox ingress. Breaking changes None intended. Existing flat SDK options remain compatible. This is an in-process contract. It does not add a plugin wire contract or Launcher APIs. Additional context Related: #15406 proposes plugin-provided Runner execution. This change starts from `master` and does not depend on that PR. It extracts the existing lifecycle before adding another provider. Related: #16125 covers remote environment support for external adapters; this PR does not implement that feature. ## What Changed - Extract and export `RunnerExecutionDriver` from the coding transport. Use it for coding and Dot options. - Share server connection negotiation across coding runs, Dot runs, and remote setup probes. - Preserve outbound validation before staging, capability checks before ingress, process dispatch before ingress connection, and verified process readiness. - Preserve warm run registration, restart guards, checkpoint hooks, and cleanup ownership. - Add 13 focused driver tests and document the lifecycle boundary. ## Verification - Targeted server execution and recovery tests: 728 passed, 35 skipped. These cover `native-session-executor`, `native-ssh-command-runner`, the new driver, warm restart transitions, and Runner restart recovery. - Targeted setup and driver tests: 109 passed after the review fix. Twelve driver tests overlap with the preceding count. The added test verifies that failed CA staging cannot leave a PRP route behind. - Runner connectivity tests: 11 passed. - Runner package lifecycle tests: 287 passed across coding transport, Dot, restart continuation, and settlement. These exercised the compiled Rust Runner. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `git diff --check`: passed. Local secret-pattern scan: clear. - The PRP transport runtime and remote process launcher remain unchanged. A byte comparison against the base confirmed this. - CI passed on `b10789dfce30a837ce890f44e352064c7a74366f`: all 55 active checks are green, including the full test shards, Runner checks, browser tests, typecheck, build, and aggregate gate. Greptile scored this commit 5/5 with no unresolved findings. - An earlier broad local shard run was not green: 34,888 passed, 12 failed, and 4,125 skipped. Failures were ten timeouts and two Git fixture assertions affected by local `init.defaultBranch=main`. Isolated retry passed all six non-route timeout suites (427 tests), two route suites, and both Git assertions with a test-process-only `init.defaultBranch=master`. The agent live-run route retry still timed out. No unrelated source changes were made. Further broad local runs were stopped at the user's request. Targeted commands: ```sh cd server node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-ssh-command-runner.test.ts src/services/native-runtime/native-runner-execution-driver.test.ts src/services/native-runtime/native-warm-transition-restart.integration.test.ts src/services/native-runtime/native-runner-restart-recovery.integration.test.ts --config vitest.config.ts node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/setup-readiness.test.ts src/services/native-runtime/native-runner-execution-driver.test.ts --config vitest.config.ts cd ../packages/paperclip-runner node ../../node_modules/vitest/vitest.mjs run src/live/runnerd-codex-transport.test.ts src/drivers/dot/runnerd-dot-driver.test.ts src/live/runnerd-restart-continuation.test.ts src/live/runnerd-codex-transport-settlement.test.ts --maxWorkers=2 cd ../.. node node_modules/vitest/vitest.mjs run packages/adapter-utils/src/runner-connectivity.test.ts ``` ## Risks - Startup ordering and connection cleanup are sensitive. Focused tests cover local registration, outbound validation, CA mapping, ingress ordering, release, and warm authority guards. - No live SSH host or sandbox provider was used. Existing mocked remote tests and real local Runner tests passed. - No database, PRP wire protocol, or Launcher API changes are included. ## Model Used OpenAI GPT-6 via Codex, with tool use and code execution. The served model variant and context window were not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the stated targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 12:58:05 -07:00
DottaandPaperclip	82884443be	perf(adapter-utils): skip host scans for unchanged restores (#16312 ) Skip the host snapshot scan when an unchanged workspace return matches its baseline, while retaining writer locking, both authority hooks, and changed-export conflict handling. Verified with 270 affected tests, final-head CI/E2E, Superagent, and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-14 12:54:31 -05:00
DottaandPaperclip	51b56706fb	fix(grok): verify file completion and recover failed sessions (#16305 ) ## Thinking Path > - Paperclip manages work and authority for AI agents. > - Grok Bot runs outside Paperclip and uses authenticated upload grants for files. > - Native completion requires a verified file publication receipt. > - The upload saved the file but did not create that receipt. Real Grok work could not finish. > - A failed run also left an assignment that the Stop action could not fence. > - This pull request connects verified uploads to normal completion and repairs that recovery action. > - The result preserves file integrity and current work-session authority. ## Linked Issues or Issue Description Refs: #16219, #16274. What happened? A real Grok Bot saved a task document and CSV. Semantic completion rejected the CSV because its upload had no verified publication receipt. After the managed run failed, Stop rejected its remaining assignment as a changed session. The driver also discarded the remote monitor's known failure reason. Expected behavior An authenticated upload of the exact requested bytes can satisfy normal completion. Stop can fence the remaining assignment of the selected failed run. An old Stop action cannot stop newer work. Known remote lease and reconnect errors remain visible without raw diagnostics or credentials. Steps to reproduce 1. Connect an experimental Grok Bot agent and verify its routine. 2. Assign a task that requires a document and a file. 3. Upload the file through its accepted assignment and cite the attachment in semantic completion. 4. Observe the missing publication receipt error on the previous implementation. 5. For recovery, select Stop after that run fails but its assignment remains active. Paperclip version or commit The live defects were observed after #16274. The current candidate merges master `739f9e40a4` and is `26c5b9601d1cb2b8f5bb7fcf730f019523763f5b`. ## What Changed - Record a server-owned publication receipt in the verified upload transaction. Bind it to the real native Grok run and exact stored file metadata. - Validate external publication against the actual upload and admitted assignment. Keep existing native and person upload rules. - Recover the committed receipt after a lost response. Reverify the same bytes when an older upload lacks proof. - Hold assignment authority through publication with grant-before-assignment and task-before-run lock order. Disconnect does not wait for remote file storage. - Allow Stop to fence the exact lingering assignment of a terminal run. Reject stale Stop calls for successor work. - Preserve only known lease/reconnect failure codes from the remote monitor. Do not expose raw stderr. - Extend real Rust, overlapping document/upload and disconnect/upload, file retry, completion and Stop regressions. Document setup, files and recovery. ## Verification - The real Rust Grok regression requests a file, checks an overlapping native document save, concurrent retries and lost-response recovery, rejects missing delivery evidence, then uses normal semantic completion and finalization. - Focused tests cover person MCP, native file handoff, completion, Stop and both external driver providers. - Local repository typecheck and production build pass on the current candidate. The five focused protocol suites pass (264 tests); all 17 external Runner driver tests pass, with the driver unchanged since the master merge. - Token gates and Storybook build pass. Nine existing Grok fixture journeys pass their browser interaction assertions; the static probe separately records five fixture API routes returning 404. This is UI fixture evidence only. - Full repository coverage passed in CI. The duplicate local `pnpm test:run` was interrupted after CI completed; its partial log is retained and is not reported as a local full-suite pass. The earlier aggregate invocation was interrupted for the second review fix. - All CI gates pass on `26c5b9601d1cb2b8f5bb7fcf730f019523763f5b`: 55 successful checks and two skipped Storybook jobs (no UI diff). This includes the complete sharded repository suites, native Runner, release checks, build and security scans. - Current-head Greptile is 5/5 with zero unresolved findings. The PR has no merge conflicts. - The prior live attempt is failed acceptance evidence. Real Grok requalification, proactive work, controller restart and active-session Stop remain release gates. This PR does not claim those passed. ## Risks - Upload and completion now share a durable run receipt. Lock order and current assignment authority must remain consistent with native mutations. - Publication fails closed if the file, run or grant changes. A retry must keep the same upload identity and bytes. - Person connections gain no agent authority. No database migration is required. - Known exit diagnostics do not prove global Grok termination. Grok spending remains unreported. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution and computer use. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 12:54:05 -05:00
DottaandPaperclip	adb69e8f09	feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683 ) ## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto \| paperclip \| legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass. The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board Try again and Resume recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal Approve completion reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 11:33:02 -05:00
Devin Foley devinfoley Claude Opus 5.5	bcfc03bc94	fix(sentry): keep provider-rejected Grok wakes out of run failure reports (#16297 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner can run an external Grok bot. Paperclip offers the bot an assignment and sends a webhook wake. The bot must accept the assignment in 10 minutes. > - When the Grok endpoint rejects every wake with HTTP 4xx, the bot never sees the work. The offer expires and the run fails with `Dot assignment acceptance or execution authority expired.` > - Paperclip already records each wake in `grok_bot_dispatches` with its HTTP status. The operator sees the rejection and can retry it. Thus Paperclip operates correctly in this case. > - But `reportRunFailure` sends this run to Sentry as an unexplained backend error. This makes noise and hides real defects. > - This pull request skips the Sentry report only when durable receipts prove that the Grok endpoint rejected every wake. > - The benefit is a cleaner Sentry signal. Local defects, uncertain delivery, and accepted work stay visible. ## Linked Issues or Issue Description Sentry issue `PAPERCLIP-BACKEND-3M` (staging, 1 event). What happened? A Grok bot run on staging failed after 10 minutes in the `preparing` stage. Sentry got the error `Dot assignment acceptance or execution authority expired.` with the code `native_provider_terminal_failed`. The stored receipts show two `mailbox_updated` wakes for the assignment. The Grok endpoint rejected both wakes with HTTP 400: the first wake and one operator retry. The assignment was never accepted. Wakes with the same shape got HTTP 200 before and after this run. Expected behavior The run fails, and the operator sees the rejected delivery. Sentry does not get a backend error for a proven external rejection. Steps to reproduce 1. Connect a Grok bot agent and verify its routine. 2. Make the Grok endpoint return HTTP 400 for work wakes. 3. Assign a task to the agent and wait 10 minutes. 4. See that the run fails and that a Sentry event is sent. Paperclip version or commit `f69c6a8` (master). ## What Changed - `server/src/services/run-failure-report.ts`: Add `isRejectedExternalBotWake`. It returns true only when all of these conditions are true: - The run failed in native mode with `native_provider_terminal_failed`. - The finalization `failureDetail.originalFailureCode` is `dot_assignment_expired`. - The run has exactly one Dot assignment, and that assignment was never accepted. - The assignment has at least one `mailbox_updated` wake, and each wake has the status `rejected` with an HTTP 4xx status. - `captureTerminalRunFailure` skips the Sentry capture when the check is true. If the receipt check itself fails, it logs a warning and the failure stays reportable. The run status, the task, and the operator recovery do not change. - `packages/db`: Add the index `grok_dispatch_assignment_kind_idx` on `grok_bot_dispatches (assignment_id, kind)` (migration `0346`). The receipt lookup and the dispatcher's per-assignment checks use this index. - `server/src/services/__tests__/run-failure-report.test.ts`: Add one test for the skipped case. Add nine tests for cases that must stay reportable: no wake, a local rejection with no HTTP status, a delivered wake, uncertain delivery, accepted work, a different provider failure code, no finalization, more than one offer, and a receipt read that fails. ## Verification - `pnpm --dir server exec vitest run src/services/__tests__/run-failure-report.test.ts`: 208 tests pass. - I removed the new catch and ran the tests again. The receipt-read-failure test failed. - `pnpm --filter @paperclipai/db run check:migrations` and `src/migration-snapshot-drift.test.ts` pass. - I removed the source change and ran the new tests again. The skipped-case test failed and the six reportable cases passed. This shows that the test proves the change. - I did not run the full `pnpm -r typecheck` locally. The server `typecheck` script needs `cargo` to build the Runner binary, and the sandbox stopped `tsc` with SIGKILL. CI runs the full typecheck. ## Risks - Low risk. The change only removes one Sentry report. It does not change run state, recovery, or what users see. - The check uses durable receipts, not message text. A wake with no HTTP status (local validation or credential failure) stays reportable, so a Paperclip defect cannot hide behind this rule. - For failed native runs with `native_provider_terminal_failed`, the check adds up to three indexed reads. Other runs do no extra reads. - Migration `0346` builds one index in a transaction. `grok_bot_dispatches` was created in `0344` and is small, so the build lock is short. The table is not on the migration gate's large-table list. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Agentic tool use with shell, code search, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 08:32:48 -07:00
Devin Foley Devin Foley Claude Opus 5.5	14f961b698	fix(runner): report Runner disconnect age when a Dot bridge exits (#16296 ) DurablePrpControlPlane records when the last authenticated Runner connection closes and exposes runnerDisconnectedForMs(). RunnerdDotDriver appends disconnectedMs=<n\|none> to dot_runner_process_exited_recovery_required errors. Diagnostic only; authority, classification and recovery are unchanged. Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 08:26:32 -07:00
DottaandPaperclip	ebeac833a6	perf(runs): bound history and reduce repeated context reads (#16286 ) ## Thinking Path > - Paperclip helps operators manage AI agents and inspect their work. > - Run history supplies recent execution records to the board and API clients. > - The list repeats eight reads of a large stored JSON context. > - A caller can also omit the limit and read the complete history. > - This pull request expands context fields once and bounds every list read. > - The benefit is lower database cost and bounded redaction work. ## Linked Issues or Issue Description Refs #958. Related community reports: #13962 and #10631. This is an independent implementation. It does not use their commits or code. It covers the history bounds reported in both. Process memory monitoring from #10631 is outside this change. What happened? The run-history query repeatedly extracts fields from large context JSON. Requests without a limit also load every historical run before authorization and redaction. Accumulated staging query statistics showed 271 ms for summary history and another 58 ms for its redaction lookup. What did you expect to happen? Normal reads must use a bounded recent history. Context projection must preserve the existing values and privacy behavior with less repeated database work. Steps to reproduce Run the disposable benchmark below against the baseline revision. It creates 1,400 synthetic runs with 192 KiB contexts. It compares the exact service SQL and every returned column for both summary and full history. ## What Changed - Expand context fields with one lateral record projection. Preserve non-object legacy contexts and the first-class issue binding. - Enforce a 200-row default and a 1,000-row maximum in the route and service. - Apply the default in the shared UI client, including both agent detail implementations. - Preserve response arrays, authorization, result projection and secret redaction. - Fetch older linked runs by ID in both agent pages. Check company and agent scope, navigation races and visible retry errors. - Update OpenAPI, data-path documentation and capability maps. - Add real PostgreSQL and HTTP regressions plus a reproducible benchmark. ## Verification - Local benchmark: summary history 37.72 ms to 19.34 ms; full history 37.84 ms to 19.01 ms. All 200 complete positional rows match in each comparison. These are synthetic local SQL timings, excluding HTTP and redaction. - History, route, privacy and redaction suites: 160 server tests passed across six files. Twenty-two UI API, selection, run-panel and log-visibility tests passed. Both agent-page variants render an older linked run with an empty history page on desktop and mobile. - `pnpm -r typecheck` and `pnpm build` passed. - Map validation and `git diff --check` passed. - All current-head CI and security checks passed, including the full general and serialized suites, browser shards, Runner verification and build. Greptile reviewed the exact commit at 5/5 with no remaining findings. - The full local sweep began before the final compatibility fixes. It was stopped after the final commits passed complete CI. The focused local suites listed above passed on the final code. Before interruption, the broader local sweep passed 31,743 tests across the general server, UI and several package groups, with no reported failures. The full local command is not claimed as a completed pass. ```sh pnpm --filter @paperclipai/server exec vitest run src/services/heartbeat/run-state.test.ts src/__tests__/heartbeat-list.test.ts src/__tests__/agent-live-run-routes.test.ts src/__tests__/run-secret-redaction.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts pnpm --filter @paperclipai/ui exec vitest run src/api/heartbeats.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-run-history.ts --baseline-ref=1e731520bb ``` ## Risks - Clients that relied on an unbounded array now receive 200 rows by default. Explicit limits allow up to 1,000. This change does not add pagination. - Legacy context conversion must stay identical. Real database regressions cover null, array, scalar, nested values and native issue precedence. - No schema migration or persistent cache is introduced. Redaction and access checks remain active. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, shell tools and local code execution. The runtime does not expose a more specific backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 10:14:41 -05:00
DottaandPaperclip	df4ee06e2f	feat(skills): select and validate tracked GitHub branches (#16288 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Skills can track a branch in an authorized GitHub repository. > - Sources previously offered only a default-branch repair action. > - People need to select a listed branch or enter a branch that GitHub has not listed. > - The server must validate the selected ref with the caller's authorized connection. > - This pull request connects the approved branch picker to production APIs. > - Installed skills remain usable while the source changes its tracked ref. ## Linked Issues or Issue Description Subsystem affected Skills UI, shared API contracts, GitHub source services, and agent API discovery. Problem or motivation The default-branch repair action does not let people choose another branch. The settings service also validates the old ref when a caller supplies a new ref other than HEAD. A missing tracked branch can prevent connection repair even when the selected account has repository access. Proposed solution Load a bounded branch list with the authorized connection. Mark the default branch. Search listed branches or accept a typed ref. Validate the selected ref before saving. Keep the chosen value after a load error, invalid ref, or concurrent update. Save the selected connection and branch together during connection repair. Alternatives considered A default-branch button does not support a named branch. A text field alone does not help people find the repository default or available branches. Roadmap alignment This improves the existing Skills source manager. It does not add a new skill import system. Additional context Refs #16066 and #16256. This is the production implementation of the approved branch-selection Storybook journeys. ## What Changed - Add an authorized branch-list endpoint and shared response/query types. Register it in agent API discovery. - Validate the requested ref, including the returned commit SHA. Preserve source leases, revision checks, project ownership, skill identities, selections, and installed versions. - Add the searchable branch picker to Sources and connection repair. Allow typed refs during branch discovery and after discovery fails. - Keep save errors in the dialog. Retain the selected branch through a concurrent source update. - Use the production components and API client in Storybook. Add keyboard, touch, loading, retry, invalid-ref, connection-repair, and concurrent-update browser journeys. - Update product documentation and the feature/access maps. ## Verification - Red/green: two new database regressions fail against the previous service and pass against the implementation. - Linux database/service suite: 85 tests passed. This includes source visibility, human/agent authorization, selected connections, immutable versions, and invalid refs. - Rebased branch and review fixes: 94 focused UI, route, branch-validation, and fixture tests passed. Two conflict-message regressions and the stale-retry fixture regression fail before the fix and pass after it. - Storybook build passed. All 26 focused browser journeys passed. - Repository typecheck and build passed. Final UI typecheck passed after the review fixes. - The full local Vitest run was interrupted when the checkout and its dependencies disappeared. It reported missing working-directory/Postgres files and worker startup errors. Its two timed-out tests pass after the checkout was restored. The complete CI suite passed on the final head, `cf5d4c8f0a`: [CI run](https://github.com/paperclipai/paperclip/actions/runs/38571068332). One unchanged workspace timestamp test required a single rerun. Its release-minus-one-millisecond value can equal the original confirmation time. The rerun passed. - Live acceptance passed on implementation head `5dd935bf14` in an isolated macOS arm64 instance with Node 24.21.0 and an authorized private GitHub repository. The UI loaded the actual default branch, saved branch and selected connection together, and preserved the installed skill/version IDs after reload. An invalid typed ref retained the draft and showed the provider error. A valid typed HEAD ref then saved successfully. The owned runtime and database were stopped after the test. Focused unit, Linux database, and browser suites passed again after the error-handling review fixes. - Greptile reviewed final head `cf5d4c8f0a` at 5/5. Both review threads are resolved; the fresh review has no remaining findings. - Token gates and map/catalog checks passed. - Review entry points: `Skills / Project repository skills / 03 Branch selection components` and `04 Branch selection journeys`. Run `pnpm storybook` to inspect them. - Browser command: `pnpm exec playwright test --config tests/storybook-visual/playwright.config.ts tests/storybook-visual/skill-source-branch-selection.spec.ts tests/storybook-visual/project-repository-skills.spec.ts`. - Manual check: open an independent GitHub source, choose a new branch, select the default or type a ref, save, and reload. Try an invalid ref and confirm the selection remains in the dialog. For a project-bound source, change its ref in project settings. ## Risks - GitHub discovery is bounded to 100 listed branches. The default branch is always included. Typed refs remain available. - A source can change while the dialog is open. The API rejects a stale revision and the UI keeps the draft for retry. - A ref change schedules the existing refresh pipeline. Installed versions remain available until a successful refresh imports new content. - No database migration or credential storage change is required. Existing HEAD requests remain supported. ## Model Used OpenAI Codex, GPT-6 family. The serving model's exact identifier and context-window size are not exposed in this session. Used reasoning, repository tools, code execution, and browser tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 09:55:43 -05:00
DottaandPaperclip	e9b6c4d44c	perf: remove history scans from task and chat queries (#16278 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task reads and external chat notices must remain responsive as run history grows. > - The chat notice sweep casts indexed UUID columns to text. > - Run redaction and task provenance filter large JSON contexts with expressions that lack matching indexes. > - These queries scan history and read large stored contexts on common requests. > - This pull request adapts two existing fixes to current master and adds the missing provenance index. > - The benefit is less database work with the same query results and access rules. ## Linked Issues or Issue Description Refs #15927. This change adapts that chat sweep fix to the extracted inbound notice module and retains the contributor credit. Refs #12499. This change adds that missing nested-context index through a current, concurrent migration. It retains the contributor credit. This combined change replaces those older layouts for the measured query paths. It also covers task creation provenance, which those pull requests do not address. What happened? The chat notice sweep scans wakeup history. Issue redaction scans run contexts because only one branch of its JSON condition has an index. The task creation-source filter also scans contexts because its native and legacy ID expression lacks an index. Expected behavior The queries use indexed lookups and return the same rows. Company access, task origin precedence, secret redaction, and chat notice ordering remain the same. Steps to reproduce Run `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-slow-queries.ts --baseline-ref=f8af817f8d`. The benchmark creates disposable PostgreSQL data and compares the old query with the current query on the same fixture. ## What Changed - Compare chat UUID columns directly with safely parsed payload IDs. Reject malformed and noncanonical IDs without a cast failure. - Add the nested `paperclipIssue.id` run-context index and the exact native/legacy task-provenance expression index. - Build both indexes concurrently. Register the migration for top-level execution and invalid-index retry repair. - Extend chat payload and migration regression coverage. - Add a repeatable synthetic SQL benchmark and document its scope and command. - Keep the five newest schema snapshots. ## Verification - Focused local checks: 76 selected tests passed in five files. These cover FIFO notices, malformed IDs, conversation state, redaction, provenance, and company boundaries. - Disposable PostgreSQL 18.1 fixture: 335 tasks, 1,400 runs, 13,000 wakeups, 377 actions, and 192 KiB synthetic contexts. - Five warm samples per query, after two warmups. Every returned SQL column matched before and after, compared as positional tuples so duplicate unnamed projections are retained. \| Query \| Before, median \| After, median \| Improvement \| \| --- \| ---: \| ---: \| ---: \| \| Chat notice candidates \| 26.21 ms \| 6.43 ms \| 4.1× \| \| Task creation-source list \| 159.69 ms \| 6.51 ms \| 24.5× \| \| Issue redaction contexts \| 131.88 ms \| 0.34 ms \| 389.1× \| The run-history context projection returned the same 200 rows and kept the same index plan and buffer counts. Its timings were noisy (37.30 ms before and 56.78 ms after in the last run; 34.25 ms and 34.14 ms in the previous run). No improvement is claimed for this control. Its repeated JSON projection remains a separate optimization opportunity. The populated plans use the wakeup primary key, the provenance expression index, and a BitmapOr over both redaction indexes. These are local SQL measurements. They do not prove an equivalent change in hosted API latency. - `pnpm -r typecheck` and `pnpm build` passed locally. - The full local `pnpm test:run` was started and then interrupted (exit 130) when the Mac reached its PostgreSQL shared-memory limit. It did not complete and is not reported as passed. The final SQL benchmark passed after its disposable database resources were released. The full general and serialized test suites passed in CI. - All CI checks for final head `193acca6abaa992acad12f4b631bd31dd3234641` passed, including full Vitest coverage, E2E checks, typecheck, build, migration order, and lifecycle boundaries. - Greptile reviewed that exact head at 5/5 with no actionable comments. ## Risks - Both indexes add storage and write cost. Initial builds read stored run contexts and can take time on large histories. - Concurrent index builds avoid holding a write-blocking table lock for the whole build. The existing migration retry path removes invalid partial indexes before retrying. - The chat parser accepts only canonical lowercase UUID text. This preserves the prior text comparison, including rejection of uppercase and malformed values. - Staging has not been migrated or deployed. Local PostgreSQL and synthetic data do not reproduce hosted load or network latency. ## Model Used - OpenAI Codex, based on GPT-6. The exact serving model ID and context window were not exposed in this session. Used reasoning, repository tools, browser inspection, and local code execution. The adapted fixes retain their original contributor credits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 07:37:51 -05:00
DottaandPaperclip	c36281d639	fix(skills): repair missing branches and show grouped import progress (#16256 ) ## Thinking Path > - Paperclip lets people manage AI agents and their reusable skills. > - GitHub skill sources must keep their installed content usable during updates. > - A source can track a branch that no longer exists. > - The UI treats that GitHub error as a connection problem, so saving a valid account does not resolve it. > - Project workspaces also show the same repository several times, and long imports give little feedback. > - This pull request adds explicit branch repair, groups repository imports, and shows import progress. > - People can repair the actual problem and see when their import is working. ## Linked Issues or Issue Description Refs #16066 and #14713. This fixes the Sources experience introduced by those changes. Related adapter-source PRs #14340 and #14341 cover runtime mounting, not this source-management flow. No overlapping source-management fix was found. What happened? An installed source tracked `main` while its GitHub repository used `master`. GitHub returned HTTP 422. The UI showed a general read error and asked for a new connection after a valid account was saved. The project repository list repeated the same repository/ref for each project. Import had no spinner or explanation of queued work. Expected behavior Identify an unavailable branch and offer an explicit repair. Retain the selected connection and installed versions. Show one repository/ref row with its projects. Give immediate feedback and show the queued and active import states. Steps to reproduce 1. Install a source with an immutable commit and a saved tracked branch that does not exist. 2. Refresh it, or try to save a valid GitHub connection. 3. Add the same repository/ref to two legacy project workspaces and open Sources. 4. Import that repository and observe the row until installation completes. ## What Changed - Distinguish a missing GitHub revision from access denial and temporary read errors. - Validate both repository identity and the tracked revision when saving a connection. - Offer Use default branch on independent sources and in connection repair. Keep `HEAD` as the default-branch alias after refresh. Preserve skill IDs, versions and selections. - Keep active project refs under project configuration. Reject source-level changes that would alter their tracking. Recheck the ref under the source lock when project attachment races with branch repair. - Group matching repository/ref workspaces. Adopt eligible workspaces and skip pending workspaces. Keep remaining project links inside the existing source card. Allow remaining imports while another workspace is pending. - Show a spinner, busy state and queued/importing text. Keep the saved authorized account selected in connection repair. - Update shared contracts, feature/access maps, database notes and production-component Storybook journeys. ## Verification - Red/green: three local regressions failed on the original code: missing-ref classification, duplicate repository rows, and branch repair instead of another connection prompt. The changed UI/reader tests pass after the fixes. - Live local app with a real authorized GitHub credential: reproduced HTTP 422 for the unavailable branch; used the UI to restore default-branch tracking; confirmed the same connection, skill ID and single immutable version remained. - Live local grouped Paperclip import: one source, two project associations and 52 installed skills. Sources survived reload. No staging data was changed. - All 16 project repository Storybook browser journeys pass on the final head, including keyboard focus recovery after branch repair. These include branch repair, grouped imports, connection selection, recovery, removal and responsive layouts. - Repository typecheck, production build, Storybook build, token gates and 28 map/catalog tests passed. Final UI typecheck and build also passed after grouping remaining project links. - Linux qualification: all 94 focused source, project import, GitHub reader and UI tests pass with supported Node 24. No database tests were skipped. Both review findings failed before their fixes and pass after them. - The full local `pnpm test:run` attempt was stopped after macOS database suites exhausted the host shared-memory limit. The complete Linux CI matrix is the full-suite gate. The complete Linux CI matrix passes for head `f886e3ac073ec1c6f051fd6429496234f5143694`. - Greptile rates head `f886e3ac073ec1c6f051fd6429496234f5143694` 5/5. Both findings are fixed and all review threads are resolved. - Storybook: `skills-project-repository-skills-02-journeys--missing-branch` and `--duplicate-projects`. ## Risks - Default-branch repair changes the tracked branch only after an explicit user action. Existing execution version pins stay intact. - A selected connection now needs access to the tracked revision, not only repository metadata. - Grouping is by repository and ref. Project access and provenance remain separate and are checked by the existing services. - Legacy source errors contain safe text. The repair-action classifier maps known errors and treats other failures as retryable. No credentials or provider response bodies are returned. - No schema migration, workflow change, lockfile change or staging deployment. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model identifier and context-window size are not exposed in this session. Tool-assisted implementation, code execution and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-13 22:27:11 -05:00

1 2 3 4 5 ...