mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 09:50:48 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs Claude and Codex agents from one shared provider pack. The pack pins an exact Claude Agent SDK (which bundles Claude Code) and an exact Codex CLI. > - #22222 added Claude Fable 5.2 (minimum Claude Code 2.1.321) and GPT-6.2 Sol and Astra (minimum Codex 0.169.1). The pack still bundles Claude Code 2.1.286 and Codex 0.160.0. > - Because of this, Paperclip Runner stops these models with a named version error, and native remote Codex accepts only `>=0.149.0 <0.161.0`. > - This pull request moves the provider pack to Agent SDK 0.3.328 (Claude Code 2.1.328) and Codex 0.170.0. It moves every constant that depends on these pins, and the remote Codex window to `<0.171.0`. > - The benefit is that Fable 5.2 and GPT-6.2 run on Paperclip Runner after a sandbox image with this pack is promoted. ## Linked Issues or Issue Description Refs #22222 (adds the models and the version floors; this pull request makes them runnable on Paperclip Runner). Refs #18488 (moves only the Codex pin to 0.166.0 and the window to `<0.167.0`; it is not updated since November 1 and has conflicts with `master`). Refs #18486 (GPT-6.1 Terra needs Codex 0.165.0; Codex 0.170.0 satisfies it). **Overlap with #18488.** #18488 changes the same Codex pin locations. This pull request uses a newer pin (0.170.0) and also moves the Claude runtime. If this pull request merges first, #18488 has no remaining change. I did not change the #18488 branch. ## What Changed - Claude runtime: `@anthropic-ai/claude-agent-sdk` 0.3.286 → 0.3.328 (Claude Code 2.1.286 → 2.1.328). This changes the root `pnpm.overrides`, `pnpm-workspace.yaml`, the ACPX `claude` profile, and the qualified Claude runtime in `installation-integrity.ts`, with new executable digests for Linux x64, Linux ARM64, macOS ARM64, and macOS x64. - Codex runtime: `@openai/codex` 0.160.0 → 0.170.0. This changes the runner dependency, the `codex-acp@1.6.2>@openai/codex` override, the `codex-acp` patch dependency line, the ACPX `codex` profile, and the qualified Linux x64 runtime with its new executable digest. - The ACP bridges do not change: `claude-agent-acp` 0.73.0 and the patched `codex-acp` 1.6.2. Their `commandDigest` values and profile versions do not change. - Generated declarations: `generated-profiles.ts` and `generated_acpx_profiles.rs` are regenerated with `generate:acpx-profiles`. - Remote native Codex window: `REMOTE_CODEX_MAXIMUM_VERSION_EXCLUSIVE` 0.161.0 → 0.171.0. The minimum stays at 0.149.0. Images with Codex 0.149.0 through 0.170.x are accepted. - Codex company observability: the qualified-version check moves to `codex-cli 0.170.0`. The `otel` configuration keys that Paperclip writes (`exporter`, `trace_exporter`, `metrics_exporter`) are present in the 0.170.0 configuration schema. The `OtelExporterKind`, `OtelHttpProtocol`, and `OtelTlsConfig` definitions are identical to 0.160.0. - Daytona runner image probes (`Dockerfile`, `Dockerfile.release`) expect `codex-cli 0.170.0` and `2.1.328 (Claude Code)`. The remote npm install spec in `server/src/adapters/registry.ts` is `@openai/codex@0.170.0`. - Tests: fixtures and assertions that mirror the pins use the new versions. The window tests accept 0.161.0 through 0.170.99 and reject 0.171.0. The ACPX gate tests now show that Fable 5.2 and GPT-6.2 are admitted. A new test mocks an older bundled runtime and keeps the named version-gap errors covered. The fallback test shows that GPT-6.2 stays on 0.170.0 and still falls back on 0.160.0. - Opt-in qualification scripts (`scripts/tests/native-cleanup-paginated-codex.mjs`, `native-provider-startup-codex.mjs`) require Codex 0.170.0. - Documentation: `docker/daytona-runner/README.md`, `doc/architecture/paperclip-runner-compatibility.md`, `doc/agent-observability.md`, `docs/deploy/environment-variables.md`, and the deferred item in `doc/adapter-model-audit-2026-11-18.md`. - `pnpm-lock.yaml` is not part of this pull request. The refresh bot regenerates it on `master`, as for the previous pin refreshes (#14942, #14993). ## Verification Sources, checked on 2026-11-19: - npm registry: `@openai/codex@0.170.0` (published 2026-11-18T03:45Z) and `@anthropic-ai/claude-agent-sdk@0.3.328` (published 2026-11-18T23:13Z, `claudeCodeVersion` 2.1.328). Both are the `latest` tags. - [Codex releases](https://github.com/openai/codex/releases) 0.161.0 through 0.170.0. The notes record no app-server protocol break. 0.169.0 keeps project configuration disabled for `codex exec` in untrusted directories; Paperclip's CLI path already uses `--skip-git-repo-check`, and the native path uses `app-server`. - [Claude Code changelog](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md) 2.1.287 through 2.1.328. See "Risks" for the headless changes that I reviewed. Digests and package shape: - I downloaded each platform archive with `npm pack`, which verifies registry integrity, and computed the SHA-256 of the executable. As a method check, the same command on the 0.3.286 Linux x64 archive gives the digest that is on `master` (`fe503f65…`). The Codex 0.160.0 archive also gives the `master` digest (`12eb3e81…`). - The extracted executables report `codex-cli 0.170.0` and `2.1.328 (Claude Code)`. - The Codex 0.170.0 Linux x64 archive no longer contains `codex-resources/zsh/bin/zsh`. Paperclip does not reference that path. - The Agent SDK 0.3.328 `optionalDependencies` have the same platform set as 0.3.286. The Codex 0.170.0 `optionalDependencies` use the same `npm:@openai/codex@<version>-<platform>` aliases. - Bridge link check: I installed `@agentclientprotocol/claude-agent-acp@0.73.0` with an override to Agent SDK 0.3.328 and imported its entry module. Node linked all of the named SDK imports, and the import completed. Real Codex qualification (no model turn, no credentials): - `PAPERCLIP_TEST_CODEX_BINARY=<installed 0.170.0 Linux x64 binary> node --import ./server/node_modules/tsx/dist/loader.mjs scripts/tests/native-cleanup-paginated-codex.mjs` passed: `green: true`, with the expected red control and paginated `thread/resume` through the real 0.170.0 `app-server`. The installed binary digest matches the pinned digest. Focused suites (Linux x64, Node 25.9.0, local `pnpm install --no-frozen-lockfile`): - `cd packages/paperclip-runner && pnpm run typecheck:typescript` passed, including `check:acpx-profiles`. - `npx tsc --noEmit` passed in `packages/adapters/codex-local` and `packages/adapters/claude-local`. - `packages/adapters/codex-local`: 625 passed. `packages/adapters/claude-local`: 419 passed, 1 skipped. - `node --test`: `acpx-codex-package-contract` 9 passed, `controlled-evals/controller` 77 passed, `scripts/acpx-patch-packaging` 26 passed, `scripts/__tests__/grok-public-install-sandbox` 27 passed. - `server`: `acpx-qualification`, `codex-runtime-compatibility`, and `codex-model-fallback` 110 passed. `native-session-executor` 1541 passed, 1 failed. The failure (`compares export process compatibility … db.transaction is not a function`) also fails on unchanged `master` on this host. - `packages/paperclip-runner` vitest (`installation-integrity`, `qualified-profiles`, `codex-security-config`, `native-execution`, `native-backend-factory`): 283 passed, 25 failed. All 25 failures are in `installation-integrity.test.ts` with "ACPX provider lifetime guardian failed to start". The same 25 tests fail on unchanged `master` on this host (master fails 30 here). Rust (toolchain 1.97.1 from `rust-toolchain.toml`, Linux x64): - `cargo test -p paperclip-runner-core --lib`: 658 passed, 2 failed, 4 ignored. `warm_run_attachment_reuses_the_provider_ingress_listener` also fails on unchanged `master` on this host. `reaping_stops_a_dead_runners_provider_and_its_escaped_tool_job` is flaky here: on this branch it passed 2 of 3 reruns. - `cargo test -p paperclip-runner-core --test acpx_warm_attachment --test native_provider_backend`: 7 and 45 passed. - The new and changed saved-session tests pass: `a_session_saved_by_the_previous_agent_runtime_is_refused_by_name`, `unrestorable_saved_sessions_keep_strict_bindings_and_name_their_class`, and `a_settled_session_saved_by_the_previous_acpx_release_continues`. `rustfmt --check` passes. - `server`: `native-session-resume.test.ts` and `acpx-qualification.test.ts` 110 passed, 2 skipped. Both new resume cases run and pass. Not run here: - `native-provider-startup-codex.mjs`, because it needs a compiled `paperclip-runnerd`. The trusted CI runner workflow covers startup and session-resume probes. - The Docker image build and any live model turn. A live Fable 5.2 and GPT-6.2 test follows the staging image rollout. ## Saved sessions from the previous release This change keeps the ACPX bridge command digests and profile versions. It changes `agentRuntimeVersion` in the `claude` and `codex` ACPX profiles. That field is part of the saved session identity. The previous release saved sessions with Agent SDK 0.3.286 or Codex 0.160.0. This build refuses those saved sessions. It does not migrate them. - **Next run (the usual case):** the saved provider descriptor names the old runtime. `rebindNativeSessionCheckpoint` does not match it, so the next run starts a fresh provider session in the same workspace with the complete handoff prompt. No run fails. The earlier provider conversation does not carry over. - **A run in flight at the deploy (settled or mid-turn):** the runner restore refuses the saved descriptor with the permanent class `provider_state_incompatible`. The server reports it as `native_provider_state_incompatible` and stops automatic retries. The runner opens no provider and keeps the saved state bytes and the workspace. The next run on the task starts a fresh provider session. - **Native Codex (`codex_app_server`) and OpenCode** sessions do not record this field, so their resume decision does not change. After a new image is promoted, native Codex resumes a saved thread with Codex 0.170.0 instead of 0.160.0. This pull request does not test that cross-version thread resume. New tests: - `acpx_provider_backend.rs`: `a_session_saved_by_the_previous_agent_runtime_is_refused_by_name` covers settled Claude and Codex sessions and a Claude turn interrupted mid-run by the deploy. `unrestorable_saved_sessions_keep_strict_bindings_and_name_their_class` has a new `agent_runtime_release` case. Each case checks the named class, that no provider session opens, and that the saved bytes do not change. - `native-session-resume.test.ts`: "a session saved by the previous Claude or Codex ACPX runtime" checks that a same-release session resumes, and that a session saved with Agent SDK 0.3.286 or Codex 0.160.0 starts fresh in the same workspace. ## Risks - Medium. Every Paperclip Runner Claude and Codex run uses the new executables. The digests, generated profiles, image probes, and window must agree. They agree locally, and CI runs the Rust and runner suites again. - Claude Code 2.1.287 through 2.1.328 change some headless defaults. I checked each one against Paperclip: - 2.1.302: `-p` and Agent SDK sessions with no permission mode start in `default` mode on third-party providers. Paperclip always sets a mode (`bypassPermissions` or `default`), so this does not apply. - 2.1.299: MCP allow rules now match only the whole server name. Paperclip already writes `mcp__<server>__*` and exact `mcp__paperclip__<tool>` rules. - 2.1.302: fork subagents are on by default in `-p` and the Agent SDK, as in the terminal. This pull request keeps the upstream default and does not set `CLAUDE_CODE_FORK_SUBAGENT=0`. Reviewers can ask to keep them off. - 2.1.299 and later: stdio MCP connections negotiate MCP protocol 2026-07-28 first and fall back to the older protocol for servers that do not answer. - Agent SDK 0.3.328 is less than one day old. If a regression appears, revert this pull request to return to 0.3.286 and Codex 0.160.0. - Sandbox images built before this change still carry Codex 0.160.0 and Claude Code 2.1.286. They stay accepted. On those images, Fable 5.2 and GPT-6.2 keep the existing named errors and fallbacks until a new image is promoted. - Until the refresh bot updates `pnpm-lock.yaml` on `master`, a local `--frozen-lockfile` install fails, as with the previous pin refreshes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude, model ID `claude-opus-5-5`, through Claude Code with tool use (shell, file edit, `gh`) and code execution. The context window and reasoning mode are not reported in this session. - The registry archives, digests, release notes, configuration schemas, and changelog were read directly. No model output was used as a source for those facts. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
159 lines
10 KiB
Markdown
159 lines
10 KiB
Markdown
# Company agent observability
|
|
|
|
Company Settings → General → Agent observability configures one company-owned
|
|
OTLP destination for Paperclip-managed Codex, Claude Code and native OpenCode executions. Export
|
|
is disabled by default. Choose Braintrust (US) or a custom HTTP(S) OTLP base URL;
|
|
use your own collector when you need fan-out. Braintrust EU and self-hosted
|
|
installations can use Custom OTLP with their required routing/authentication
|
|
headers. The receiver must be reachable from every execution environment;
|
|
`localhost` refers to the execution host, including a remote sandbox.
|
|
|
|
This is separate from Paperclip's first-party Telemetry and its content-free
|
|
[infrastructure tracing](observability.md). Configuring it does not change either data path.
|
|
|
|
## Capture and supported runtimes
|
|
|
|
The qualified versions are Codex 0.170.0 and Claude Code through ACP bridge
|
|
0.73.0 with Agent SDK 0.3.328. Both logs and traces use HTTP/protobuf. Prompts,
|
|
assistant responses and tool content default on. Additional tool details are a
|
|
Claude Code and native OpenCode control; Codex has no equivalent switch. Codex requests up to 64 KiB
|
|
of tool output (or zero when disabled); native event availability, truncation
|
|
and response caps still apply. Claude Code applies its own native limits.
|
|
Metrics, raw API bodies and Codex Guardian assessments are disabled.
|
|
|
|
Native OpenCode 1.18.34 uses a runner-owned HTTP/JSON exporter. Each completed,
|
|
failed or cancelled turn produces a trace with Paperclip company, agent, run and
|
|
turn IDs. Authenticated Paperclip tool dispatch outcomes, native tool outcomes and
|
|
message-level token/cost receipts are child spans;
|
|
repeated usage updates replace the same message receipt. Missing prices remain
|
|
unknown. This projection excludes hidden reasoning, raw provider payloads and
|
|
system instructions. Prompt, response, tool-detail and tool-content switches
|
|
apply before export. Content is bounded to 8 KiB per field and 64 tools plus 64
|
|
usage records per turn; truncation is marked. With both tool-detail and tool-content
|
|
capture enabled, Paperclip tool spans include `braintrust.input` diagnostics
|
|
(`paperclip.opencode.tool-diagnostics.v1`): the upstream `tools/call` method,
|
|
argument JSON type and types for a fixed list of envelope fields. These are
|
|
shape diagnostics, not a copy of the arguments. Arbitrary field names, argument
|
|
values, nested payloads and full model request bodies are not captured.
|
|
Authenticated schema rejections also produce failed tool spans without invoking
|
|
the controller handler. Conflicting request IDs produce separate failed items,
|
|
so a rejection cannot overwrite a previous dispatch outcome.
|
|
Paperclip tool spans use authenticated dispatch IDs and the name `tool <name>`.
|
|
Provider SSE records for those declared tools use `opencode tool <name>` and
|
|
remain available when failure occurs before dispatch, such as an oversized
|
|
request. These are separate observations, not correlated calls. The
|
|
`paperclip.tool.source` attribute distinguishes dispatch and provider records.
|
|
Native tools retain their provider spans without submitted-argument diagnostics. OpenCode's performance-only OTLP sink stays separate.
|
|
|
|
For known native task envelopes, OpenCode selects the task identifier, title and
|
|
prompt before the 8 KiB limit. A valid leading fresh-session handoff packet is
|
|
omitted from this excerpt so earlier history cannot hide the current request.
|
|
Continuation excerpts put pending user messages before recovery guidance.
|
|
Plain text and unknown envelope formats retain their existing text projection.
|
|
History omission also sets the turn's truncation flag; the flag can also mean a
|
|
text or row limit was reached.
|
|
|
|
When prompt capture is enabled, `paperclip.observability.input.events` gives a
|
|
relative URL on the originating Paperclip instance:
|
|
`/api/heartbeat-runs/<run-id>/events`. Sign in to that instance, read pages using
|
|
`afterSeq` and `limit`, and find the `turn.submitted` event for the trace's
|
|
`paperclip.turn.id`. Its `payload.prpEvent.payload.text` holds the retained
|
|
submission, including history, subject to existing redaction and retention.
|
|
The URL does not grant access. Run telemetry permission, company membership and
|
|
task privacy checks still apply. The reference is absent when prompt capture is
|
|
disabled. Tool argument values remain excluded from company traces.
|
|
|
|
The exporter sends once at turn completion with a three-second deadline, at most
|
|
four requests in flight and no idle polling or persistent queue. Closing drains
|
|
pending requests and marks an unfinished turn failed. Export failures emit a
|
|
sanitized diagnostic and do not change task outcomes. Abrupt process termination
|
|
can lose an unexported turn. Recovered in-flight turns without a new submission
|
|
are not replayed just to produce an observability record.
|
|
|
|
The native runner and legacy local adapters receive transient launch material.
|
|
Native Codex writes a private managed config.toml; legacy exec uses a unique
|
|
invocation profile, preserving a selected source profile. Its profile injection
|
|
requires the qualified executable version. Codex ACPX writes its isolated home.
|
|
The legacy ACPX adapter keeps export configuration in a private home per agent
|
|
and settings compatibility key, and shares conversation files with its stable managed home;
|
|
export changes do not change its logical conversation fingerprint.
|
|
Codex receives authentication through `OTEL_EXPORTER_OTLP_LOGS_HEADERS` and
|
|
`OTEL_EXPORTER_OTLP_TRACES_HEADERS` in its process environment. Its managed
|
|
configuration and invocation profiles contain empty header maps, so exporting
|
|
does not save the ingestion credential in those files. The bundled Rust OTLP
|
|
exporter percent-decodes environment header values; launch material encodes
|
|
them once to preserve literal percent sequences. Credentials remain literal
|
|
values in the company secret; config-file environment interpolation is not used.
|
|
A caller-owned/preconfigured Codex home cannot be automatically configured in
|
|
this version: its files remain untouched and execution emits a diagnostic.
|
|
Legacy OpenCode, other harnesses, personal Codex desktop sessions and hosted claude_managed are
|
|
outside this feature.
|
|
|
|
## Changes and process lifetime
|
|
|
|
Paperclip reads settings once when an execution begins preparation. That
|
|
snapshot governs instruction reservation, compatibility and launch. Already
|
|
preparing or running work retains its configuration. A changed settings
|
|
revision or resolved credential version retires an idle incompatible warm
|
|
process using existing ownership checks. Conversation compatibility is
|
|
separate, so a safe replacement can resume the saved conversation. Recovery
|
|
never replays an uncertain turn merely to apply new export settings.
|
|
|
|
Missing, disabled, invalid or unavailable configuration runs without company
|
|
export and reports a sanitized availability diagnostic where relevant. Native
|
|
export rejection/timeouts do not determine task success; exporter deadlines
|
|
are three seconds. OpenCode starts its bounded asynchronous request when the terminal event is
|
|
observed; event delivery and task settlement do not wait for the receiver.
|
|
|
|
## Credentials and access
|
|
|
|
Only board users with Manage connections permission can read or change this
|
|
configuration. Agents cannot retrieve it through the settings API. The destination stores no credential
|
|
values. Authentication headers live in one encrypted Paperclip-managed company
|
|
secret, bound to the system consumer company-observability. Updates rotate that
|
|
secret; removal uses the secret service to retire it. Braintrust project
|
|
routing is derived from the current project ID, independently of the API key.
|
|
|
|
Reads return configured status and header names. Blank existing custom header
|
|
values retain the saved secret at the same HTTP origin; a change of scheme, host,
|
|
or port requires every retained header value to be entered again. New/renamed headers require values. Removing a
|
|
row removes that header. API-key replacement is explicit, and Cancel clears
|
|
entered secrets. Revisions prevent another administrator's edit being silently
|
|
overwritten. Credential values are excluded from task environment manifests,
|
|
run/config snapshots, argv, activity records, recovery configuration and query
|
|
cache data. OpenCode keeps exporter credentials in the runner wrapper and excludes them
|
|
from its child environment and config. Codex and Claude Code receive headers through their
|
|
process environment, which their tools can inherit or inspect.
|
|
This feature does not isolate export credentials from agent code. Use a credential
|
|
limited to telemetry ingestion for the selected destination; do not supply a broad
|
|
organization administration key. A user-operated collector can hold upstream
|
|
credentials when a deployment requires a separate trust boundary.
|
|
|
|
## Connection tests
|
|
|
|
Saving never contacts the destination. Test saved settings sends one synthetic
|
|
root span and a correlated log with a unique marker from the control plane,
|
|
even when agent export is disabled. The receiver receives no production task
|
|
content. Both requests share a ten-second deadline, cannot redirect, and use
|
|
the deployment's guarded HTTP policy with DNS pinning and response-size limits.
|
|
Receipts contain sanitized HTTP/signal results and no receiver body text.
|
|
A receipt is retained only for the revision that was tested.
|
|
|
|
“Test payload accepted” means the server's synthetic payload was accepted. It
|
|
does not prove agent export, remote network reachability, downstream indexing,
|
|
or that a provider UI rendered the records. Harness exporters use their own
|
|
network clients under the execution environment's policy; the server's DNS
|
|
pinning applies only to the explicit control-plane test. Verify real tasks and
|
|
the destination's records before relying on a deployment.
|
|
|
|
## Verification
|
|
|
|
The focused suites cover schema bounds, concurrent saves, secret/header
|
|
lifecycle, authorization, protobuf transport and rejection, private-network
|
|
policy, timeout, revision fencing, environment projections, private native
|
|
configuration, warm compatibility, and UI conflict/company-switch behavior.
|
|
Storybook IDs agent-observability-01-settings, agent-observability-02-states and
|
|
agent-observability-03-journey use fake credentials and production components.
|
|
Real local/remote native export evidence must be reported separately from
|
|
these automated and synthetic checks.
|