mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 05:16:50 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E journeys verify saved application behavior across deployments. > - Task creation and questions need reusable smoke selections and exact run attribution. > - Operators also need to inspect the OpenCode execution behind a measured question. > - Company observability does not currently support native OpenCode. > - This change adds the journeys and a bounded projection of normalized OpenCode run events. > - The projection follows the configured capture policy and retains the current run identity across warm reuse. ## Linked Issues or Issue Description Related foundation: #17262. Related alternative: #17989 exports metadata from OpenCode's native OTLP stream. This change uses normalized run events to support the existing content switches and per-run linkage. It does not forward the native stream. The two PRs overlap in launch registration and must be reconciled before both land. **What existing behavior does this improve?** Reusable Product E2E smoke selections and company observability for native OpenCode. **Current behavior** The journey catalog lacks reusable task creation and attached-agent question checks. Native OpenCode is marked unsupported for company observability, so a successful destination connection test does not prove that agent traces will appear. **Proposed behavior** Add explicit task-create and ask-question selections. Verify saved state and exact company, issue, agent, model, environment and successful run attribution. Export normalized OpenCode turns, tools and usage to the configured OTLP destination. Honor each capture switch and redact known secrets. **Reason and benefit** CI can run a small measured smoke selection, retain its screenshots and checks, and link results to traces belonging to the same run. **Breaking changes** Existing selectors remain unchanged. OpenCode export is opt-in through existing company settings. Legacy OpenCode stays unsupported. ## What Changed - Add reusable task creation and question journeys, configured-agent capability, owned-fixture checkpoints and strict evidence validation. - Add an OpenCode-owned projection of normalized run events. No provider SDK or raw payload schema enters heartbeat coordination. - Extend the private launch union and reserved environment allowlist for native OpenCode. With export disabled, no receiver request runs. - Keep destination headers in the parent wrapper and out of the actual provider child environment and configuration. - Bound each turn to 64 tool and 64 usage entries, 8 KiB per text field and a 1 MiB request. Use a three-second request deadline, four concurrent exports, and no retries or idle worker. - Pass the rotated runner authority to each proxy turn. A retained process no longer attributes later turns to its original launch run. - Update settings copy, observability documentation and capability maps. ## Verification - `pnpm -r typecheck` and `pnpm build` passed after the implementation changes. - Adapter launch/redaction: 9 tests passed. Projection and actual proxy: 12 tests passed before the review fix; the expanded projection suite passed all 12 cases after the fix. The proxy test exercises two runs in one process against a real local HTTP receiver and a fake provider. - Full OpenCode driver/projection suite: 72 passed, one existing response_wake test timed out at five seconds. A focused check then passed both existing omitted-continuation cases and the new trace export case. The timeout cause remains unconfirmed. - Runner package TypeScript and Rust `cargo check` passed. Map and UI token checks passed. - Prior journey validation: 273 tests across 12 files and the journey TypeScript check passed. - Broad local `pnpm test:run` encountered native-runtime integration failures with repeated embedded PostgreSQL initialization failures and was stopped. All 16 failing cases passed in an isolated run outside the filesystem sandbox (961 unrelated cases unselected). The broad local run is not a clean acceptance result. A later focused database-backed chat/heartbeat run passed all six selected observability cases, including native OpenCode launch and successful allocation (1,809 unrelated cases were unselected). - All CI gates passed for `21085f880d8babc9efeb22d0623db99613d33093`, including packaged-runner, clean-install, browser and build checks: https://github.com/paperclipai/paperclip/actions/runs/40884658995 . Greptile reviewed the same commit at 5/5 with no unresolved findings. - Verified OTLP JSON acceptance and indexing with an explicitly synthetic transport marker. No live OpenCode model trace or deployed smoke acceptance is claimed yet. ## Risks Export is best effort. Abrupt termination, recovered in-flight turns without a new submitted event, limits or receiver failure can lose traces. Export failure does not change task success. The projection represents observed normalized events, not raw native model spans or hidden reasoning. Known-secret redaction is not an OS isolation boundary. UI changes can break journey locators. Deployment adapters retain credential, ownership and cleanup responsibility. Both the controller and native runner bundle must be updated before live qualification. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution and browser tools. The exact serving model ID and context size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
127 lines
7.9 KiB
Markdown
127 lines
7.9 KiB
Markdown
# Company agent observability
|
|
|
|
Company Settings → General → Agent observability configures one company-owned
|
|
OTLP destination for Paperclip-managed Codex, Claude Code and native OpenCode executions. Export
|
|
is disabled by default. Choose Braintrust (US) or a custom HTTP(S) OTLP base URL;
|
|
use your own collector when you need fan-out. Braintrust EU and self-hosted
|
|
installations can use Custom OTLP with their required routing/authentication
|
|
headers. The receiver must be reachable from every execution environment;
|
|
`localhost` refers to the execution host, including a remote sandbox.
|
|
|
|
This is separate from Paperclip's first-party Telemetry and its content-free
|
|
[infrastructure tracing](observability.md). Configuring it does not change either data path.
|
|
|
|
## Capture and supported runtimes
|
|
|
|
The qualified versions are Codex 0.160.0 and Claude Code through ACP bridge
|
|
0.73.0 with Agent SDK 0.3.286. Both logs and traces use HTTP/protobuf. Prompts,
|
|
assistant responses and tool content default on. Additional tool details are a
|
|
Claude Code and native OpenCode control; Codex has no equivalent switch. Codex requests up to 64 KiB
|
|
of tool output (or zero when disabled); native event availability, truncation
|
|
and response caps still apply. Claude Code applies its own native limits.
|
|
Metrics, raw API bodies and Codex Guardian assessments are disabled.
|
|
|
|
Native OpenCode 1.18.34 uses a runner-owned HTTP/JSON exporter. Each completed,
|
|
failed or cancelled turn produces a trace with Paperclip company, agent, run and
|
|
turn IDs. Tool outcomes and message-level token/cost receipts are child spans;
|
|
repeated usage updates replace the same message receipt. Missing prices remain
|
|
unknown. This projection excludes hidden reasoning, raw provider payloads and
|
|
system instructions. Prompt, response, tool-detail and tool-content switches
|
|
apply before export. Content is bounded to 8 KiB per field and 64 tools plus 64
|
|
usage records per turn; truncation is marked. Tool inputs and full model request
|
|
bodies are not captured. OpenCode's performance-only OTLP sink stays separate.
|
|
|
|
The exporter sends once at turn completion with a three-second deadline, at most
|
|
four requests in flight and no idle polling or persistent queue. Closing drains
|
|
pending requests and marks an unfinished turn failed. Export failures emit a
|
|
sanitized diagnostic and do not change task outcomes. Abrupt process termination
|
|
can lose an unexported turn. Recovered in-flight turns without a new submission
|
|
are not replayed just to produce an observability record.
|
|
|
|
The native runner and legacy local adapters receive transient launch material.
|
|
Native Codex writes a private managed config.toml; legacy exec uses a unique
|
|
invocation profile, preserving a selected source profile. Its profile injection
|
|
requires the qualified executable version. Codex ACPX writes its isolated home.
|
|
The legacy ACPX adapter keeps export configuration in a private home per agent
|
|
and settings compatibility key, and shares conversation files with its stable managed home;
|
|
export changes do not change its logical conversation fingerprint.
|
|
Codex receives authentication through `OTEL_EXPORTER_OTLP_LOGS_HEADERS` and
|
|
`OTEL_EXPORTER_OTLP_TRACES_HEADERS` in its process environment. Its managed
|
|
configuration and invocation profiles contain empty header maps, so exporting
|
|
does not save the ingestion credential in those files. The bundled Rust OTLP
|
|
exporter percent-decodes environment header values; launch material encodes
|
|
them once to preserve literal percent sequences. Credentials remain literal
|
|
values in the company secret; config-file environment interpolation is not used.
|
|
A caller-owned/preconfigured Codex home cannot be automatically configured in
|
|
this version: its files remain untouched and execution emits a diagnostic.
|
|
Legacy OpenCode, other harnesses, personal Codex desktop sessions and hosted claude_managed are
|
|
outside this feature.
|
|
|
|
## Changes and process lifetime
|
|
|
|
Paperclip reads settings once when an execution begins preparation. That
|
|
snapshot governs instruction reservation, compatibility and launch. Already
|
|
preparing or running work retains its configuration. A changed settings
|
|
revision or resolved credential version retires an idle incompatible warm
|
|
process using existing ownership checks. Conversation compatibility is
|
|
separate, so a safe replacement can resume the saved conversation. Recovery
|
|
never replays an uncertain turn merely to apply new export settings.
|
|
|
|
Missing, disabled, invalid or unavailable configuration runs without company
|
|
export and reports a sanitized availability diagnostic where relevant. Native
|
|
export rejection/timeouts do not determine task success; exporter deadlines
|
|
are three seconds. OpenCode starts its bounded asynchronous request when the terminal event is
|
|
observed; event delivery and task settlement do not wait for the receiver.
|
|
|
|
## Credentials and access
|
|
|
|
Only board users with Manage connections permission can read or change this
|
|
configuration. Agents cannot retrieve it through the settings API. The destination stores no credential
|
|
values. Authentication headers live in one encrypted Paperclip-managed company
|
|
secret, bound to the system consumer company-observability. Updates rotate that
|
|
secret; removal uses the secret service to retire it. Braintrust project
|
|
routing is derived from the current project ID, independently of the API key.
|
|
|
|
Reads return configured status and header names. Blank existing custom header
|
|
values retain the saved secret at the same HTTP origin; a change of scheme, host,
|
|
or port requires every retained header value to be entered again. New/renamed headers require values. Removing a
|
|
row removes that header. API-key replacement is explicit, and Cancel clears
|
|
entered secrets. Revisions prevent another administrator's edit being silently
|
|
overwritten. Credential values are excluded from task environment manifests,
|
|
run/config snapshots, argv, activity records, recovery configuration and query
|
|
cache data. OpenCode keeps exporter credentials in the runner wrapper and excludes them
|
|
from its child environment and config. Codex and Claude Code receive headers through their
|
|
process environment, which their tools can inherit or inspect.
|
|
This feature does not isolate export credentials from agent code. Use a credential
|
|
limited to telemetry ingestion for the selected destination; do not supply a broad
|
|
organization administration key. A user-operated collector can hold upstream
|
|
credentials when a deployment requires a separate trust boundary.
|
|
|
|
## Connection tests
|
|
|
|
Saving never contacts the destination. Test saved settings sends one synthetic
|
|
root span and a correlated log with a unique marker from the control plane,
|
|
even when agent export is disabled. The receiver receives no production task
|
|
content. Both requests share a ten-second deadline, cannot redirect, and use
|
|
the deployment's guarded HTTP policy with DNS pinning and response-size limits.
|
|
Receipts contain sanitized HTTP/signal results and no receiver body text.
|
|
A receipt is retained only for the revision that was tested.
|
|
|
|
“Test payload accepted” means the server's synthetic payload was accepted. It
|
|
does not prove agent export, remote network reachability, downstream indexing,
|
|
or that a provider UI rendered the records. Harness exporters use their own
|
|
network clients under the execution environment's policy; the server's DNS
|
|
pinning applies only to the explicit control-plane test. Verify real tasks and
|
|
the destination's records before relying on a deployment.
|
|
|
|
## Verification
|
|
|
|
The focused suites cover schema bounds, concurrent saves, secret/header
|
|
lifecycle, authorization, protobuf transport and rejection, private-network
|
|
policy, timeout, revision fencing, environment projections, private native
|
|
configuration, warm compatibility, and UI conflict/company-switch behavior.
|
|
Storybook IDs agent-observability-01-settings, agent-observability-02-states and
|
|
agent-observability-03-journey use fake credentials and production components.
|
|
Real local/remote native export evidence must be reported separately from
|
|
these automated and synthetic checks.
|