Commit Graph
2 Commits
Author SHA1 Message Date
DottaandPaperclip b81b870732 Add QA smoke journeys and OpenCode run traces (#18098)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Product E2E journeys verify saved application behavior across
deployments.
> - Task creation and questions need reusable smoke selections and exact
run attribution.
> - Operators also need to inspect the OpenCode execution behind a
measured question.
> - Company observability does not currently support native OpenCode.
> - This change adds the journeys and a bounded projection of normalized
OpenCode run events.
> - The projection follows the configured capture policy and retains the
current run identity across warm reuse.

## Linked Issues or Issue Description

Related foundation: #17262. Related alternative: #17989 exports metadata
from OpenCode's native OTLP stream. This change uses normalized run
events to support the existing content switches and per-run linkage. It
does not forward the native stream. The two PRs overlap in launch
registration and must be reconciled before both land.

**What existing behavior does this improve?**

Reusable Product E2E smoke selections and company observability for
native OpenCode.

**Current behavior**

The journey catalog lacks reusable task creation and attached-agent
question checks. Native OpenCode is marked unsupported for company
observability, so a successful destination connection test does not
prove that agent traces will appear.

**Proposed behavior**

Add explicit task-create and ask-question selections. Verify saved state
and exact company, issue, agent, model, environment and successful run
attribution. Export normalized OpenCode turns, tools and usage to the
configured OTLP destination. Honor each capture switch and redact known
secrets.

**Reason and benefit**

CI can run a small measured smoke selection, retain its screenshots and
checks, and link results to traces belonging to the same run.

**Breaking changes**

Existing selectors remain unchanged. OpenCode export is opt-in through
existing company settings. Legacy OpenCode stays unsupported.

## What Changed

- Add reusable task creation and question journeys, configured-agent
capability, owned-fixture checkpoints and strict evidence validation.
- Add an OpenCode-owned projection of normalized run events. No provider
SDK or raw payload schema enters heartbeat coordination.
- Extend the private launch union and reserved environment allowlist for
native OpenCode. With export disabled, no receiver request runs.
- Keep destination headers in the parent wrapper and out of the actual
provider child environment and configuration.
- Bound each turn to 64 tool and 64 usage entries, 8 KiB per text field
and a 1 MiB request. Use a three-second request deadline, four
concurrent exports, and no retries or idle worker.
- Pass the rotated runner authority to each proxy turn. A retained
process no longer attributes later turns to its original launch run.
- Update settings copy, observability documentation and capability maps.

## Verification

- `pnpm -r typecheck` and `pnpm build` passed after the implementation
changes.
- Adapter launch/redaction: 9 tests passed. Projection and actual proxy:
12 tests passed before the review fix; the expanded projection suite
passed all 12 cases after the fix. The proxy test exercises two runs in
one process against a real local HTTP receiver and a fake provider.
- Full OpenCode driver/projection suite: 72 passed, one existing
response_wake test timed out at five seconds. A focused check then
passed both existing omitted-continuation cases and the new trace export
case. The timeout cause remains unconfirmed.
- Runner package TypeScript and Rust `cargo check` passed. Map and UI
token checks passed.
- Prior journey validation: 273 tests across 12 files and the journey
TypeScript check passed.
- Broad local `pnpm test:run` encountered native-runtime integration
failures with repeated embedded PostgreSQL initialization failures and
was stopped. All 16 failing cases passed in an isolated run outside the
filesystem sandbox (961 unrelated cases unselected). The broad local run
is not a clean acceptance result. A later focused database-backed
chat/heartbeat run passed all six selected observability cases,
including native OpenCode launch and successful allocation (1,809
unrelated cases were unselected).
- All CI gates passed for `21085f880d8babc9efeb22d0623db99613d33093`,
including packaged-runner, clean-install, browser and build checks:
https://github.com/paperclipai/paperclip/actions/runs/40884658995 .
Greptile reviewed the same commit at 5/5 with no unresolved findings.
- Verified OTLP JSON acceptance and indexing with an explicitly
synthetic transport marker. No live OpenCode model trace or deployed
smoke acceptance is claimed yet.

## Risks

Export is best effort. Abrupt termination, recovered in-flight turns
without a new submitted event, limits or receiver failure can lose
traces. Export failure does not change task success. The projection
represents observed normalized events, not raw native model spans or
hidden reasoning. Known-secret redaction is not an OS isolation
boundary. UI changes can break journey locators. Deployment adapters
retain credential, ownership and cleanup responsibility. Both the
controller and native runner bundle must be updated before live
qualification.

## Model Used

OpenAI Codex, GPT-6, with reasoning, code execution and browser tools.
The exact serving model ID and context size are not exposed to this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted suites; broad
local limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-29 08:57:12 -05:00
DottaandPaperclip ac9459bae3 feat: configure company agent observability (#16874)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Companies use several agent harnesses and execution environments.
> - Operators need one place to select where native agent logs and
traces go.
> - An inference proxy would add a dependency to the agent request path.
> - This pull request supplies native OpenTelemetry settings to managed
Codex and Claude Code processes.
> - Company secrets protect destination credentials. Export failure does
not change the task result.

## Linked Issues or Issue Description

**Subsystem affected**

Company settings, shared contracts, database, server, adapters, and
runner.

**Problem or motivation**

An operator cannot configure a company destination for native agent logs
and traces. Manual harness configuration is hard to apply across local
and remote execution.

**Proposed solution**

Add one company destination: Braintrust or a custom OTLP HTTP/protobuf
endpoint. Apply a transient snapshot when execution preparation starts.
Keep credentials out of API reads, task environment manifests, recovery
files, and command arguments. Replace a warm process when its export
settings change and preserve its conversation.

**Alternatives considered**

Multiple destinations can use a user-operated collector for fan-out.
This change does not add a collector, an inference proxy, or transcript
scraping.

**Roadmap alignment**

This extends company settings, secret management, and support for
multiple harnesses. The maintainer requested this implementation.
Searches for observability and Braintrust found no duplicate company
destination feature. Related infrastructure tracing work in #16236 has a
separate content-free contract.

## What Changed

- Add strict schemas, revision checks, encrypted secret storage,
activity records, and board-only settings routes.
- Add explicit synthetic logs and traces tests with a bounded HTTP
request and sanitized results.
- Supply native Codex configuration and Claude Code environment values
through local and remote launch paths.
- Preserve Codex login refresh through the existing managed credential
link. Retain unsaved drafts across background reads. Require complete
custom header re-entry when the endpoint origin changes.
- Separate process export compatibility from conversation identity.
Isolate generated Codex configuration and preserve resumable sessions,
including remote home refresh after export changes.
- Advance the immutable Pi/Copilot source attestations for shared
environment and sidecar code changes. Keep their provider payloads
unchanged and retain prior identities for decoding.
- Redact destination credentials throughout launch diagnostics and HTTP
logs.
- Add the production settings form, Storybook states and journey,
documentation, and feature/access map entries.

## Verification

- Real managed local Codex 0.160.0 and Claude Code 2.1.286 tasks
exported to Braintrust. API readback and the Braintrust UI showed the
test records. A real remote Daytona Codex task exported through the
legacy `exec` path.
- Live tests exercised content capture off, export disabled, warm
conversation reuse, and a destination that accepted connections but
never replied. Agent work succeeded and export requests stopped at the
configured three-second timeout. The UI connection test returned “Test
payload accepted”; the saved API key stayed masked after reload.
- All current-head CI test lanes, typechecks, packaging/build checks,
and required aggregates passed on
`71e994b2be9c8fb6d6e938a4ad7cf4c95fdafe31` ([CI
run](https://github.com/paperclipai/paperclip/actions/runs/39319194318)).
Greptile completed on that exact commit at 5/5, with zero unresolved
review threads. The additional optional Paperclip Review check remains
queued waiting for its assigned agent.
- Local workspace typecheck and build passed. Focused verification
included 239 combined runner tests, 789 API/native-session tests, 107
Codex home/ACP/sandbox-site tests, 204 feature/privacy tests, and the
affected Rust suites. Migration, token, map, Storybook catalog, and
Storybook build checks passed during acceptance. These checks ran at the
relevant implementation or merge boundary; the current-head CI is the
complete final gate.
- Managed AI credential projections now receive enabled and disabled
company export settings; genuine caller-owned Codex homes remain
untouched. Two regressions reproduced the skip before the repair. All
276 engine/lifecycle tests and adapter-utils typecheck pass afterward.
The launched TOML retains provider/MCP settings, credential continuity,
and redaction.
- Settings saves, removal, and the real company-deletion owner acquire
secret mutation ownership before database locks. PostgreSQL NOWAIT
probes cover all three paths; concurrent save/rotation/removal/deletion
requests settle correctly. The affected suites passed 137 tests,
followed by all 32 service/connection cases after expanded deletion
coverage. Server and Codex adapter typechecks pass.
- Codex source profiles use the effective CLI argument list. A spawned
fixture CLI reproduced both ignored-argument failures; all three
precedence/fallback cases now pass.
- Real tar staging reproduced remote conversation loss in both API-key
and subscription modes before the repair. Both now pass: changing
managed settings preserves sessions, archives, native indexes, and
refreshed subscription credentials. A corrupt upload preserves the prior
config and conversation.
- Preserved failure evidence: initial CI caught stale Pi/Copilot source
attestations, a missing test-harness helper, and packaging fixture/probe
mismatches; those were corrected and covered by focused checks.
Superseded heads had intermittent attachment-composer timeouts despite
successful uploads; [the original trace and screenshot remain
available](https://github.com/paperclipai/paperclip/actions/runs/39309701089/artifacts/12215109203).
The same attachment case passed on the final head: [25/25 tests in shard
3](https://github.com/paperclipai/paperclip/actions/runs/39319194318/job/118288637051).
- A connection-fault fixture also exposed a stale pooled socket after
its hang case; closing fixture responses isolated scenarios and the full
affected suite passed.
- The broad local test run was not clean: database resource failures,
timeouts, an older cached module graph, and a stale optional-resource
manifest assumption were recorded. Focused checks passed after the
documented fixes. This does not establish a clean full local pass.
- Native remote Codex, remote Claude, and the full ACPX Codex bridge
remain unqualified. The direct Codex resume test does not establish full
bridge acceptance.

## Risks

- Native exporters perform network requests from each execution
environment. A server connection test does not prove remote
reachability.
- Prompts, assistant responses, and tool content are enabled by default
when supported. Native caps and harness-specific controls still apply.
- Caller-owned Codex homes are unsupported for automatic configuration
and remain untouched.
- Native harnesses and their tool processes can inspect export
credentials. The settings form and documentation advise using
ingestion-scoped credentials. Retained headers remain bound to their
saved scheme, host, and port; changing origin requires fresh values or
removal.
- Migration 0357 adds a company-scoped row and secret reference. It is
safe to reapply. Secret removal and company deletion use the existing
secret lifecycle.
- Slow or unavailable destinations must not block task settlement.
Export settings take effect at the next execution preparation boundary.

## Model Used

OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and
browser verification. The exact serving model ID and context window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 15:36:04 -05:00