mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 17:45:53 +01:00
DottaandPaperclip
044628f59e
## Thinking Path > - Paperclip manages AI agents and their work. > - Runner Evals exercise real providers against a mock control plane. > - The controlled gateway route creates a disposable provider home. > - That route had no explicit native trace configuration. > - Codex skips normal home preparation for a preconfigured home. > - This change validates and applies the existing observability launch contract before provider creation. > - Operators can collect native evidence without passing vendor keys to the child. ## Linked Issues or Issue Description Refs #17440. Related: #17452 and #17469. This PR is based directly on master and does not include those pending controller changes. **Current behavior** The eval-session CLI cannot pass an explicit native observability launch. Passing environment alone cannot configure Codex in the disposable gateway home. **Proposed behavior** Accept an opt-in descriptor bound to the gateway attempt. Validate the existing provider launch contract, add a scoped relay token in memory, and apply it only to the owned provider home and environment. **Reason and benefit** A controller can route native evidence through its bounded local relay while the CLI keeps configuration distinct from verified ingestion. This is the CLI contract; hosted controller integration remains separate work. ## What Changed - Add a strict, bounded descriptor reader for native Codex and ACPX Claude. Bind campaign, execution, attempt, project, trace, span, and mock authority identifiers. - Reuse Runner observability materialization and redaction. Preserve the model gateway route. Reject upstream destinations, extra environment fields, credentials in headers, and unknown exporter settings. - Add `--observability-config`, requiring an explicit gateway. Keep direct and unconfigured paths unchanged. - Retain secret-free configured/unverified metadata in success and failure artifacts. - Document the contract and controller responsibilities. No workflow, runtime dependency, paid schedule, or public report changes. ## Verification - Eval CLI regression selection: 81 passed before the test-only relocation. The 17 new cross-package observability tests now pass from `tests/runner-e2e/runner-eval-observability.test.ts`; their integration-suite typecheck also passes. They use the actual shared launch generator, real disposable homes, CLI success/failure paths, and credential redaction. The first run exposed two test setup references introduced by a broad edit; both were corrected and its failure log remains retained. - Runner TypeScript build and release daemon build passed. Full `pnpm -r typecheck` passed. - Actual native Codex and ACPX Claude reached a local receiver with authenticated OTLP protobuf and the expected Braintrust parent. Codex sent four log and two trace batches; Claude sent three log and one trace batch. All recorded processes exited, disposable homes were removed, and 29 retained files contained neither synthetic credential. - The local model endpoint deliberately returned 401. Codex retained a failed terminal turn; Claude retained a provider timeout. These are route/export proofs, not semantic eval passes. Native retries reached only the fake endpoint. No paid provider or Braintrust request was made. - The first local probe failed before provider launch because its TypeScript loader path was wrong. It was corrected to the installed loader; both attempts remain retained. - Full repository build passed at initial head7b75c0349f. The full local test suite encountered repeated embedded PostgreSQL startup failures before database-backed test bodies. It was stopped after five affected suites (exit 143), so it is incomplete. Its log remains retained. The same host had independently demonstrated PostgreSQL semaphore exhaustion earlier in this task; this run did not collect that lower-level error. Full hosted workflow 40131255078 passed on final head23a172af32, including the database-backed suites, Runner, build, typecheck and browser checks. Greptile is 5/5 with zero open threads. The secondary Paperclip reviewer ended neutral without an assessment; it is not counted as a successful review. A follow-up header-validation repair uses a map without a prototype and passes the added negative regression. No exporter behavior changed. - The CLI still reports native ingestion as unverified. No complete capture, destination indexing, vendor billing bound, hosted admission, or paid recurrence qualification is claimed. - Package-boundary validation found two new imports from the standalone Runner tests into other repository packages. Those tests now live in the repository integration suite and use the actual launch generator there. Its first run failed on a TOML parser resolved from the wrong package; resolving that dependency from its Runner owner fixed the test. Three pre-existing boundary/tracked-import findings remain in unchanged files; this PR adds none. ## Risks - This descriptor is intentionally narrow. Changes to the shared launch contract must be reviewed against its validator and integration tests. - The local relay receives native content under the selected capture policy. Its controller must enforce ingestion quotas, shutdown, private retention, and qualified cost bounds. - Descriptor validation runs once before launch. It reads at most 128 KiB and refuses symlinks and non-regular files. Existing owned-home config writing remains synchronous and small. - Relay credentials are held in child memory. Persisted diagnostics and artifacts use the existing redactor. The descriptor itself must remain secret-free. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. This runtime does not expose the exact model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>