0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-25 17:45:53 +01:00

Files

T

DottaandPaperclip 3b550c80fa fix(codex): correct startup trust, history reads, and resume usage (#13110 )

## Thinking Path

> - Paperclip runs Codex locally and in remote sandboxes.
> - The runner must preserve startup configuration and session identity.
> - Missing project trust can disable repository configuration.
> - Full-history requests use deprecated provider fields.
> - Resume usage describes old work and must not become new run usage.
> - This change corrects startup trust, state reads, and usage
classification.

## Linked Issues or Issue Description

**What happened?**

Normal Codex runs could show repository-trust and history-deprecation
warnings.
Resume could report the preceding turn's token snapshot as a late-turn
warning.
The historical last-usage value could also be attributed to the new run.

**Expected behavior**

Trust the server-selected startup root in isolated configuration. Read
lightweight
provider state and paginated evidence. Use historical cumulative usage
as a
baseline without a new charge or user-facing warning.

**Steps to reproduce**

1. Start a native Codex task in a selected repository.
2. Finish the turn and resume the provider thread.
3. Inspect provider notices, history requests, and per-run usage.
4. Repeat startup and cold resume inside a Daytona sandbox.

**Paperclip version or commit**

Codex CLI 0.153.4 is the pinned runtime and reproduced baseline.
Replayed onto master at 6abeb6733. Related authority work: Refs #13092.
This PR retains its startup cleanup and protocol-integrity checks.

**Deployment mode**

Local source checkout and disposable Daytona sandbox.

## What Changed

- Classify the exact historical resume usage event before the generic
stale-turn warning.
- Persist cumulative usage baselines across recovery of the same run.
- Use excludeTurns on resume and lightweight thread reads.
- Page turn metadata and selected turn items with cursor and identity
validation.
- Reject unsupported or incomplete history instead of guessing that
execution is idle.
- Trust the startup execution root on its host, including Git worktree
trust keys.
- Start Codex in that root and retain the selected sandbox profile on
later turns.
- Keep unrelated isolated configuration and Codex's separate hook trust
policy.
- Add Rust, TypeScript, accounting, native integration, and local
run-log documentation.

## Verification

- Codex and native-transport TypeScript: 333 passed before PR replay.
- Adjacent OpenCode/ACPX driver and accounting tests: 49 passed.
- Rust library, serialized: 226 passed. Native Codex integration: 72
passed, 1 ignored, plus two pagination regressions.
- Repository typecheck and build passed. All repository test groups have
passing coverage after fixture and resource retests; the initial
monolithic command was not clean.
- Fresh real Codex native browser tasks returned correct answers without
the three targeted notices. Answers persisted after refresh and restart.
- Real same-thread TypeScript driver tests passed locally and in
Daytona, including cold resume, configuration, skills, and an approved
harmless hook.
- Local usage summed to 64,607 tokens. Daytona usage summed to 42,737
tokens. Each sum matched its final session total exactly.
- See doc/plans/2026-09-09-codex-integration-acceptance.md for the scope
and limits of the live tests.
- After replay onto current master and review fixes: 334 Codex, backend,
and live-session tests passed, including checkpoint serialization and
real-runner process restart. TypeScript checks passed.
- The native Codex integration run passed 83 tests; the large lineage
test passed separately with the release runner (its debug build exceeded
the test deadline).
- All GitHub checks passed on the final PR head. Greptile is 5/5 with no
unresolved review threads. CI regenerates the lockfile for the added
TOML dependency, per repository policy.
- The first server shard hit a timing-dependent duplicate-key failure in
the unchanged artifact-document concurrency test. Its focused 11-test
suite passed locally. One CI retry on the same head passed all 103 files
and 1,405 tests (2 skipped): [retry
result](https://github.com/paperclipai/paperclip/actions/runs/34398832930/job/102631274667).

## Risks

- Trust applies only to the server-selected startup root and isolated
configuration. Sandbox and tool permissions remain authoritative.
- Codex still requires approval of individual hook hashes. This change
does not bypass that policy.
- Providers without the required history APIs fail explicitly.
- Daytona acceptance used the production TypeScript driver. Remote
Paperclip UI and remote Rust execution were not tested.
- No new public API, database state, recovery policy, or UI control is
included.

## Model Used

OpenAI Codex, GPT-6 (`gpt-6-astra`). Used for reasoning, code edits,
tool use,
and test execution. The exact context-window limit is not exposed in
this
session. Real-provider acceptance used Codex CLI 0.153.4 with
`gpt-5.6-sol`.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-09-09 15:35:18 -05:00

6.6 KiB

Raw Blame History

Run-Log Events

Run-log events write to the heartbeat_run_events table (packages/db/src/schema/heartbeat_run_events.ts:6-20). They are not Paperclip Telemetry events, and they are not OpenTelemetry exports. A run-log event needs no operator endpoint.

Native PRP Run-Log Events

The hidden native coordinator writes each validated PRP event to the bound run's existing event stream before it acknowledges the runner. The row keeps the PRP eventType, source instance, source event ID, source sequence, protocol schema version, and a SHA-256 digest of the canonical source envelope. Its payload is { "prpEvent": <canonical PRP event> }.

The writer locks the native heartbeat_runs row and allocates the existing per-run seq cursor. A byte-equivalent retry reuses the first row; a changed retry or source-sequence gap is rejected. Company, issue, agent, run, session, and runner-source bindings must match the persisted native run. Bootstrap tickets, reconnect leases, authentication proofs, encryption keys, and raw credential material are never written to the run log.

These records remain run-log events. They do not create an OpenTelemetry or Paperclip Telemetry export, and legacy adapters do not use this writer.

Native Restart Recovery Run-Log Event

Paperclip writes a native.recovery.transition event for every native restart classification and for graceful restart suspension. This immutable run-log record lets operators reconstruct recovery decisions without exporting data to Paperclip Telemetry or OpenTelemetry.

The payload contains the restart kind, recovery request id when one exists, runner disposition, and the controller generation and provider attempt for a claimed recovery. Live-runner adoption also records the runner PID, process group, and process-start fingerprint. A non-claim disposition records a bounded reason instead. Graceful suspension records the signal and confirms that it did not create a retry run.

The event never includes bootstrap tickets, reconnect leases, authentication proofs, encryption keys, environment variables, provider credentials, command arguments, or an unsanitized stderr stream. Detailed failed-attempt diagnostics remain in the bounded native_run_finalizations.recovery_history ledger.

Sandbox Startup Run-Log Event

Paperclip writes one run.startup.step event to the run log for each bring-up step. This event is a run-log record, not a first-party telemetry event. The generated telemetry contract does not cover it, so this section is its canonical contract.

The event payload carries only three fields.

Field	Type	Meaning
`step`	string	The bring-up step name, for example `stage.sync`.
`durationMs`	number	The wall time of the step. A skipped step reports `0`.
`outcome`	string	The step outcome (`ok`, `skipped`, or `failed`).

The event no longer carries the per-step round-trip count or the provider duration fields. It dropped roundTrips, providerExecMs, providerGetMs, createRuntimeMs, and ensureSessionMs. The startup spans in doc/observability.md carry that detail now. The sandbox.exec child spans hold the round-trip and provider durations. The acp.handshake step span holds the create-runtime and ensure-session sub-times.

To read the detailed timing, use the startup spans. The spans need an OTLP endpoint. A run with no endpoint keeps only the three run-log fields above.

Run Phase Timing Run-Log Event

Paperclip writes one run.phase.timing event to the run log for each run-lifecycle phase. This event is a run-log record, not a first-party telemetry event. The generated telemetry contract does not cover it, so this section is its canonical contract. The producer is emitRunPhaseTiming in packages/adapter-utils/src/acpx-engine/startup-timing.ts.

The event payload carries only three fields.

Field	Type	Meaning
`phase`	string	The run-lifecycle phase name from the closed allowlist below.
`durationMs`	number	The wall time of the phase. A negative or a non-finite value clamps to `0`.
`outcome`	string	The phase outcome (`ok` or `failed`).

The phase field is one member of a closed, low-cardinality allowlist. The producer drops any event whose phase name is outside this allowlist, so a free-form label never reaches the run log. The allowlist has twelve phase names.

Phase	Meaning
`place_workspace`	Place the run workspace.
`start_transport`	Start the agent transport.
`create_runtime`	Create the agent runtime.
`ensure_session`	Ensure the agent session exists.
`configure_session`	Configure the agent session.
`prepare_turn`	Prepare the turn.
`turn`	Run the turn.
`end_session`	End the agent session.
`settle_reuse`	Settle the session for reuse.
`stop_transport`	Stop the agent transport.
`sync_back`	Sync the workspace back.
`release_staging_lease`	Release the staging lease.

The payload never carries a command, an argument, a path, an environment value, or a raw identifier. The event rides the ctx.onEvent run-event bridge and is run-log-only. It needs no OTLP endpoint.

The sandbox duplex transport also writes one run-log event as one of its three sinks. See the Sandbox Duplex Transport Instrumentation section in the Observability contract.

Execution recovery

Provider identity diagnostics remain in the local run log. They record the notification method, expected and received thread/turn identifiers, and the classification (root, verified descendant, stale, unrelated informational, or invalid authoritative). They omit the original provider payload and credentials. Repeated informational notices are bounded.

Recovery lifecycle events retain the original structured failure code, retry attempt, next retry time, and predecessor/successor identifiers. Durable status delivery uses an idempotency marker; delivery grants no provider authority. Failed publication is retried without repeating provider work. These records are not first-party Telemetry.

Codex resume usage snapshot

The native runner retains a bounded local harness.diagnostic event with code codex_resume_usage_snapshot. It identifies thread/tokenUsage/updated as resume_usage_snapshot, retains the reported thread and completed-turn IDs, and records cumulative usage counters. It does not include provider credentials or message content. The event establishes the accounting baseline; it is not a new billable usage receipt or a user-facing provider warning. Other provider identity checks remain in force.

6.6 KiB Raw Blame History