mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:58:52 +01:00
Nicky LeachandPaperclip
27be859044
## Thinking Path > - Paperclip manages agents and tasks. > - Runner owns each authenticated run and its provider process. > - Runtime clients need to report successful activity in that run. > - A retained process can outlive a run, so an operation needs current authority. > - This PR captures that authority before the operation and records activity afterward. > - Core can consume the event through the existing run stream. ## Linked Issues or Issue Description Builds on merged #18129 and targets master. The dependent consumer is #18131. Test stability fixes were separated into merged #18128. Earlier combined-head verification remains in [the archive](https://github.com/paperclipai/paperclip/pull/18050#issuecomment-6489369676). **Subsystem affected** Runner activity recording and provider command environments. **Problem or motivation** A retained provider process can complete an operation after its original run ends. Runtime clients need a host-attributed event and a helper that works inside the provider sandbox. **Proposed solution** Capture opaque Runner context before use. Publish a structured notification after success. Runner checks current authority and records `runtime.activity.recorded`. ## What Changed - Add `notification-context` and `record-activity CONTEXT NAMESPACE ACTIVITY_KEY`. Rotate context on warm attachment and process recovery. Accept no caller run ID. - Use one private named pipe. Reuse Runner's bounded JSON-line reader. Serialize concurrent publishers with a read-only lock. A blocking reader queues at most 128 records; the existing control loop handles at most 16 per pass. - Grant Codex access to the exact helper, descriptor, lock and pipe. Preserve other filesystem and network limits. Preserve host discovery values through provider launch boundaries and reject configured-environment overrides. - Update source identities to Pi 34, Copilot 81 and Grok 12. Keep historical identity fixtures and recovery behavior. ## Verification - Targeted release notification and sandbox-policy tests: seven passed. They cover run rotation, recovery, Unicode limits, concurrent writers, malformed and partial records, bounded queues, failed persistence and fixed-path cleanup. - Real CLI integration and pinned Codex 0.160.0 sandbox checks: passed. The sandbox check uses no model, credential or network call. It verifies activity recording while unrelated state reads and parent-directory writes remain denied. - ACPX profile generator check, formatting and diff checks: passed. - Rebased onto master `31b927e30` at head `64d0248a8`. Master used the same profile versions, so the resolution preserves its Pi 33, Copilot 79 and Grok 11 fixtures byte-for-byte and creates new identities. All identity checks pass. Five targeted provider/environment/recovery files pass 168 tests with one existing skip; the Rust Pi admission regression passes. The notification module and Codex sandbox implementation are byte-identical to the tested pipe head `671373fe0`. - Removed the mailbox traversal reported by security review. Its regression replaces the pipe path with a 100-level tree and an unreadable leaf. Shutdown preserves the leaf, performs no traversal and completes promptly, including with a partial record. - An initial targeted test failed because its run-rotation fixture updated config without updating durable state. The corrected fixture passes. Runner also rejects inconsistent run bindings. - Previous head `26abbf62b` passed Runner, build and browser CI. Server shard 11/12 failed in unchanged `attention-service.test.ts`: cleanup/test timeouts were followed by a duplicate `ATQ` fixture key. The final head needs fresh CI and review. - CI on `64d0248a8` completed with 52 successful checks and four intentional skips. Runner Vitest shard 1/2 failed when retained-state copies encountered the live FIFO; the aggregate also failed. Standard review scored 4/5 and found relative pipe paths plus late context publication. Follow-up `5c3f14d32` places IPC in a private canonical temporary directory and publishes context before welcome commands. Its relative-state CLI test, two authenticated reconnect cases, seven notification/policy cases and real Codex sandbox pass. One affected retained-settlement case passes with the matching release binary; the other 18 cases remain for CI. The sandbox denies sibling-directory and lock writes. All build, test and security checks pass on `5c3f14d32`; standard review is 5/5 with no unresolved threads. The requested Apex review is 5/5 on this exact head, with no findings or unresolved threads. - Earlier broad test results in the archive apply to their recorded source revisions. Per the requested workflow, final local checks stay targeted; GitHub CI supplies broad coverage. - After Apex completed, master advanced again and conflicted on Copilot identity. Head `e94a9359d` is rebased on master `8ac5a128e`; it preserves the published Copilot 80 fixture and uses Copilot 81. The notification module, Codex sandbox and durable control-loop code are byte-identical to Apex-reviewed `5c3f14d32`. All 131 targeted installation/compatibility/recovery cases and identity generator checks pass. All 55 checks pass with four intentional skips on the rebased head. Standard review is 5/5 with no unresolved threads. Apex is 5/5 on `e94a9359d`, with zero findings and no unresolved threads. GitHub confirms this head is conflict-free and ready for review. ## Risks Activity remains an optional hint and grants no access. A successful helper exit means publication to the pipe; Runner persists accepted events through its existing durable outbox. Full queues, unavailable helpers or failed writes can drop a hint without failing the operation. Context reads are local and do not wait for the control loop. Publishers have a 500 ms lock deadline and a 2 KiB record limit. IPC lives outside durable state. Idle transport blocks; cleanup removes its fixed files and empty private directory. The host must provide `/usr/bin/mkfifo`. Rebuild matching Runner/provider artifacts for the updated source identities. No dependency, migration, live provider call or deployment is included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact serving snapshot is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
153 lines
8.4 KiB
Markdown
153 lines
8.4 KiB
Markdown
# PRP v1/v2 Contract
|
||
|
||
The JSON Schema files in `schemas/` are the language-neutral source of truth for
|
||
Paperclip Runner Protocol versions 1 and 2. The fixtures in `fixtures/` define
|
||
accepted and rejected compatibility cases.
|
||
|
||
## Compatibility
|
||
|
||
- `protocolVersion`, `fixtureVersion`, and `event.schemaVersion` are required.
|
||
- A consumer fails closed when a required version or schema discriminator is
|
||
not supported.
|
||
- A v1 envelope can contain an unknown optional property when its schema marks
|
||
that object as extensible.
|
||
- A consumer ignores an unknown optional property until a later contract gives
|
||
it meaning.
|
||
- A required field, enum value, or typed structured-input field is not optional.
|
||
- Question and answer identifiers are stable across the provider boundary.
|
||
- Peers negotiate the highest mutually supported protocol version. Existing v1
|
||
runners remain compatible but cannot receive v2-only session-goal commands.
|
||
|
||
The `unknown-optional-fields.json` fixture must be accepted. The
|
||
`unsupported-required-version.json` fixture must be rejected.
|
||
|
||
## Session goals (v2)
|
||
|
||
PRP v2 adds a provider-neutral durable session-goal lifecycle. Every v2
|
||
capability snapshot includes `sessionGoals`, even when its availability is
|
||
`unsupported` or `policy_disabled`. Paperclip sends controls only when the
|
||
negotiated capability advertises the corresponding action.
|
||
|
||
Commands:
|
||
|
||
- `session.goal.get`
|
||
- `session.goal.set` for objective, status, and optional token budget changes
|
||
- `session.goal.clear`
|
||
|
||
Events:
|
||
|
||
- `session.capabilities.updated`
|
||
- `session.goal.snapshot`
|
||
- `session.goal.updated`
|
||
- `session.goal.cleared`
|
||
|
||
Goal state is separate from Paperclip's company/business goal hierarchy. The
|
||
snapshot distinguishes the durable status from `workingNow`, because an active
|
||
goal can be idle between autonomous turns. A runner emits the full capability
|
||
and authoritative snapshot after every session open or resume. Missing v1
|
||
capability is unsupported; clients do not infer support from an adapter name.
|
||
|
||
## Harness credential updates
|
||
|
||
`auth.credentials.updated` is a P0 runner event available in the v1, v2, and v3 event
|
||
schemas. Its payload uses `paperclip.auth.credentials.updated.v1` and contains
|
||
only `provider`, `bindingId`, `revision`, and `detection`. No tokens, account
|
||
identifiers, credential paths, or credential fingerprints are permitted.
|
||
|
||
An updated consumer opts in with `observeCredentials: true` in `run.prepare`,
|
||
before `session.open`. The bundled runner transport opts in for fresh sessions;
|
||
set its `observeCredentials` option to `false` to disable observation for a
|
||
fresh session. Older hosts that omit the field receive no new event. Consumers
|
||
must update their schemas before opting in: an event-enum addition is not an
|
||
unknown optional property. Existing runner binaries ignore the optional flag
|
||
and do not provide observation; absence of an event does not establish support.
|
||
|
||
The current Unix runner observes native Codex's managed `CODEX_HOME/auth.json`
|
||
and the Codex/Grok ACPX session homes. It uses a non-recursive directory watch
|
||
plus a one-second polling fallback, so atomic file replacement is handled.
|
||
Grok's initial inline subscription credential is the baseline, allowing a
|
||
refresh during launcher startup to be detected. A first valid credential file
|
||
without a prior baseline establishes the baseline without claiming a refresh.
|
||
Account identity changes also emit updates, even when token values stay the
|
||
same, and establish the baseline for later refreshes on the new account.
|
||
Hosts must evaluate the current credential through their private credential
|
||
channel and account checks; the event does not authorize an account switch.
|
||
Metadata-only changes, partial writes, and deletion do not emit updates.
|
||
Unsupported harnesses and refreshes retained only in memory need
|
||
their own observer or native-hook integration; adding this event does not make
|
||
them observable.
|
||
|
||
`bindingId: "harness-auth"` identifies the selected harness authentication
|
||
within the envelope's normalized session. Revisions increase for that binding
|
||
and survive runner restarts and warm run attachments. The baseline fingerprint
|
||
and pending event are persisted together in a private runner state file. Source
|
||
retries retain the same event identity until the durable PRP outbox accepts it.
|
||
Hosts should commit an idempotent reaction or durable work item through
|
||
`onCommittedEvent` before returning and advancing the cumulative ACK.
|
||
The bundled transport exposes `onCredentialsUpdated(event)` for this commit.
|
||
It awaits the callback before ACK, including during warm idle, attachment, and
|
||
provider cleanup. Callback failures leave delivery unacknowledged for retry;
|
||
callbacks must therefore deduplicate by the original source event identity.
|
||
When configured, credential updates bypass the turn notification queue.
|
||
Paperclip commits them directly to the run log in a separate credential source
|
||
namespace, preserving the original PRP envelope and avoiding collisions with
|
||
the driver's normalized event sequence. A turn-only notification consumer
|
||
cannot reliably handle idle or cleanup updates; use this callback instead.
|
||
|
||
`detection: "credential_store"` reports an observed credential change, not proof
|
||
that an OAuth refresh succeeded. Multiple changes between observations can be
|
||
coalesced; this is not an audit of every refresh exchange. The schema also
|
||
allows `native_hook` for integrations with an explicit signal. Observation
|
||
continues during warm idle and checks at turn terminal and close boundaries,
|
||
including Grok's private refresh handoff after live-store cleanup. Retained
|
||
runner observations enter the durable outbox before a suspend or shutdown
|
||
command completes, without draining the provider backlog. A final check after
|
||
provider cleanup also enters the outbox and waits for the host ACK. The runner
|
||
retains its terminal recovery fence if that delivery fails.
|
||
|
||
The event authorizes no credential access or write on its own. A host that
|
||
needs rotated token values must use a separately authorized handoff with its
|
||
existing account-identity and freshness safeguards. The default Paperclip
|
||
credential copy-back behavior remains in place.
|
||
|
||
## Scope
|
||
|
||
The first provider descriptor and adapter fixture cover Codex only. The schemas
|
||
for provider-neutral events and semantic receipts do not enable those actions.
|
||
Discovery and authorization are separate contracts.
|
||
|
||
The conformance manifest records every source file and its SHA-256 digest. Run
|
||
`pnpm generate:protocol-manifest` from this package after a source change. CI
|
||
runs the same generator with `--check` to reject drift. This check also compiles
|
||
the JSON Schemas and validates every replay, question, and cross-language
|
||
conformance fixture against its declared schema.
|
||
|
||
The files in `fixtures/replay/golden/` are deterministic reducer oracles. Each
|
||
accepted replay fixture has a complete session snapshot and a compact parity
|
||
summary. `pnpm generate:replay-goldens` updates them after an intentional
|
||
reducer change; package build and CI fail when they drift.
|
||
|
||
The files in `fixtures/local-runner/scripts/` drive the package-local fake
|
||
harness. They cover successful, failed, interrupted, interactive, duplicate
|
||
terminal, process-cleanup, and oversized-frame behavior without starting a
|
||
production adapter.
|
||
|
||
`runtime.activity.recorded` is a P1 Runner event in v1, v2 and v3 with exactly
|
||
`{ namespace, activityKey }`. Both opaque identifiers contain 1–160 Unicode code
|
||
points and exclude C0 controls and DEL. `namespace` groups a producer's activity
|
||
types; `activityKey` names an activity within that group. Consumers use the pair
|
||
to select a response to activity in the run. The authenticated outer envelope
|
||
provides run and session identity, source sequence and emission time. Core resolves
|
||
the run's company, task and agent through its existing authorized run binding;
|
||
callers cannot supply those identities in the payload. Recording activity does
|
||
not independently verify the reported action or grant access to any resource.
|
||
The event uses the ordinary durable outbox, acknowledgement and replay contract.
|
||
|
||
Local clients capture an opaque context with `paperclip-runnerd notification-context`
|
||
before an operation and invoke `record-activity CONTEXT NAMESPACE ACTIVITY_KEY`
|
||
after success. Runner publishes its matching absolute executable as host-owned
|
||
`PAPERCLIP_RUNTIME_NOTIFICATION_HELPER` and the stable endpoint descriptor path
|
||
as `PAPERCLIP_RUNTIME_NOTIFICATION_CONTEXT`. Configured task environment cannot
|
||
override either value. Context is invalidated on warm attachment and process
|
||
recovery, so a caller cannot attribute a later completion to a different run.
|