diff --git a/packages/paperclip-runner/docs/architecture.md b/packages/paperclip-runner/docs/architecture.md index 82f57155f5a..8eb1a35e547 100644 --- a/packages/paperclip-runner/docs/architecture.md +++ b/packages/paperclip-runner/docs/architecture.md @@ -196,6 +196,38 @@ lease expiry, storage pressure, drain, and revoke. The browser recovery page calls that peer through guarded package-local Vite middleware and renders only safe trace fields. +## Phase 7 capability-model boundary + +```text +Paperclip skill + 7 references Paperclip Evals corpus (106 cases) + | | + +-------------------+------------------+ + v + generated capability contract (258 rows, 41 MCP aliases) + | + +-----------------------+-----------------------+ + v v v + semantic tool catalog authorization engine eval conformance suite + \ | / + \ v / + +-----> in-process mock ControlPlanePort <--+ + | + v + read-only browser scenario explorer +``` + +Phase 7 is a package-local model of a native Paperclip run. It classifies every +capability as control-plane-owned, always-agent-tool, or optional-agent-tool, +exposes the always/optional set as a transport-neutral semantic tool catalog, +gates optional tools behind grants, and proves 106 eval-derived cases against an +in-process mock `ControlPlanePort`. The mock adapter is the only coupling point, +so a real adapter can replace it later without touching the catalog, +authorization rules, or conformance suite. Phase 7 contacts no Paperclip +service, database, ACPX session, or provider credential; the +[forbidden-imports checker](#forbidden-dependencies) keeps it that way. Real +integration is Phase 8 (ACPX) and requires separate approval; see +[the future binding boundary](phase-07-future-binding-boundary.md). + ## Future integration rule The [implementation plan](../spec/paperclip-native-runner-implementation-plan.md) diff --git a/packages/paperclip-runner/docs/index.md b/packages/paperclip-runner/docs/index.md index 178856d60e1..28bf11705c4 100644 --- a/packages/paperclip-runner/docs/index.md +++ b/packages/paperclip-runner/docs/index.md @@ -14,6 +14,7 @@ Each tutorial starts with an explanation of what the phase is and what the phase - [Phase 4b: run the live Codex protocol console](tutorials/phase-04b-live-console.md) — chat with a live session in the browser, steer it, stop it three ways, answer its requests, change its goal, break its connection, and replay the record. - [Phase 5: run the SDK console and mini consumer](tutorials/phase-05-sdk-console.md) — exercise the versioned public browser/React surface in two independent consumers against fake and real drivers. - [Phase 6: run the thin Paperclip adapter](tutorials/phase-06-thin-paperclip-adapter.md) — prove mock/real conformance, run one feature-flagged local task, inspect replay/finalization, disable the kill switch, and verify legacy fallback. +- [Phase 7: explore the capability contract and scenario explorer](tutorials/phase-07-scenario-explorer.md) — prove the 106-case conformance suite, open the read-only browser explorer over the mock control plane, and run the focused verification set, all from a clean checkout with no Paperclip service or external eval repository. - [Cumulative end-to-end tutorial](tutorials/end-to-end.md) — the shortest complete workflow available at the current phase. ## Reference @@ -27,6 +28,16 @@ Each tutorial starts with an explanation of what the phase is and what the phase - [Phase 4b live console](phase-04b-live-console.md) - [Phase 5 browser SDK and reference console](phase-05-sdk.md) - [Phase 6 thin Paperclip adapter](phase-06-thin-paperclip-adapter.md) +- [Phase 7 capability contract (generated)](phase-07-capability-contract.md) +- [Phase 7 capability disposition](phase-07-capability-disposition.md) +- [Phase 7 mock ControlPlanePort](phase-07-mock-control-plane-port.md) +- [Phase 7 semantic tool catalog](phase-07-semantic-tools.md) +- [Phase 7 authorization and exposure](phase-07-authorization-and-exposure.md) +- [Phase 7 eval-derived conformance](phase-07-eval-conformance.md) +- [Phase 7 browser scenario explorer](phase-07-scenario-explorer.md) +- [Phase 7 future binding boundary (Phase 8 / ACPX)](phase-07-future-binding-boundary.md) +- [Phase 7 verification commands](phase-07-verification-commands.md) +- [Phase 7 scenario explorer UX interaction map](design/phase-7-scenario-explorer-ux.md) - [Engineering journal guide](journal.md) - [Dated shadcn/ui and AI Elements compatibility note](research/2026-08-07-ui-library-compatibility.md) - [Phase 4b live-console interaction map](design/phase-4b-interaction-map.md) diff --git a/packages/paperclip-runner/docs/phase-07-authorization-and-exposure.md b/packages/paperclip-runner/docs/phase-07-authorization-and-exposure.md new file mode 100644 index 00000000000..05e5c61e76e --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-authorization-and-exposure.md @@ -0,0 +1,99 @@ +# Phase 7 Authorization and Exposure + +Two decisions gate every capability: **exposure** (is the tool even offered to +this actor?) and **invocation** (may this specific call proceed?). The +authorization engine answers both, produces typed denials that carry no +protected state, and redacts secrets before any observable boundary. + +Sources: `src/tools/phase7-tool-authorization.ts`, +`src/tools/phase7-semantic-tool-runtime.ts`, `src/tools/phase7-tool-bindings.ts`. + +## Three-way classification + +- **Control-plane-owned** — a frozen operation list (`checkout_task`, + `release_task`, `select_work`, `route_wake`, `enforce_budget`, + `append_audit_record`, `persist_run`, `replay_run`, `schedule_blocker_wake`, + `reconcile_run`). `authorizeInvocation` short-circuits these to outcome + `absent` with reason `control_plane_owned_operation`. No agent tool exists for + them and none can be invoked. +- **Always-agent tool** — exposed to any actor whose task is active. +- **Optional-agent tool** — exposed only when a grant unlocks it. + +## Exposure vs. invocation + +- `computeVisibleTools()` walks the whole catalog and records, per tool, an + exposure of `exposed` or `absent`, returning only the allowed tools. +- `authorizeInvocation()` re-evaluates at call time, producing `allowed` or + `denied`. + +Invocation is evaluated in order and fails at the first rule that rejects: +policy-denied operation IDs; task-mode membership +(`operation_not_available_in_task_mode`); role check +(`actor_role_not_authorized`); policy-denied claims +(`required_claim_denied_by_policy`); missing grants +(`required_claim_missing`); `read_secret_value` requires +`policy.allowSecretValueAccess` (`secret_value_access_disabled`); +`generic_api_request` requires the escape hatch to be enabled and the target to +be allowlisted (`generic_escape_hatch_disabled` / `..._target_denied`); and the +interaction-kind policy for `request_human_input`. Allowed calls carry the +reason `task_scoped_always_tool` or `required_claims_granted`. + +Self-approval is prohibited even with the role and an explicit decision claim. + +## How grants unlock optional tools + +The actor's grants are the union of `capabilityGrants` (the actor's own) and +`scenarioGrants` (seeded per scenario). An optional tool unlocks only when +**every** entry in its `requiredClaims` is present in that union. Each +authorization record stores the `consideredClaims`, `grantedClaims`, and +`missingClaims`, so a denial explains exactly which claim was absent. + +## Typed denials carry no protected state + +A denial is a `Phase7PolicyDenial`: `ok: false` with an `error.code` of +`policy_denied`, `operation_absent`, `input_invalid`, or `operation_unsupported`, +the `operationId`, a generic `reason`, and the authorization record. It carries +no fixture state and no protected payload. If the underlying mock operation +throws, the runtime emits `operation_unsupported` with reason +`mock_operation_rejected` and swallows the underlying error, so protected state +cannot leak through an exception. + +## Secret redaction (the PAP-16909 rule) + +Redaction is two layers: + +1. **Model-only delivery is separated from every observable boundary.** + `invoke()` returns a redacted `observableResult`. The only path to an + unredacted secret is `invokeForModel()`, which returns a frozen capsule whose + `toJSON()` and default serialization always yield the redacted form; only an + explicit `readModelResult()` opens the raw value. The capsule uses a distinct + schema (`paperclip.phase7.model-tool-result.v1`) so it cannot be assigned to + an observable sink by mistake. +2. **Path redaction rules.** `read_secret_value` redacts `$.value` to + `[SECRET_VALUE]` across output, error, and authorization-record channels; + `generic_api_request` redacts `$.headers.authorization` and + `$.headers.cookie` to `[REDACTED]` across all channels. The scenario runner + applies the same redaction again defensively at the explorer artifact + boundary and injects only a placeholder secret value. + +This is the security-gate outcome (PAP-16902) and its remediation (PAP-16909): +a real secret value reaches the model surface only, never a trace, artifact, or +browser view. + +## Running the tests + +```sh +pnpm --filter @paperclipai/paperclip-runner exec vitest run \ + src/tools/phase7-semantic-tools.test.ts +``` + +The "exposure and authorization" and "security policy" describes cover typed +denials without protected state, model-only secret delivery, the self-approval +prohibition, and the test-only, explicitly granted, allowlisted escape hatch. + +## Related + +- [Capability disposition](phase-07-capability-disposition.md) +- [Semantic tool catalog](phase-07-semantic-tools.md) +- [Scenario explorer](phase-07-scenario-explorer.md) + diff --git a/packages/paperclip-runner/docs/phase-07-capability-disposition.md b/packages/paperclip-runner/docs/phase-07-capability-disposition.md new file mode 100644 index 00000000000..bd9c3b72e1e --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-capability-disposition.md @@ -0,0 +1,85 @@ +# Phase 7 Capability Disposition + +This page explains how Phase 7 classifies every Paperclip capability. The +classification itself lives in a generated file, +[the Phase 7A capability contract](phase-07-capability-contract.md), which +carries a "DO NOT EDIT" header and is rewritten by `generate:phase7-inventory`. +Read this page to understand what the generated table means; read the table for +the authoritative rows. + +## Normative sources + +Only two sources are normative: + +1. The Paperclip skill and its seven references (`SKILL.md` plus + `references/*.md`), contributing **152 headings**. +2. The Paperclip Evals corpus, contributing **106 cases across 16 groups**. + +Together these produce **258 normative rows**. The legacy Paperclip MCP tool +surface (**41 tools**) is not a production capability surface; each MCP name is +folded one-to-one into a normative eval row as a traceability alias and inherits +that row's disposition. The contract prints the alias index only so the +normative target is easy to audit. + +## The three dispositions + +Every capability is classified as exactly one of: + +- **`control_plane_owned`** — the control plane performs this; no agent tool + exists for it. Checkout, inbox resolution, blocker diagnostics, and status + arbitration are control-plane-owned. In the explorer these appear as + control-plane actions labelled "no agent tool exists for this," never as a + callable tool. +- **`always_agent_tool`** — the agent always has this as a semantic tool. Adding + a comment, writing a document, opening an interaction, and registering a + deliverable are always-agent-tool capabilities. +- **`optional_agent_tool`** — the agent may have this tool, but only when a + grant unlocks it. Absent the grant, the capability is not exposed and calling + it is denied. See [authorization and exposure](phase-07-authorization-and-exposure.md). + +The disposition is what makes the boundary legible: it states, per capability, +whether an agent can act, must wait for the control plane, or needs a grant +first. + +## Reading the generated contract + +The contract has three parts: + +- **Baseline counts** — the heading, case, row, and alias totals above, plus the + per-group case table. +- **Skill / reference rows** — one row per heading, its primary disposition, and + its `file:line` source anchor. +- **Legacy MCP alias index** — each MCP name, the normative row it folds into, + the inherited disposition, and its `packages/mcp-server/src/tools.ts` anchor. + +## Regenerating and checking + +Generation reads the live in-repo skill/reference sources, the legacy MCP tool +source, and the Paperclip Evals corpus, so it **requires** the external eval +repository (via `PAPERCLIP_EVALS_ROOT` or a known local path) and is not part of +the offline path. Checking and testing read only the checked-in derivatives +under `spec/phase-07/` and need no external repository. + +```sh +# Rewrite every generated file. Requires the external Paperclip Evals corpus. +pnpm --filter @paperclipai/paperclip-runner generate:phase7-inventory + +# Validate counts, uniqueness, normative dispositions, one-to-one MCP folds, +# required fields, and generated-file drift. Offline; no external eval repo. +pnpm --filter @paperclipai/paperclip-runner check:phase7-inventory + +# Prove the validator rejects an independent MCP classification and rejects +# missing, duplicate, or unknown MCP folds. Offline. +pnpm --filter @paperclipai/paperclip-runner test:phase7-inventory +``` + +`check:phase7-inventory` diffs the checked-in generated files against what the +live in-repo sources imply and fails on any drift or stale anchor, so the +contract cannot silently fall out of sync. + +## Related + +- [Semantic tool catalog](phase-07-semantic-tools.md) +- [Authorization and exposure](phase-07-authorization-and-exposure.md) +- [Eval conformance](phase-07-eval-conformance.md) + diff --git a/packages/paperclip-runner/docs/phase-07-eval-conformance.md b/packages/paperclip-runner/docs/phase-07-eval-conformance.md new file mode 100644 index 00000000000..76f71ab8c57 --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-eval-conformance.md @@ -0,0 +1,83 @@ +# Phase 7 Eval-Derived Conformance + +Phase 7E turns the Paperclip Evals corpus into an executable, offline +conformance suite. The suite derives its cases from the checked-in Phase 7 +traceability derivative (`spec/phase-07/eval-traceability.yaml`) — it does +**not** clone or read an external eval repository, and it starts only the +in-process mock control plane. It hard-fails unless the derivative declares +schema version 2, exactly 106 rows, exactly 16 groups, and unique case IDs. + +Sources: `src/conformance/phase7-eval-suite.ts` and its test +`src/conformance/phase7-eval-suite.test.ts`; the reporter +`scripts/run-phase7-eval-suite.mjs`. + +## The corpus + +- **106 cases across 16 groups.** Per-group counts: hb 5, co 6, st 8, cm 6, + se 4, su 4, bl 5, dp 3, ix 9, ap 6, ar 4, er 9, rf 22, mh 4, rs 3, wk 8. +- Each case declares an actor role, a task mode, an input scenario, and the + expected outcome, all bound to a row in + [the capability contract](phase-07-capability-contract.md). + +## Assertion classes + +Every case belongs to one assertion class, and each class checks a different +kind of invariant: + +- **`agent_tool_contract`** — a semantic tool call produces the expected typed + effect and state change. +- **`authorization_policy`** — a capability is exposed, denied, or unlocked by a + grant exactly as its disposition requires. +- **`control_plane_invariant`** — a control-plane-owned action happens without + any agent tool, and no tool can perform it. +- **`combined_multi_hop`** — a sequence of operations across turns produces the + expected cumulative state and respects forbidden-operation rules. +- **`restraint_no_call`** — the correct behavior is to make **no** call; the + case passes only if the agent deliberately does nothing further. + +## Fake-agent matrix and bounded Codex sample + +The suite executes each case with a deterministic fake agent whose plan is +fixed by a fixture seed, so repeat runs are byte-identical. Optional-tool rows +are run twice — once with the unlocking grants (must be allowed and must mutate +state) and once ungranted (must be absent or denied with no state change); +control-plane-owned rows remain absent in every configuration; and +`restraint_no_call` rows must produce an empty state diff. + +The fake-agent surface is 14 always-agent tools plus 4 optional tools unlocked +by four seed grants (`discovery:tasks:read`, `discovery:agents:read`, +`delegation:tasks:create`, `governance:approvals:request`) — **18 operations**. +The suite binds that surface through both the fake-agent and Codex bindings and +asserts the two operation lists are byte-identical (**18/18**). + +A **bounded Codex binding sample** picks one representative case from nine +groups (`hb`, `dp`, `bl`, `ap`, `ar`, `ix`, `mh`, `rs`, `wk`) and checks that +`checkout_task` is absent from the Codex surface while every other sampled +operation is present. It is an offline parity check; no real Codex or network is +contacted, and the browser explorer holds no credential. + +## Running it + +```sh +# Run the 106-case suite in-process (one vitest file drives all cases). +pnpm --filter @paperclipai/paperclip-runner test:phase7-evals + +# Build the public surface and write the parity report with per-group counts, +# assertion classes, the fake-agent matrix, the bounded Codex sample, and the +# semantic-operation execution counts. +pnpm --filter @paperclipai/paperclip-runner report:phase7-evals +``` + +The reporter writes `knowledge/evidence/phase-07/eval-parity-report.{json,md}`. +Each failure carries its case ID, assertion class, semantic operation, +authorization decision, and final state diff. The report is generated on demand +and is not committed; delete it before running `docs:validate` (it carries no +OKF frontmatter). See the +[verification commands reference](phase-07-verification-commands.md). + +## Related + +- [Capability disposition](phase-07-capability-disposition.md) +- [Semantic tool catalog](phase-07-semantic-tools.md) +- [Scenario explorer](phase-07-scenario-explorer.md) + diff --git a/packages/paperclip-runner/docs/phase-07-future-binding-boundary.md b/packages/paperclip-runner/docs/phase-07-future-binding-boundary.md new file mode 100644 index 00000000000..772cb35bd38 --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-future-binding-boundary.md @@ -0,0 +1,52 @@ +# Phase 7 Future Binding Boundary (Phase 8 / ACPX) + +Phase 7 is a package-local model. It does not integrate the runner into +Paperclip, and it must not. This page states exactly what Phase 7 defers to +Phase 8 and what clean seam it preserves so the deferral stays cheap. + +## What Phase 7 does not do + +Nothing in Phase 7 contacts a real Paperclip control plane, database, ACPX +session, or provider credential. The 106-case conformance suite and the browser +explorer run entirely against the in-process +[mock ControlPlanePort](phase-07-mock-control-plane-port.md) and checked-in +fixtures. The [forbidden-imports boundary](architecture.md) rejects any import +that crosses into `server/`, `ui/`, `cli/`, `@paperclipai/db`, or other +Paperclip workspace internals. + +## The preserved seam + +The dependency arrow always points from an implementation toward a contract. +The package owns two contracts a future consumer implements: + +- **`ControlPlanePort`** — the narrow surface through which a runner opens a + run, appends ordered events, records semantic operations, and submits a + terminal result. Phase 7 injects the *mock* adapter behind this port; Phase 8 + injects a real one. The catalog, authorization engine, and conformance suite + bind to the port, not to any adapter. +- **`NativeSessionBackend`** — the normalized session surface for a future + control-plane consumer. + +Because the port is the only coupling point, Phase 8 replaces the adapter +without touching the tool catalog, the authorization rules, or the eval-derived +conformance suite. The package remains independently buildable, testable, and +runnable against the mock adapter after the real one exists. + +## What Phase 8 (ACPX) will bind + +Phase 8 binds a real Paperclip `ControlPlanePort` implementation behind the same +seam so the semantic tools and authorization engine act against a live control +plane instead of the mock. That work is out of scope here and requires separate +CTO approval at the Phase 7 checkpoint (`PAP-16908`). Until then: + +- ACPX is Phase 8, not Phase 7. +- No Phase 7 documentation claims Phase 8 capability. +- Real integration is a separately reviewed phase, consistent with the + [future integration rule](architecture.md) recorded for every prior phase. + +## Related + +- [Architecture and dependency boundary](architecture.md) +- [Mock ControlPlanePort](phase-07-mock-control-plane-port.md) +- [Capability disposition](phase-07-capability-disposition.md) + diff --git a/packages/paperclip-runner/docs/phase-07-mock-control-plane-port.md b/packages/paperclip-runner/docs/phase-07-mock-control-plane-port.md new file mode 100644 index 00000000000..3add9e4526f --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-mock-control-plane-port.md @@ -0,0 +1,95 @@ +# Phase 7 Mock ControlPlanePort + +`Phase7MockControlPlaneAdapter` is a deterministic, serializable, in-memory +model of Paperclip's control-plane **semantics** — not its transport. It imports +no server route, service, database, or provider binding, and it holds no +credential. + +Sources: `src/mock-core/phase7-mock-control-plane-adapter.ts`, +`src/mock-core/phase7-control-plane-types.ts`. Shared contract: +`src/conformance/control-plane-port.ts`. + +## Entity domains + +The adapter models ten renderable entity domains, which are exactly the domains +the [scenario explorer](phase-07-scenario-explorer.md) diffs: + +`tasks`, `comments`, `documents`, `interactions`, `approvals`, `artifacts` +(artifacts and work products), `blockers`, `workspace` (workspace services), +`budget`, and `run`. + +The fixture state also carries supporting records — company, actors, wakes, +audit, decisions, idempotency, faults, counters, and a fixture clock — that back +behavior but are not diff domains. + +## Operations + +- **Lifecycle:** `start()` / `stop()` only change local object state. +- **Runs:** `openRun()` / `openFixtureRun()` return a run context with atomic + checkout; `context(runId)` reads it back. +- **Events:** `appendEvent()` accepts both canonical PRP events and native run + events with deduplication, replay-conflict detection, and contiguous-sequence + tracking; `replayEvents()` replays from an exclusive cursor. +- **Commands:** `applyCommand(envelope)` executes a semantic command with + idempotency and an optimistic `expectedRevision`. +- **Sessions:** `loadSessionCheckpoint()` / `checkpointSession()`. +- **Terminal:** `completeRun()` reconciles a terminal disposition — `done`, + `blocked`, `needs_review`, or `yielded` — and fans out blocker wakes. +- **Introspection:** `snapshot()`, `decisionRecords()`, `serialize()`, and + static `restore()`. + +The eighteen semantic command kinds (`report_progress`, `write_document`, +`request_human_input`, `resolve_human_input`, `register_deliverable`, +`set_dependencies`, `create_task`, `request_approval`, `decide_approval`, +`comment_on_approval`, `control_workspace_service`, `record_budget_usage`, +`finish_task`, `block_task`, `request_review`, `release_task`, `schedule_wake`) +are the mock side of the [semantic tool catalog](phase-07-semantic-tools.md). + +## Determinism + +- **Fixture clock.** Every timestamp derives from `clock.epochMs + tick*1000` + and the tick increments on each read. The default epoch is + `2026-08-09T00:00:00Z`. There is no RNG anywhere. +- **Deterministic IDs.** Every generated ID is `fixture--` from a + monotonic per-kind counter. +- **Immutability.** Inputs and outputs are `structuredClone`d and every returned + value is deep-frozen. Canonical JSON with sorted keys backs every dedupe and + replay comparison. +- **Serializable.** `serialize()` / `restore()` round-trip under the schema tag + `paperclip.phase7.mock-state.v1`. +- **Scripted faults.** A fault rule injects a bounded number of + `retryable_error` or `lost_ack` effects; a lost-ack command commits once and + returns its stored result on retry. + +## Boundary + +The adapter is a pure state machine: no network, no database, no provider SDK, +no credentials. Secrets appearing in wake payloads are redacted against +`/authorization|credential|api.?key|secret|token/i`; the adapter test seeds +`apiKey` and `Bearer` values and asserts they never surface. See +[authorization and exposure](phase-07-authorization-and-exposure.md). + +## Shared conformance + +The adapter satisfies the same `runControlPlanePortConformance` contract used by +earlier phases: it rejects open-binding violations, recovers from source gaps, +treats duplicates idempotently, and rejects event-id, sequence, replay-binding, +and result mutations (failing closed with `native_event_replay_conflict`). + +## Running the tests + +```sh +pnpm --filter @paperclipai/paperclip-runner exec vitest run \ + src/conformance/control-plane-port.test.ts \ + src/mock-core/phase7-mock-control-plane-adapter.test.ts +``` + +Nine tests, no npm alias of their own — they also run inside +`pnpm --filter @paperclipai/paperclip-runner test`. + +## Related + +- [Semantic tool catalog](phase-07-semantic-tools.md) +- [Authorization and exposure](phase-07-authorization-and-exposure.md) +- [Future binding boundary](phase-07-future-binding-boundary.md) + diff --git a/packages/paperclip-runner/docs/phase-07-scenario-explorer.md b/packages/paperclip-runner/docs/phase-07-scenario-explorer.md new file mode 100644 index 00000000000..4e5fe85a0bc --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-scenario-explorer.md @@ -0,0 +1,86 @@ +# Phase 7 Browser Scenario Explorer + +The scenario explorer is a read-only browser surface over all 106 conformance +cases. It runs each scenario in the page against the +[mock control plane](phase-07-mock-control-plane-port.md) through the +[semantic tool runtime](phase-07-semantic-tools.md) and renders the resulting +run artifact. It never re-judges parity, never leaves its own origin, and holds +no credential. + +Sources: `src/phase7/` (scenario index, runner, parity, state diff) and +`examples/phase7-explorer/`. Interaction contract: +[Phase 7 scenario explorer UX](design/phase-7-scenario-explorer-ux.md). + +## The run artifact + +Running a scenario produces one immutable artifact carrying: + +- **Tool exposure** — which semantic tools were available, and for optional + tools, the grant that unlocked each one. +- **Control-plane actions** — control-plane-owned steps (for example checkout), + each labelled "no agent tool exists for this." +- **Authorization records** — allow and typed-deny decisions; a denial carries + the missing claim and no protected task data. +- **State diff** — an immutable diff over the ten entity domains, with unchanged + domains collapsed. +- **Parity verdict** — the runtime's own verdict, plus Phase 7E's per-case + result carried through in a separately labelled block. The explorer displays + these; it does not recompute them. + +## Read-only and frozen SDK + +The explorer imports the frozen **0.1.2** public SDK through the package-local +alias `@paperclip-runner-local/phase7`, which is deliberately not the published +package name. It adds no SDK export and changes no published surface. Fake mode +runs entirely in the page against checked-in fixtures and renders fixture time +only, so two loads of the same route produce identical settled DOM. + +## What the explorer shows + +- **Home** — 16 group facets whose counts sum to 106, corpus stats, and example + deep links. +- **Picker** — a listbox with live filter chips, disabled zero-count values, and + a clear-filters control. +- **Transcript** — both channels (agent-visible and control-plane). +- **Inspector** — four tabs: context (exposure and grants), authorization + (allow/deny), state diff, and traceability. + +## Boundary + +- No network request leaves the explorer's own origin. +- `localStorage` holds no run artifact, grant, or fixture payload. +- Codex mode is a disabled option with a stated reason; wiring it to the Phase 5 + relay is deferred, and the browser holds no provider credential either way. +- Secrets are redacted before display; redaction chips name the rule and never + the value. See [authorization and exposure](phase-07-authorization-and-exposure.md). + +## Accessibility + +The 7F browser suite asserts: listbox arrow-key navigation with +`aria-activedescendant`, WAI-ARIA tab arrow-key activation with exactly one +tablist, every interactive control named, a polite live region announcing the +settled verdict, three landmarks, a single `h1`, and — at 390px — one segment at +a time with zero horizontal overflow. + +## Running it + +```sh +# Open the explorer on 127.0.0.1:4183. +pnpm --filter @paperclipai/paperclip-runner demo:phase7 + +# Scenario runtime, explorer components, and route determinism (45 tests). +pnpm --filter @paperclipai/paperclip-runner test:phase7 + +# Browser IA, determinism, evidence routes, boundary, a11y, responsive (19). +pnpm --filter @paperclipai/paperclip-runner test:browser:phase7 + +# Deterministic 24-image acceptance set (12 routes x 2 viewports). +pnpm --filter @paperclipai/paperclip-runner record:phase7 +``` + +## Related + +- [Eval conformance](phase-07-eval-conformance.md) +- [Verification commands](phase-07-verification-commands.md) +- [Phase 7F explorer verification](../knowledge/evidence/2026-08-09-phase-07f-explorer-verification.md) + diff --git a/packages/paperclip-runner/docs/phase-07-semantic-tools.md b/packages/paperclip-runner/docs/phase-07-semantic-tools.md new file mode 100644 index 00000000000..0578be33296 --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-semantic-tools.md @@ -0,0 +1,87 @@ +# Phase 7 Semantic Tool Catalog + +The semantic tool catalog is the transport-neutral surface an agent may call. It +is a frozen set of JSON-schema descriptors — no HTTP method, path, or provider +detail. Provider-specific shapes are produced only by *bindings*, so every +provider sees the same operation set. + +Sources: `src/tools/phase7-semantic-tool-catalog.ts`, +`src/tools/phase7-semantic-tool-types.ts`, `src/tools/phase7-tool-bindings.ts`, +barrel `src/tools/index.ts`. + +## Two dispositions, plus a separate control-plane list + +A tool descriptor's `disposition` is either **`always_agent_tool`** or +**`optional_agent_tool`**. Control-plane-owned operations are a separate frozen +list, not a disposition — they are never exposed as tools. See +[capability disposition](phase-07-capability-disposition.md) and +[authorization and exposure](phase-07-authorization-and-exposure.md). + +The catalog holds **36 tools**: 14 always-agent tools and 22 optional tools +across 10 groups. + +### Always-agent tools (14) + +`get_task_context`, `get_task_history`, `list_documents`, `read_document`, +`list_document_revisions`, `report_progress`, `answer_status_question`, +`finish_task`, `block_task`, `request_review`, `write_document`, +`request_human_input`, `register_deliverable`, `inspect_operation_result`. + +### Optional tools (22), by group + +| Group | Tools | +| --- | --- | +| discovery | `search_tasks`, `list_agents`, `list_projects`, `list_goals` | +| delegation_dependencies | `create_task`, `set_dependencies` | +| governance | `list_approvals`, `request_approval`, `decide_approval`, `comment_on_approval` | +| cases | `list_cases`, `upsert_case` | +| workspace_runtime | `get_workspace_runtime`, `control_workspace_service` | +| routines | `list_routines`, `manage_routine` | +| company_skills | `list_company_skills`, `sync_company_skills` | +| secrets | `list_secret_metadata`, `read_secret_value` | +| portability_admin | `export_company`, `administer_company` | +| test_escape_hatch | `generic_api_request` | + +## Descriptor shape + +Each descriptor carries `operationId`, `version` (1), `title`, `description`, +input/output JSON schemas, `disposition`, an optional `optionalGroup`, +`requiredClaims`, and optionally `allowedRoles`, `taskModes`, a `sideEffectClass` +(`read`, `task_write`, `company_write`, `governance`, `workspace_control`, +`secret_read`, `admin`, `test_escape_hatch`), an `idempotency` level, `redaction` +rules, and an abstract `mockCommandMapping`. + +The `mockCommandMapping` is one of `context_read`, `snapshot_read`, +`semantic_command`, `operation_result`, or `mock_extension` — describing *what* +the tool does against the mock, never *how* a transport would carry it. + +## Transport neutrality + +Bindings, not descriptors, produce provider shapes, both derived from the same +`visibleTools.tools` array: + +- `Phase7FakeAgentToolBinding` emits `{operationId, description, inputSchema, + outputSchema}`. +- `Phase7CodexToolBinding` emits `{type: "function", name, description, strict: + true, parameters}`. + +Because both bindings derive from one array, the fake-agent and Codex operation +surfaces are byte-identical. The [conformance suite](phase-07-eval-conformance.md) +asserts this parity (the 18/18 fake-agent/Codex operation matrix). + +## Running the tests + +```sh +pnpm --filter @paperclipai/paperclip-runner exec vitest run \ + src/tools/phase7-semantic-tools.test.ts +``` + +The "catalog" describe asserts a unique, versioned, provider-neutral catalog and +that every optional group is present. + +## Related + +- [Authorization and exposure](phase-07-authorization-and-exposure.md) +- [Mock ControlPlanePort](phase-07-mock-control-plane-port.md) +- [Eval conformance](phase-07-eval-conformance.md) + diff --git a/packages/paperclip-runner/docs/phase-07-verification-commands.md b/packages/paperclip-runner/docs/phase-07-verification-commands.md new file mode 100644 index 00000000000..054125b4318 --- /dev/null +++ b/packages/paperclip-runner/docs/phase-07-verification-commands.md @@ -0,0 +1,42 @@ +# Phase 7 Verification Commands + +Every command here runs from the repository root, is offline and deterministic, +starts no Paperclip service, and holds no credential. All are prefixed +`pnpm --filter @paperclipai/paperclip-runner`. The observed results for the +acceptance record are in +[the Phase 7 verification evidence](../knowledge/evidence/2026-08-09-phase-07-verification.md). + +| Surface | Command | Expected result | +| --- | --- | --- | +| Capability contract completeness and drift | `check:phase7-inventory` | `Phase 7 inventory completeness and generated-output checks passed.` | +| Contract validator negatives | `test:phase7-inventory` | 4 tests pass | +| Mock control plane + shared port | `exec vitest run src/conformance/control-plane-port.test.ts src/mock-core/phase7-mock-control-plane-adapter.test.ts` | 9 tests pass | +| Semantic tools + authorization/redaction | `exec vitest run src/tools/phase7-semantic-tools.test.ts` | 9 tests pass | +| 106-case conformance | `test:phase7-evals` | 1 file (all 106 cases) passes | +| Fake-agent matrix + bounded Codex + parity report | `report:phase7-evals` | `Phase 7 eval conformance passed: 106 cases across 16 groups.` | +| Scenario runtime + explorer + routes | `test:phase7` | 45 tests pass | +| Browser IA, accessibility, determinism, boundary | `test:browser:phase7` | 19 Playwright tests pass | +| Screenshot acceptance set | `record:phase7` | 24 deterministic images | +| Doc links + OKF bundle | `docs:validate` | 62 files, 26 OKF concepts, 5 indexes | + +## Notes + +- **No Rust toolchain is needed** for any Phase 7 command. The full + `verify` target still builds the Rust workspace, but nothing above does. +- **Determinism.** Fake-mode runs render fixture time only, so repeat runs and + repeat screenshots are byte-identical. +- **The parity report is generated on demand.** `report:phase7-evals` writes + `knowledge/evidence/phase-07/eval-parity-report.{json,md}`, which are not + committed. Because the `.md` carries no OKF frontmatter, run `docs:validate` + **before** generating the report, or delete the report first. This dirty-tree + interaction is tracked for the Phase 7H QA gate (`PAP-16907`). +- **Browser libraries.** On minimal or rootless hosts, extract Playwright's + browser libraries first; the package's `verify:rootless` target shows the + pattern. + +## Related + +- [Clean-start tutorial](tutorials/phase-07-scenario-explorer.md) +- [Eval conformance](phase-07-eval-conformance.md) +- [Scenario explorer](phase-07-scenario-explorer.md) + diff --git a/packages/paperclip-runner/docs/tutorials/phase-07-scenario-explorer.md b/packages/paperclip-runner/docs/tutorials/phase-07-scenario-explorer.md new file mode 100644 index 00000000000..0b566d53e43 --- /dev/null +++ b/packages/paperclip-runner/docs/tutorials/phase-07-scenario-explorer.md @@ -0,0 +1,188 @@ +# Phase 7 Clean-Start Tutorial: Capability Contract, Mock Control Plane, and Scenario Explorer + +**Time to first success: about 5 minutes.** Two commands take you from a clean +checkout to 106 passing conformance cases and a browser you can click through. +The whole tutorial runs from the repository root. It starts no Paperclip +service, contacts no Paperclip control plane, clones no external eval +repository, and holds no provider credential. Everything it needs is checked in +under `packages/paperclip-runner/`. + +Phase 7 does not integrate the runner into Paperclip. It builds a package-local +model of what a native Paperclip run *would* do — a deterministic mock control +plane, a transport-neutral semantic tool catalog, an authorization engine, a +106-case conformance suite derived from the Paperclip Evals corpus, and a +read-only browser explorer over all of it. Real integration is Phase 8 and +requires separate approval. See +[the future binding boundary reference](../phase-07-future-binding-boundary.md). + +## What you need + +- Node.js 20 or newer and pnpm 9 or newer. This tutorial was verified with Node + 22.22.2 and pnpm 9.15.4. +- No Rust toolchain. Every command in this tutorial is TypeScript/Node only. (A + full `verify` still builds Rust, but nothing here does.) +- No network access after `pnpm install`. + +Install the package workspace from the repository root: + +```sh +pnpm install --filter @paperclipai/paperclip-runner --lockfile=false --offline --ignore-scripts --dev +``` + +## 1. Prove the 106-case conformance suite (about 1 minute) + +```sh +pnpm --filter @paperclipai/paperclip-runner test:phase7-evals +``` + +This runs the eval-derived conformance suite entirely in-process against the +mock control plane. Expected final line: + +``` + Test Files 1 passed (1) + Tests 1 passed (1) +``` + +The single test file drives all 106 cases across the 16 eval groups. To see the +per-group counts, assertion classes, the fake-agent operation matrix, and the +bounded Codex binding sample, generate the parity report: + +```sh +pnpm --filter @paperclipai/paperclip-runner report:phase7-evals +``` + +Expected final line: + +``` +Phase 7 eval conformance passed: 106 cases across 16 groups. +``` + +The report is written to +`knowledge/evidence/phase-07/eval-parity-report.{json,md}`. It is a +generated-on-demand artifact, not a committed file; a clean checkout does not +contain it. Delete it before you run `pnpm --filter @paperclipai/paperclip-runner docs:validate` +(the report carries no OKF frontmatter and would otherwise fail the knowledge +bundle check — see [Known gaps](#known-gaps-and-boundaries)). + +The 16 groups and case counts are fixed by the capability contract: + +| Group | Cases | Group | Cases | Group | Cases | Group | Cases | +| --- | ---: | --- | ---: | --- | ---: | --- | ---: | +| hb | 5 | co | 6 | st | 8 | cm | 6 | +| se | 4 | su | 4 | bl | 5 | dp | 3 | +| ix | 9 | ap | 6 | ar | 4 | er | 9 | +| rf | 22 | mh | 4 | rs | 3 | wk | 8 | + +## 2. Open the scenario explorer (about 2 minutes) + +```sh +pnpm --filter @paperclipai/paperclip-runner demo:phase7 +``` + +Open `http://127.0.0.1:4183/phase7-explorer/`. The explorer is read-only. It +runs each scenario in the browser against the same mock control plane and +renders the run artifact; it never re-judges parity and never leaves its own +origin. Walk this path: + +1. **Home.** Confirm 16 group facets whose counts sum to 106. +2. **Pick a scenario.** Filter to `ap` and open `ap-mcp-gate-01`. The + authorization view shows a deny row and a "1 deny" tab chip; the typed + denial card carries no task data. +3. **Read a heartbeat.** Open `hb-context-01`. The control-plane checkout entry + is labelled "no agent tool exists for this" — it is control-plane-owned, not + a semantic tool. +4. **Inspect a manager scenario.** Open `rf-api-mgr-heartbeat-01`. Optional + tools are listed with the grant that unlocked them, alongside the + control-plane "no tool" list and a traceability panel. +5. **Confirm the credential boundary.** Codex mode is a disabled option with a + stated reason. `localStorage` holds no run artifact or grant, and the page + makes no request beyond its own assets. + +The explorer imports the frozen `0.1.2` SDK through the package-local +`@paperclip-runner-local/phase7` alias — deliberately not the published package +name. See [the browser explorer reference](../phase-07-scenario-explorer.md). + +## 3. Read what each surface guarantees + +The explorer renders four surfaces produced by the runtime. Each has a +package-local reference page: + +- [Capability disposition](../phase-07-capability-disposition.md) — how every + skill/reference behavior and eval case is classified as control-plane-owned, + always-agent-tool, or optional-agent-tool, and how the generated contract is + produced and checked. +- [Mock ControlPlanePort](../phase-07-mock-control-plane-port.md) — the + deterministic in-memory adapter, its entity domains, and its boundary. +- [Semantic tool catalog](../phase-07-semantic-tools.md) — the + transport-neutral tools an agent may call. +- [Authorization and exposure](../phase-07-authorization-and-exposure.md) — + grants, typed denials, and secret redaction. +- [Eval conformance](../phase-07-eval-conformance.md) — how the 106 cases are + derived from the checked-in traceability derivative with no external repo. + +## 4. Run the full focused verification set + +Every command below is offline, deterministic, and package-local. The +[verification commands reference](../phase-07-verification-commands.md) lists +each one with its purpose and expected result. + +```sh +# Capability contract: completeness, uniqueness, one-to-one MCP folds, no drift. +pnpm --filter @paperclipai/paperclip-runner check:phase7-inventory +pnpm --filter @paperclipai/paperclip-runner test:phase7-inventory + +# Mock control plane adapter and shared ControlPlanePort conformance. +pnpm --filter @paperclipai/paperclip-runner exec vitest run \ + src/conformance/control-plane-port.test.ts \ + src/mock-core/phase7-mock-control-plane-adapter.test.ts + +# Semantic tool catalog and authorization/redaction engine. +pnpm --filter @paperclipai/paperclip-runner exec vitest run \ + src/tools/phase7-semantic-tools.test.ts + +# 106-case conformance and the parity/fake-agent/bounded-Codex report. +pnpm --filter @paperclipai/paperclip-runner test:phase7-evals +pnpm --filter @paperclipai/paperclip-runner report:phase7-evals + +# Scenario runtime, explorer components, and route determinism (45 tests). +pnpm --filter @paperclipai/paperclip-runner test:phase7 + +# Browser information architecture, accessibility, determinism, and boundary. +pnpm --filter @paperclipai/paperclip-runner test:browser:phase7 + +# Deterministic 24-image screenshot acceptance set. +pnpm --filter @paperclipai/paperclip-runner record:phase7 + +# Documentation links and the OKF knowledge bundle (run with no generated +# report present). +pnpm --filter @paperclipai/paperclip-runner docs:validate +``` + +The exact results captured for the acceptance record are in the +[Phase 7 verification evidence](../../knowledge/evidence/2026-08-09-phase-07-verification.md). + +## Known gaps and boundaries + +- **The parity report breaks the OKF bundle if left in place.** + `report:phase7-evals` writes `eval-parity-report.md` into the knowledge + bundle without OKF frontmatter, so `docs:validate` fails while it is present. + Delete it before validating. A clean checkout never contains it. This is + recorded for the Phase 7H QA gate. +- **Codex mode is disabled.** The explorer offers a bounded real-Codex binding + sample only through `report:phase7-evals`; the browser holds no credential + and the in-page Codex mode is inert with a stated reason. +- **No Paperclip integration.** Nothing here touches a real control plane, + database, or provider. Phase 8 (ACPX) binds the real adapter behind the same + seam and requires separate CTO approval. + +## Where this sits + +- Plan: Phase 7 plan, issue `PAP-16897` (see its `#document-plan` document). +- Child issues: 7A capability inventory (PAP-16898), 7B UX map (PAP-16899), + 7C mock adapter (PAP-16900), 7D semantic tools and authorization (PAP-16901, + security gate PAP-16902, remediation PAP-16909), 7E conformance suite + (PAP-16903), 7F browser explorer (PAP-16904, UX gate PAP-16905), 7G docs and + evidence (PAP-16906), 7H clean-room QA (PAP-16907), Phase 7 checkpoint + (PAP-16908). + + diff --git a/packages/paperclip-runner/knowledge/evidence/2026-08-09-phase-07-verification.md b/packages/paperclip-runner/knowledge/evidence/2026-08-09-phase-07-verification.md new file mode 100644 index 00000000000..fbc0cbaef1e --- /dev/null +++ b/packages/paperclip-runner/knowledge/evidence/2026-08-09-phase-07-verification.md @@ -0,0 +1,105 @@ +--- +type: Verification Evidence +title: Phase 7 documentation, tutorial, and evidence manifest +description: Package-local acceptance manifest cross-linking every focused Phase 7 verification command, its observed result, the parity/screenshot artifacts, and the security, UX, and QA gate issues. +tags: [native-runner, phase-7, verification, conformance, evidence, manifest] +status: stable +generated: { by: anthropic/claude-opus-4-8, at: 2026-08-09T13:42:50Z } +--- + +# Scope + +This record is the Phase 7G evidence manifest (PAP-16906). It links each +package-local, offline, deterministic verification command to the result +observed on 2026-08-09, and to the reference pages, tutorial, parity report, +and screenshot set that describe the Phase 7 surfaces. + +Phase 7 contacts no Paperclip service, ACPX session, database, or provider +credential. Every command below runs from the repository root against +checked-in fixtures and the in-process mock control plane. + +# Environment + +- Node.js 22.22.2, pnpm 9.15.4. No Rust toolchain is required for any Phase 7 + focused command. +- vitest 4.1.10; Playwright with its bundled Chromium. +- No network access after `pnpm install`. + +# Verification matrix + +All commands are prefixed `pnpm --filter @paperclipai/paperclip-runner`. +Results were captured on 2026-08-09 between 13:37Z and 13:39Z. + +| Surface | Command | Observed result | +| --- | --- | --- | +| Capability contract completeness and drift | `check:phase7-inventory` | `Phase 7 inventory completeness and generated-output checks passed.` (exit 0) | +| Contract validator negatives | `test:phase7-inventory` | 4 tests, 4 pass, 0 fail | +| Mock control plane + shared port | `exec vitest run src/conformance/control-plane-port.test.ts src/mock-core/phase7-mock-control-plane-adapter.test.ts` | 2 files, 9 tests pass | +| Semantic tools + authorization/redaction | `exec vitest run src/tools/phase7-semantic-tools.test.ts` | 1 file, 9 tests pass | +| 106-case conformance | `test:phase7-evals` | 1 file, 1 test (drives all 106 cases) pass | +| Fake-agent matrix + bounded Codex sample + parity report | `report:phase7-evals` | `Phase 7 eval conformance passed: 106 cases across 16 groups.` | +| Scenario runtime + explorer components + routes | `test:phase7` | 3 files, 45 tests pass | +| Browser IA, accessibility, determinism, boundary | `test:browser:phase7` | 19 Playwright tests pass (6.8s) | +| Screenshot acceptance set | `record:phase7` | 24 deterministic images (12 routes × 2 viewports) | +| Doc links + OKF bundle | `docs:validate` | `Documentation link validation passed (62 files).` and `OKF v0.2 validation passed (26 concepts, 5 indexes).` | + +# Conformance shape + +- 106 cases across 16 groups: hb 5, co 6, st 8, cm 6, se 4, su 4, bl 5, dp 3, + ix 9, ap 6, ar 4, er 9, rf 22, mh 4, rs 3, wk 8. +- Assertion classes: `agent_tool_contract`, `authorization_policy`, + `combined_multi_hop`, `control_plane_invariant`, `restraint_no_call`. +- Fake-agent/Codex operations: 18/18 executed. +- Capability contract baseline: 152 skill/reference headings, 106 eval cases, + 258 normative rows, 41 legacy MCP aliases folded one-to-one into normative + rows. See [the generated contract](../../docs/phase-07-capability-contract.md). + +# Artifacts + +- Parity report: generated on demand by `report:phase7-evals` into + `knowledge/evidence/phase-07/eval-parity-report.{json,md}`. It is not a + committed file; a clean checkout does not contain it, and the explorer renders + "Not run" when it is absent. +- Screenshots: the committed 24-image acceptance set and its manifest are under + [Phase 7 evidence](phase-07/index.md). +- Prior surface evidence: + [Phase 7F browser scenario explorer verification](2026-08-09-phase-07f-explorer-verification.md). + +# Documentation delivered + +- Tutorial: [Phase 7 clean-start tutorial](../../docs/tutorials/phase-07-scenario-explorer.md). +- Reference pages: + [capability disposition](../../docs/phase-07-capability-disposition.md), + [mock ControlPlanePort](../../docs/phase-07-mock-control-plane-port.md), + [semantic tools](../../docs/phase-07-semantic-tools.md), + [authorization and exposure](../../docs/phase-07-authorization-and-exposure.md), + [eval conformance](../../docs/phase-07-eval-conformance.md), + [scenario explorer](../../docs/phase-07-scenario-explorer.md), + [future binding boundary](../../docs/phase-07-future-binding-boundary.md), + [verification commands](../../docs/phase-07-verification-commands.md). + +# Gate issues + +- Security gate: `PAP-16902` (authorization and credential isolation review), + with remediation `PAP-16909` (redact secret invocation results before trace + or browser exposure). The `rs-secret-redaction` case and the + `src/tools/phase7-semantic-tools.test.ts` redaction assertions exercise the + remediated path. +- UX gate: `PAP-16905` (explorer acceptance and screenshots), against the + approved interaction map §9 twelve-route matrix. +- QA gate: `PAP-16907` (clean-room QA and human checkpoint), which consumes this + manifest and the tutorial from a clean checkout. + +# Known gaps + +- `report:phase7-evals` writes `eval-parity-report.md` into the knowledge bundle + without OKF frontmatter, and `evidence/phase-07/index.md` does not link it, so + `docs:validate` fails while the generated report is present. Delete the report + before validating, or run validation first. Recorded for the Phase 7H QA gate + as a candidate to relocate the generated report outside the bundle or exclude + it from the OKF scan. +- Codex mode in the explorer is disabled with a stated reason; the only real + provider exercise is the bounded binding sample in `report:phase7-evals`. +- No Paperclip control plane, database, or provider is contacted anywhere in + Phase 7. Real integration is Phase 8 (ACPX) and requires separate approval. + diff --git a/packages/paperclip-runner/knowledge/evidence/index.md b/packages/paperclip-runner/knowledge/evidence/index.md index 2e427681a9b..11cef188db5 100644 --- a/packages/paperclip-runner/knowledge/evidence/index.md +++ b/packages/paperclip-runner/knowledge/evidence/index.md @@ -14,4 +14,5 @@ * [Phase 5 browser SDK verification](2026-08-08-phase-05-verification.md) - Targeted and package-acceptance evidence for the versioned SDK, both public consumers, fake/real drivers, accessibility, reconnect, replay, and screenshots. * [Phase 6 thin Paperclip adapter verification](2026-08-09-phase-06-verification.md) - Mock and database-backed conformance, authoritative completion, restart recovery, migration repair, atomic liveness, internal canary, and deterministic post-kill-switch legacy evidence. * [Phase 7F browser scenario explorer verification](2026-08-09-phase-07f-explorer-verification.md) - Scenario index, run artifacts, exposure, authorization, state diff, parity, route determinism, accessibility, and the 24-image screenshot acceptance set. +* [Phase 7 documentation, tutorial, and evidence manifest](2026-08-09-phase-07-verification.md) - Command→result matrix for every focused Phase 7 verification, artifact links, and the security, UX, and QA gate issues. * [Phase 7 evidence](phase-07/) - Screenshot acceptance captures and the eval conformance parity report. diff --git a/packages/paperclip-runner/knowledge/journal/2026-08-09-phase-07.md b/packages/paperclip-runner/knowledge/journal/2026-08-09-phase-07.md new file mode 100644 index 00000000000..43ba3b4dafd --- /dev/null +++ b/packages/paperclip-runner/knowledge/journal/2026-08-09-phase-07.md @@ -0,0 +1,102 @@ +--- +type: Engineering Journal Entry +title: Phase 7 reference docs, tutorial, evidence, and OKF +description: Decisions, observed facts, gaps, and readiness evidence for the package-local Phase 7 documentation set and clean-start tutorial. +tags: [native-runner, phase-7, documentation, tutorial, evidence, okf] +status: stable +generated: { by: anthropic/claude-opus-4-8, at: 2026-08-09T13:42:50Z } +entry_kind: phase +phase: "7" +--- + +# Context + +Phase 7 (PAP-16897) built a package-local model of a native Paperclip run: +a generated capability contract (7A), a UX interaction map (7B), a deterministic +mock `ControlPlanePort` (7C), a transport-neutral semantic tool catalog and +authorization engine (7D, security gate 7D/PAP-16902, remediation PAP-16909), +an eval-derived 106-case conformance suite (7E), and a read-only browser +scenario explorer (7F, UX gate PAP-16905). Phase 7G (PAP-16906) produces the +reference documentation, the clean-start tutorial, the evidence manifest, and +this OKF entry. It was blocked until 7E and 7F were done because the tutorial +and evidence must describe and exercise the final conformance and browser +surfaces. + +# Decisions + +1. Keep the reference set flat under `docs/phase-07-*.md`, matching the existing + per-phase reference convention, rather than introducing a new folder. Eight + reference pages: capability disposition, mock control plane, semantic tools, + authorization/exposure, eval conformance, scenario explorer, future binding + boundary, and verification commands. +2. Treat the hand-written capability-disposition page as a reader's guide to the + generated `docs/phase-07-capability-contract.md`; never restate the generated + table, which carries a "DO NOT EDIT" header and is rewritten by + `generate:phase7-inventory`. +3. Write the tutorial to run from the repository root with two commands to first + success, no Rust toolchain, no Paperclip service, and no external eval + repository. State time-to-first-success and exact expected output. +4. Make the verification evidence file the single manifest: a command→result + matrix, artifact links, and the security/UX/QA gate issues. Capture real + observed output rather than asserting expected output. +5. Do not commit the generated parity report. It is an on-demand artifact; a + clean checkout does not contain it, and the tracked + `evidence/phase-07/index.md` already documents it as generated. +6. Update `docs/index.md`, `docs/architecture.md`, `README.md`, and the two + knowledge indexes additively. Keep ACPX as Phase 8 and state that real + Paperclip integration requires separate approval. + +# Observed facts + +All commands ran from the repository root, offline, on Node 22.22.2 / pnpm +9.15.4, between 13:37Z and 13:39Z on 2026-08-09. Full matrix in +[the verification evidence](../evidence/2026-08-09-phase-07-verification.md). + +- `check:phase7-inventory` and `test:phase7-inventory` pass; the contract holds + 152 skill/reference headings, 106 eval cases, 258 normative rows, and 41 + legacy MCP aliases folded one-to-one. +- Mock control plane (9 tests), semantic tools and authorization/redaction + (9 tests), scenario runtime and explorer (45 tests), and the 106-case + conformance suite all pass in-process. +- `report:phase7-evals` reports 106 cases across 16 groups, 18/18 fake-agent + operations, and the bounded Codex binding sample. +- `test:browser:phase7` passes 19 Playwright checks (6.8s) for information + architecture, route determinism, evidence routes, the credential boundary, + accessibility, and responsive layout. +- On a clean tree `docs:validate` passes 62 documentation files and an OKF + bundle of 26 concepts across 5 indexes. + +# Evidence + +- [Phase 7 verification manifest](../evidence/2026-08-09-phase-07-verification.md) +- [Phase 7F explorer verification](../evidence/2026-08-09-phase-07f-explorer-verification.md) +- [Clean-start tutorial](../../docs/tutorials/phase-07-scenario-explorer.md) +- [Verification commands reference](../../docs/phase-07-verification-commands.md) + +# Failures + +- With the generated parity report present, `validate-okf` fails: + `eval-parity-report.md` has no OKF frontmatter and is not linked from + `evidence/phase-07/index.md`. Deleting the on-demand report restored a clean + pass. This is a real dirty-tree gap, not a documentation error; it is recorded + below and handed to the QA gate. + +# Known gaps + +- The generated parity report is incompatible with the OKF bundle scan when left + in the knowledge tree. Candidate fixes for Phase 7H (PAP-16907): relocate the + generated report outside `knowledge/`, add OKF frontmatter plus an index link + during generation, or exclude generated reports from the OKF scan. +- Codex mode is disabled in the explorer; the only real provider exercise is the + bounded binding sample in `report:phase7-evals`. +- Phase 7 adds no real Paperclip integration. Phase 8 (ACPX) binds the real + adapter behind the preserved seam and requires separate CTO approval + (PAP-16908). + +# Follow-up questions + +- Should the generated parity report move under `knowledge/evidence/phase-07/` + with generator-authored OKF frontmatter, or out of the bundle entirely? +- Which of the eight reference pages should the Phase 8 binding work extend + first when the real adapter lands behind the seam? + diff --git a/packages/paperclip-runner/knowledge/journal/index.md b/packages/paperclip-runner/knowledge/journal/index.md index 7d4133ba5ce..bd0abc52457 100644 --- a/packages/paperclip-runner/knowledge/journal/index.md +++ b/packages/paperclip-runner/knowledge/journal/index.md @@ -16,3 +16,4 @@ * [Phase 6 thin Paperclip adapter design](2026-08-09-phase-06-design.md) - Proposed public-port seam, feature flag, kill switch, company/auth/governance boundary, durable replay, conformance matrix, and tutorial contract. * [Phase 6 feature-flagged Paperclip adapter implementation](2026-08-09-phase-06-implementation.md) - Persisted selection, production port, public-session task, canonical event allocation, server-owned finalization, cancellation, and kill-switch evidence. * [Phase 6 thin Paperclip adapter implementation](2026-08-09-phase-06-implementation.md) - Default-off native selection, package-owned sessions, canonical persistence/replay, server-owned finalization, recovery, and compatibility evidence. +* [Phase 7 reference docs, tutorial, evidence, and OKF](2026-08-09-phase-07.md) - Documentation decisions, observed verification facts across every focused command, the parity-report OKF gap, and the deferred Phase 8 binding boundary.