mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-26 18:31:34 +01:00
docs(runner): add Phase 7G reference docs, tutorial, evidence, and OKF
Phase 7G (PAP-16906): package-local reference pages for capability disposition, mock ControlPlanePort, semantic tools, authorization and exposure, eval-derived conformance, browser explorer, the future Phase 8 binding boundary, and the focused verification commands; a clean-start tutorial that runs from the repo root with no Paperclip service or external eval repo; an OKF evidence manifest cross-linking every focused command to its observed result and the security/UX/QA gate issues (PAP-16902, PAP-16905, PAP-16907); and the Phase 7 OKF journal entry. Additive updates to the docs index, architecture boundary, and knowledge indexes. Keeps ACPX as Phase 8. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
144817ea2c
commit
59b72ca1d2
15 files changed
+1069
No files matched your search
@@ -196,6 +196,38 @@ lease expiry, storage pressure, drain, and revoke. The browser recovery page
|
||||
calls that peer through guarded package-local Vite middleware and renders only
|
||||
safe trace fields.
|
||||
|
||||
## Phase 7 capability-model boundary
|
||||
|
||||
```text
|
||||
Paperclip skill + 7 references Paperclip Evals corpus (106 cases)
|
||||
| |
|
||||
+-------------------+------------------+
|
||||
v
|
||||
generated capability contract (258 rows, 41 MCP aliases)
|
||||
|
|
||||
+-----------------------+-----------------------+
|
||||
v v v
|
||||
semantic tool catalog authorization engine eval conformance suite
|
||||
\ | /
|
||||
\ v /
|
||||
+-----> in-process mock ControlPlanePort <--+
|
||||
|
|
||||
v
|
||||
read-only browser scenario explorer
|
||||
```
|
||||
|
||||
Phase 7 is a package-local model of a native Paperclip run. It classifies every
|
||||
capability as control-plane-owned, always-agent-tool, or optional-agent-tool,
|
||||
exposes the always/optional set as a transport-neutral semantic tool catalog,
|
||||
gates optional tools behind grants, and proves 106 eval-derived cases against an
|
||||
in-process mock `ControlPlanePort`. The mock adapter is the only coupling point,
|
||||
so a real adapter can replace it later without touching the catalog,
|
||||
authorization rules, or conformance suite. Phase 7 contacts no Paperclip
|
||||
service, database, ACPX session, or provider credential; the
|
||||
[forbidden-imports checker](#forbidden-dependencies) keeps it that way. Real
|
||||
integration is Phase 8 (ACPX) and requires separate approval; see
|
||||
[the future binding boundary](phase-07-future-binding-boundary.md).
|
||||
|
||||
## Future integration rule
|
||||
|
||||
The [implementation plan](../spec/paperclip-native-runner-implementation-plan.md)
|
||||
|
||||
@@ -14,6 +14,7 @@ Each tutorial starts with an explanation of what the phase is and what the phase
|
||||
- [Phase 4b: run the live Codex protocol console](tutorials/phase-04b-live-console.md) — chat with a live session in the browser, steer it, stop it three ways, answer its requests, change its goal, break its connection, and replay the record.
|
||||
- [Phase 5: run the SDK console and mini consumer](tutorials/phase-05-sdk-console.md) — exercise the versioned public browser/React surface in two independent consumers against fake and real drivers.
|
||||
- [Phase 6: run the thin Paperclip adapter](tutorials/phase-06-thin-paperclip-adapter.md) — prove mock/real conformance, run one feature-flagged local task, inspect replay/finalization, disable the kill switch, and verify legacy fallback.
|
||||
- [Phase 7: explore the capability contract and scenario explorer](tutorials/phase-07-scenario-explorer.md) — prove the 106-case conformance suite, open the read-only browser explorer over the mock control plane, and run the focused verification set, all from a clean checkout with no Paperclip service or external eval repository.
|
||||
- [Cumulative end-to-end tutorial](tutorials/end-to-end.md) — the shortest complete workflow available at the current phase.
|
||||
|
||||
## Reference
|
||||
@@ -27,6 +28,16 @@ Each tutorial starts with an explanation of what the phase is and what the phase
|
||||
- [Phase 4b live console](phase-04b-live-console.md)
|
||||
- [Phase 5 browser SDK and reference console](phase-05-sdk.md)
|
||||
- [Phase 6 thin Paperclip adapter](phase-06-thin-paperclip-adapter.md)
|
||||
- [Phase 7 capability contract (generated)](phase-07-capability-contract.md)
|
||||
- [Phase 7 capability disposition](phase-07-capability-disposition.md)
|
||||
- [Phase 7 mock ControlPlanePort](phase-07-mock-control-plane-port.md)
|
||||
- [Phase 7 semantic tool catalog](phase-07-semantic-tools.md)
|
||||
- [Phase 7 authorization and exposure](phase-07-authorization-and-exposure.md)
|
||||
- [Phase 7 eval-derived conformance](phase-07-eval-conformance.md)
|
||||
- [Phase 7 browser scenario explorer](phase-07-scenario-explorer.md)
|
||||
- [Phase 7 future binding boundary (Phase 8 / ACPX)](phase-07-future-binding-boundary.md)
|
||||
- [Phase 7 verification commands](phase-07-verification-commands.md)
|
||||
- [Phase 7 scenario explorer UX interaction map](design/phase-7-scenario-explorer-ux.md)
|
||||
- [Engineering journal guide](journal.md)
|
||||
- [Dated shadcn/ui and AI Elements compatibility note](research/2026-08-07-ui-library-compatibility.md)
|
||||
- [Phase 4b live-console interaction map](design/phase-4b-interaction-map.md)
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Phase 7 Authorization and Exposure
|
||||
|
||||
Two decisions gate every capability: **exposure** (is the tool even offered to
|
||||
this actor?) and **invocation** (may this specific call proceed?). The
|
||||
authorization engine answers both, produces typed denials that carry no
|
||||
protected state, and redacts secrets before any observable boundary.
|
||||
|
||||
Sources: `src/tools/phase7-tool-authorization.ts`,
|
||||
`src/tools/phase7-semantic-tool-runtime.ts`, `src/tools/phase7-tool-bindings.ts`.
|
||||
|
||||
## Three-way classification
|
||||
|
||||
- **Control-plane-owned** — a frozen operation list (`checkout_task`,
|
||||
`release_task`, `select_work`, `route_wake`, `enforce_budget`,
|
||||
`append_audit_record`, `persist_run`, `replay_run`, `schedule_blocker_wake`,
|
||||
`reconcile_run`). `authorizeInvocation` short-circuits these to outcome
|
||||
`absent` with reason `control_plane_owned_operation`. No agent tool exists for
|
||||
them and none can be invoked.
|
||||
- **Always-agent tool** — exposed to any actor whose task is active.
|
||||
- **Optional-agent tool** — exposed only when a grant unlocks it.
|
||||
|
||||
## Exposure vs. invocation
|
||||
|
||||
- `computeVisibleTools()` walks the whole catalog and records, per tool, an
|
||||
exposure of `exposed` or `absent`, returning only the allowed tools.
|
||||
- `authorizeInvocation()` re-evaluates at call time, producing `allowed` or
|
||||
`denied`.
|
||||
|
||||
Invocation is evaluated in order and fails at the first rule that rejects:
|
||||
policy-denied operation IDs; task-mode membership
|
||||
(`operation_not_available_in_task_mode`); role check
|
||||
(`actor_role_not_authorized`); policy-denied claims
|
||||
(`required_claim_denied_by_policy`); missing grants
|
||||
(`required_claim_missing`); `read_secret_value` requires
|
||||
`policy.allowSecretValueAccess` (`secret_value_access_disabled`);
|
||||
`generic_api_request` requires the escape hatch to be enabled and the target to
|
||||
be allowlisted (`generic_escape_hatch_disabled` / `..._target_denied`); and the
|
||||
interaction-kind policy for `request_human_input`. Allowed calls carry the
|
||||
reason `task_scoped_always_tool` or `required_claims_granted`.
|
||||
|
||||
Self-approval is prohibited even with the role and an explicit decision claim.
|
||||
|
||||
## How grants unlock optional tools
|
||||
|
||||
The actor's grants are the union of `capabilityGrants` (the actor's own) and
|
||||
`scenarioGrants` (seeded per scenario). An optional tool unlocks only when
|
||||
**every** entry in its `requiredClaims` is present in that union. Each
|
||||
authorization record stores the `consideredClaims`, `grantedClaims`, and
|
||||
`missingClaims`, so a denial explains exactly which claim was absent.
|
||||
|
||||
## Typed denials carry no protected state
|
||||
|
||||
A denial is a `Phase7PolicyDenial`: `ok: false` with an `error.code` of
|
||||
`policy_denied`, `operation_absent`, `input_invalid`, or `operation_unsupported`,
|
||||
the `operationId`, a generic `reason`, and the authorization record. It carries
|
||||
no fixture state and no protected payload. If the underlying mock operation
|
||||
throws, the runtime emits `operation_unsupported` with reason
|
||||
`mock_operation_rejected` and swallows the underlying error, so protected state
|
||||
cannot leak through an exception.
|
||||
|
||||
## Secret redaction (the PAP-16909 rule)
|
||||
|
||||
Redaction is two layers:
|
||||
|
||||
1. **Model-only delivery is separated from every observable boundary.**
|
||||
`invoke()` returns a redacted `observableResult`. The only path to an
|
||||
unredacted secret is `invokeForModel()`, which returns a frozen capsule whose
|
||||
`toJSON()` and default serialization always yield the redacted form; only an
|
||||
explicit `readModelResult()` opens the raw value. The capsule uses a distinct
|
||||
schema (`paperclip.phase7.model-tool-result.v1`) so it cannot be assigned to
|
||||
an observable sink by mistake.
|
||||
2. **Path redaction rules.** `read_secret_value` redacts `$.value` to
|
||||
`[SECRET_VALUE]` across output, error, and authorization-record channels;
|
||||
`generic_api_request` redacts `$.headers.authorization` and
|
||||
`$.headers.cookie` to `[REDACTED]` across all channels. The scenario runner
|
||||
applies the same redaction again defensively at the explorer artifact
|
||||
boundary and injects only a placeholder secret value.
|
||||
|
||||
This is the security-gate outcome (PAP-16902) and its remediation (PAP-16909):
|
||||
a real secret value reaches the model surface only, never a trace, artifact, or
|
||||
browser view.
|
||||
|
||||
## Running the tests
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
|
||||
src/tools/phase7-semantic-tools.test.ts
|
||||
```
|
||||
|
||||
The "exposure and authorization" and "security policy" describes cover typed
|
||||
denials without protected state, model-only secret delivery, the self-approval
|
||||
prohibition, and the test-only, explicitly granted, allowlisted escape hatch.
|
||||
|
||||
## Related
|
||||
|
||||
- [Capability disposition](phase-07-capability-disposition.md)
|
||||
- [Semantic tool catalog](phase-07-semantic-tools.md)
|
||||
- [Scenario explorer](phase-07-scenario-explorer.md)
|
||||
</content>
|
||||
@@ -0,0 +1,85 @@
|
||||
# Phase 7 Capability Disposition
|
||||
|
||||
This page explains how Phase 7 classifies every Paperclip capability. The
|
||||
classification itself lives in a generated file,
|
||||
[the Phase 7A capability contract](phase-07-capability-contract.md), which
|
||||
carries a "DO NOT EDIT" header and is rewritten by `generate:phase7-inventory`.
|
||||
Read this page to understand what the generated table means; read the table for
|
||||
the authoritative rows.
|
||||
|
||||
## Normative sources
|
||||
|
||||
Only two sources are normative:
|
||||
|
||||
1. The Paperclip skill and its seven references (`SKILL.md` plus
|
||||
`references/*.md`), contributing **152 headings**.
|
||||
2. The Paperclip Evals corpus, contributing **106 cases across 16 groups**.
|
||||
|
||||
Together these produce **258 normative rows**. The legacy Paperclip MCP tool
|
||||
surface (**41 tools**) is not a production capability surface; each MCP name is
|
||||
folded one-to-one into a normative eval row as a traceability alias and inherits
|
||||
that row's disposition. The contract prints the alias index only so the
|
||||
normative target is easy to audit.
|
||||
|
||||
## The three dispositions
|
||||
|
||||
Every capability is classified as exactly one of:
|
||||
|
||||
- **`control_plane_owned`** — the control plane performs this; no agent tool
|
||||
exists for it. Checkout, inbox resolution, blocker diagnostics, and status
|
||||
arbitration are control-plane-owned. In the explorer these appear as
|
||||
control-plane actions labelled "no agent tool exists for this," never as a
|
||||
callable tool.
|
||||
- **`always_agent_tool`** — the agent always has this as a semantic tool. Adding
|
||||
a comment, writing a document, opening an interaction, and registering a
|
||||
deliverable are always-agent-tool capabilities.
|
||||
- **`optional_agent_tool`** — the agent may have this tool, but only when a
|
||||
grant unlocks it. Absent the grant, the capability is not exposed and calling
|
||||
it is denied. See [authorization and exposure](phase-07-authorization-and-exposure.md).
|
||||
|
||||
The disposition is what makes the boundary legible: it states, per capability,
|
||||
whether an agent can act, must wait for the control plane, or needs a grant
|
||||
first.
|
||||
|
||||
## Reading the generated contract
|
||||
|
||||
The contract has three parts:
|
||||
|
||||
- **Baseline counts** — the heading, case, row, and alias totals above, plus the
|
||||
per-group case table.
|
||||
- **Skill / reference rows** — one row per heading, its primary disposition, and
|
||||
its `file:line` source anchor.
|
||||
- **Legacy MCP alias index** — each MCP name, the normative row it folds into,
|
||||
the inherited disposition, and its `packages/mcp-server/src/tools.ts` anchor.
|
||||
|
||||
## Regenerating and checking
|
||||
|
||||
Generation reads the live in-repo skill/reference sources, the legacy MCP tool
|
||||
source, and the Paperclip Evals corpus, so it **requires** the external eval
|
||||
repository (via `PAPERCLIP_EVALS_ROOT` or a known local path) and is not part of
|
||||
the offline path. Checking and testing read only the checked-in derivatives
|
||||
under `spec/phase-07/` and need no external repository.
|
||||
|
||||
```sh
|
||||
# Rewrite every generated file. Requires the external Paperclip Evals corpus.
|
||||
pnpm --filter @paperclipai/paperclip-runner generate:phase7-inventory
|
||||
|
||||
# Validate counts, uniqueness, normative dispositions, one-to-one MCP folds,
|
||||
# required fields, and generated-file drift. Offline; no external eval repo.
|
||||
pnpm --filter @paperclipai/paperclip-runner check:phase7-inventory
|
||||
|
||||
# Prove the validator rejects an independent MCP classification and rejects
|
||||
# missing, duplicate, or unknown MCP folds. Offline.
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7-inventory
|
||||
```
|
||||
|
||||
`check:phase7-inventory` diffs the checked-in generated files against what the
|
||||
live in-repo sources imply and fails on any drift or stale anchor, so the
|
||||
contract cannot silently fall out of sync.
|
||||
|
||||
## Related
|
||||
|
||||
- [Semantic tool catalog](phase-07-semantic-tools.md)
|
||||
- [Authorization and exposure](phase-07-authorization-and-exposure.md)
|
||||
- [Eval conformance](phase-07-eval-conformance.md)
|
||||
</content>
|
||||
@@ -0,0 +1,83 @@
|
||||
# Phase 7 Eval-Derived Conformance
|
||||
|
||||
Phase 7E turns the Paperclip Evals corpus into an executable, offline
|
||||
conformance suite. The suite derives its cases from the checked-in Phase 7
|
||||
traceability derivative (`spec/phase-07/eval-traceability.yaml`) — it does
|
||||
**not** clone or read an external eval repository, and it starts only the
|
||||
in-process mock control plane. It hard-fails unless the derivative declares
|
||||
schema version 2, exactly 106 rows, exactly 16 groups, and unique case IDs.
|
||||
|
||||
Sources: `src/conformance/phase7-eval-suite.ts` and its test
|
||||
`src/conformance/phase7-eval-suite.test.ts`; the reporter
|
||||
`scripts/run-phase7-eval-suite.mjs`.
|
||||
|
||||
## The corpus
|
||||
|
||||
- **106 cases across 16 groups.** Per-group counts: hb 5, co 6, st 8, cm 6,
|
||||
se 4, su 4, bl 5, dp 3, ix 9, ap 6, ar 4, er 9, rf 22, mh 4, rs 3, wk 8.
|
||||
- Each case declares an actor role, a task mode, an input scenario, and the
|
||||
expected outcome, all bound to a row in
|
||||
[the capability contract](phase-07-capability-contract.md).
|
||||
|
||||
## Assertion classes
|
||||
|
||||
Every case belongs to one assertion class, and each class checks a different
|
||||
kind of invariant:
|
||||
|
||||
- **`agent_tool_contract`** — a semantic tool call produces the expected typed
|
||||
effect and state change.
|
||||
- **`authorization_policy`** — a capability is exposed, denied, or unlocked by a
|
||||
grant exactly as its disposition requires.
|
||||
- **`control_plane_invariant`** — a control-plane-owned action happens without
|
||||
any agent tool, and no tool can perform it.
|
||||
- **`combined_multi_hop`** — a sequence of operations across turns produces the
|
||||
expected cumulative state and respects forbidden-operation rules.
|
||||
- **`restraint_no_call`** — the correct behavior is to make **no** call; the
|
||||
case passes only if the agent deliberately does nothing further.
|
||||
|
||||
## Fake-agent matrix and bounded Codex sample
|
||||
|
||||
The suite executes each case with a deterministic fake agent whose plan is
|
||||
fixed by a fixture seed, so repeat runs are byte-identical. Optional-tool rows
|
||||
are run twice — once with the unlocking grants (must be allowed and must mutate
|
||||
state) and once ungranted (must be absent or denied with no state change);
|
||||
control-plane-owned rows remain absent in every configuration; and
|
||||
`restraint_no_call` rows must produce an empty state diff.
|
||||
|
||||
The fake-agent surface is 14 always-agent tools plus 4 optional tools unlocked
|
||||
by four seed grants (`discovery:tasks:read`, `discovery:agents:read`,
|
||||
`delegation:tasks:create`, `governance:approvals:request`) — **18 operations**.
|
||||
The suite binds that surface through both the fake-agent and Codex bindings and
|
||||
asserts the two operation lists are byte-identical (**18/18**).
|
||||
|
||||
A **bounded Codex binding sample** picks one representative case from nine
|
||||
groups (`hb`, `dp`, `bl`, `ap`, `ar`, `ix`, `mh`, `rs`, `wk`) and checks that
|
||||
`checkout_task` is absent from the Codex surface while every other sampled
|
||||
operation is present. It is an offline parity check; no real Codex or network is
|
||||
contacted, and the browser explorer holds no credential.
|
||||
|
||||
## Running it
|
||||
|
||||
```sh
|
||||
# Run the 106-case suite in-process (one vitest file drives all cases).
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7-evals
|
||||
|
||||
# Build the public surface and write the parity report with per-group counts,
|
||||
# assertion classes, the fake-agent matrix, the bounded Codex sample, and the
|
||||
# semantic-operation execution counts.
|
||||
pnpm --filter @paperclipai/paperclip-runner report:phase7-evals
|
||||
```
|
||||
|
||||
The reporter writes `knowledge/evidence/phase-07/eval-parity-report.{json,md}`.
|
||||
Each failure carries its case ID, assertion class, semantic operation,
|
||||
authorization decision, and final state diff. The report is generated on demand
|
||||
and is not committed; delete it before running `docs:validate` (it carries no
|
||||
OKF frontmatter). See the
|
||||
[verification commands reference](phase-07-verification-commands.md).
|
||||
|
||||
## Related
|
||||
|
||||
- [Capability disposition](phase-07-capability-disposition.md)
|
||||
- [Semantic tool catalog](phase-07-semantic-tools.md)
|
||||
- [Scenario explorer](phase-07-scenario-explorer.md)
|
||||
</content>
|
||||
@@ -0,0 +1,52 @@
|
||||
# Phase 7 Future Binding Boundary (Phase 8 / ACPX)
|
||||
|
||||
Phase 7 is a package-local model. It does not integrate the runner into
|
||||
Paperclip, and it must not. This page states exactly what Phase 7 defers to
|
||||
Phase 8 and what clean seam it preserves so the deferral stays cheap.
|
||||
|
||||
## What Phase 7 does not do
|
||||
|
||||
Nothing in Phase 7 contacts a real Paperclip control plane, database, ACPX
|
||||
session, or provider credential. The 106-case conformance suite and the browser
|
||||
explorer run entirely against the in-process
|
||||
[mock ControlPlanePort](phase-07-mock-control-plane-port.md) and checked-in
|
||||
fixtures. The [forbidden-imports boundary](architecture.md) rejects any import
|
||||
that crosses into `server/`, `ui/`, `cli/`, `@paperclipai/db`, or other
|
||||
Paperclip workspace internals.
|
||||
|
||||
## The preserved seam
|
||||
|
||||
The dependency arrow always points from an implementation toward a contract.
|
||||
The package owns two contracts a future consumer implements:
|
||||
|
||||
- **`ControlPlanePort`** — the narrow surface through which a runner opens a
|
||||
run, appends ordered events, records semantic operations, and submits a
|
||||
terminal result. Phase 7 injects the *mock* adapter behind this port; Phase 8
|
||||
injects a real one. The catalog, authorization engine, and conformance suite
|
||||
bind to the port, not to any adapter.
|
||||
- **`NativeSessionBackend`** — the normalized session surface for a future
|
||||
control-plane consumer.
|
||||
|
||||
Because the port is the only coupling point, Phase 8 replaces the adapter
|
||||
without touching the tool catalog, the authorization rules, or the eval-derived
|
||||
conformance suite. The package remains independently buildable, testable, and
|
||||
runnable against the mock adapter after the real one exists.
|
||||
|
||||
## What Phase 8 (ACPX) will bind
|
||||
|
||||
Phase 8 binds a real Paperclip `ControlPlanePort` implementation behind the same
|
||||
seam so the semantic tools and authorization engine act against a live control
|
||||
plane instead of the mock. That work is out of scope here and requires separate
|
||||
CTO approval at the Phase 7 checkpoint (`PAP-16908`). Until then:
|
||||
|
||||
- ACPX is Phase 8, not Phase 7.
|
||||
- No Phase 7 documentation claims Phase 8 capability.
|
||||
- Real integration is a separately reviewed phase, consistent with the
|
||||
[future integration rule](architecture.md) recorded for every prior phase.
|
||||
|
||||
## Related
|
||||
|
||||
- [Architecture and dependency boundary](architecture.md)
|
||||
- [Mock ControlPlanePort](phase-07-mock-control-plane-port.md)
|
||||
- [Capability disposition](phase-07-capability-disposition.md)
|
||||
</content>
|
||||
@@ -0,0 +1,95 @@
|
||||
# Phase 7 Mock ControlPlanePort
|
||||
|
||||
`Phase7MockControlPlaneAdapter` is a deterministic, serializable, in-memory
|
||||
model of Paperclip's control-plane **semantics** — not its transport. It imports
|
||||
no server route, service, database, or provider binding, and it holds no
|
||||
credential.
|
||||
|
||||
Sources: `src/mock-core/phase7-mock-control-plane-adapter.ts`,
|
||||
`src/mock-core/phase7-control-plane-types.ts`. Shared contract:
|
||||
`src/conformance/control-plane-port.ts`.
|
||||
|
||||
## Entity domains
|
||||
|
||||
The adapter models ten renderable entity domains, which are exactly the domains
|
||||
the [scenario explorer](phase-07-scenario-explorer.md) diffs:
|
||||
|
||||
`tasks`, `comments`, `documents`, `interactions`, `approvals`, `artifacts`
|
||||
(artifacts and work products), `blockers`, `workspace` (workspace services),
|
||||
`budget`, and `run`.
|
||||
|
||||
The fixture state also carries supporting records — company, actors, wakes,
|
||||
audit, decisions, idempotency, faults, counters, and a fixture clock — that back
|
||||
behavior but are not diff domains.
|
||||
|
||||
## Operations
|
||||
|
||||
- **Lifecycle:** `start()` / `stop()` only change local object state.
|
||||
- **Runs:** `openRun()` / `openFixtureRun()` return a run context with atomic
|
||||
checkout; `context(runId)` reads it back.
|
||||
- **Events:** `appendEvent()` accepts both canonical PRP events and native run
|
||||
events with deduplication, replay-conflict detection, and contiguous-sequence
|
||||
tracking; `replayEvents()` replays from an exclusive cursor.
|
||||
- **Commands:** `applyCommand(envelope)` executes a semantic command with
|
||||
idempotency and an optimistic `expectedRevision`.
|
||||
- **Sessions:** `loadSessionCheckpoint()` / `checkpointSession()`.
|
||||
- **Terminal:** `completeRun()` reconciles a terminal disposition — `done`,
|
||||
`blocked`, `needs_review`, or `yielded` — and fans out blocker wakes.
|
||||
- **Introspection:** `snapshot()`, `decisionRecords()`, `serialize()`, and
|
||||
static `restore()`.
|
||||
|
||||
The eighteen semantic command kinds (`report_progress`, `write_document`,
|
||||
`request_human_input`, `resolve_human_input`, `register_deliverable`,
|
||||
`set_dependencies`, `create_task`, `request_approval`, `decide_approval`,
|
||||
`comment_on_approval`, `control_workspace_service`, `record_budget_usage`,
|
||||
`finish_task`, `block_task`, `request_review`, `release_task`, `schedule_wake`)
|
||||
are the mock side of the [semantic tool catalog](phase-07-semantic-tools.md).
|
||||
|
||||
## Determinism
|
||||
|
||||
- **Fixture clock.** Every timestamp derives from `clock.epochMs + tick*1000`
|
||||
and the tick increments on each read. The default epoch is
|
||||
`2026-08-09T00:00:00Z`. There is no RNG anywhere.
|
||||
- **Deterministic IDs.** Every generated ID is `fixture-<kind>-<nnnn>` from a
|
||||
monotonic per-kind counter.
|
||||
- **Immutability.** Inputs and outputs are `structuredClone`d and every returned
|
||||
value is deep-frozen. Canonical JSON with sorted keys backs every dedupe and
|
||||
replay comparison.
|
||||
- **Serializable.** `serialize()` / `restore()` round-trip under the schema tag
|
||||
`paperclip.phase7.mock-state.v1`.
|
||||
- **Scripted faults.** A fault rule injects a bounded number of
|
||||
`retryable_error` or `lost_ack` effects; a lost-ack command commits once and
|
||||
returns its stored result on retry.
|
||||
|
||||
## Boundary
|
||||
|
||||
The adapter is a pure state machine: no network, no database, no provider SDK,
|
||||
no credentials. Secrets appearing in wake payloads are redacted against
|
||||
`/authorization|credential|api.?key|secret|token/i`; the adapter test seeds
|
||||
`apiKey` and `Bearer` values and asserts they never surface. See
|
||||
[authorization and exposure](phase-07-authorization-and-exposure.md).
|
||||
|
||||
## Shared conformance
|
||||
|
||||
The adapter satisfies the same `runControlPlanePortConformance` contract used by
|
||||
earlier phases: it rejects open-binding violations, recovers from source gaps,
|
||||
treats duplicates idempotently, and rejects event-id, sequence, replay-binding,
|
||||
and result mutations (failing closed with `native_event_replay_conflict`).
|
||||
|
||||
## Running the tests
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
|
||||
src/conformance/control-plane-port.test.ts \
|
||||
src/mock-core/phase7-mock-control-plane-adapter.test.ts
|
||||
```
|
||||
|
||||
Nine tests, no npm alias of their own — they also run inside
|
||||
`pnpm --filter @paperclipai/paperclip-runner test`.
|
||||
|
||||
## Related
|
||||
|
||||
- [Semantic tool catalog](phase-07-semantic-tools.md)
|
||||
- [Authorization and exposure](phase-07-authorization-and-exposure.md)
|
||||
- [Future binding boundary](phase-07-future-binding-boundary.md)
|
||||
</content>
|
||||
@@ -0,0 +1,86 @@
|
||||
# Phase 7 Browser Scenario Explorer
|
||||
|
||||
The scenario explorer is a read-only browser surface over all 106 conformance
|
||||
cases. It runs each scenario in the page against the
|
||||
[mock control plane](phase-07-mock-control-plane-port.md) through the
|
||||
[semantic tool runtime](phase-07-semantic-tools.md) and renders the resulting
|
||||
run artifact. It never re-judges parity, never leaves its own origin, and holds
|
||||
no credential.
|
||||
|
||||
Sources: `src/phase7/` (scenario index, runner, parity, state diff) and
|
||||
`examples/phase7-explorer/`. Interaction contract:
|
||||
[Phase 7 scenario explorer UX](design/phase-7-scenario-explorer-ux.md).
|
||||
|
||||
## The run artifact
|
||||
|
||||
Running a scenario produces one immutable artifact carrying:
|
||||
|
||||
- **Tool exposure** — which semantic tools were available, and for optional
|
||||
tools, the grant that unlocked each one.
|
||||
- **Control-plane actions** — control-plane-owned steps (for example checkout),
|
||||
each labelled "no agent tool exists for this."
|
||||
- **Authorization records** — allow and typed-deny decisions; a denial carries
|
||||
the missing claim and no protected task data.
|
||||
- **State diff** — an immutable diff over the ten entity domains, with unchanged
|
||||
domains collapsed.
|
||||
- **Parity verdict** — the runtime's own verdict, plus Phase 7E's per-case
|
||||
result carried through in a separately labelled block. The explorer displays
|
||||
these; it does not recompute them.
|
||||
|
||||
## Read-only and frozen SDK
|
||||
|
||||
The explorer imports the frozen **0.1.2** public SDK through the package-local
|
||||
alias `@paperclip-runner-local/phase7`, which is deliberately not the published
|
||||
package name. It adds no SDK export and changes no published surface. Fake mode
|
||||
runs entirely in the page against checked-in fixtures and renders fixture time
|
||||
only, so two loads of the same route produce identical settled DOM.
|
||||
|
||||
## What the explorer shows
|
||||
|
||||
- **Home** — 16 group facets whose counts sum to 106, corpus stats, and example
|
||||
deep links.
|
||||
- **Picker** — a listbox with live filter chips, disabled zero-count values, and
|
||||
a clear-filters control.
|
||||
- **Transcript** — both channels (agent-visible and control-plane).
|
||||
- **Inspector** — four tabs: context (exposure and grants), authorization
|
||||
(allow/deny), state diff, and traceability.
|
||||
|
||||
## Boundary
|
||||
|
||||
- No network request leaves the explorer's own origin.
|
||||
- `localStorage` holds no run artifact, grant, or fixture payload.
|
||||
- Codex mode is a disabled option with a stated reason; wiring it to the Phase 5
|
||||
relay is deferred, and the browser holds no provider credential either way.
|
||||
- Secrets are redacted before display; redaction chips name the rule and never
|
||||
the value. See [authorization and exposure](phase-07-authorization-and-exposure.md).
|
||||
|
||||
## Accessibility
|
||||
|
||||
The 7F browser suite asserts: listbox arrow-key navigation with
|
||||
`aria-activedescendant`, WAI-ARIA tab arrow-key activation with exactly one
|
||||
tablist, every interactive control named, a polite live region announcing the
|
||||
settled verdict, three landmarks, a single `h1`, and — at 390px — one segment at
|
||||
a time with zero horizontal overflow.
|
||||
|
||||
## Running it
|
||||
|
||||
```sh
|
||||
# Open the explorer on 127.0.0.1:4183.
|
||||
pnpm --filter @paperclipai/paperclip-runner demo:phase7
|
||||
|
||||
# Scenario runtime, explorer components, and route determinism (45 tests).
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7
|
||||
|
||||
# Browser IA, determinism, evidence routes, boundary, a11y, responsive (19).
|
||||
pnpm --filter @paperclipai/paperclip-runner test:browser:phase7
|
||||
|
||||
# Deterministic 24-image acceptance set (12 routes x 2 viewports).
|
||||
pnpm --filter @paperclipai/paperclip-runner record:phase7
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [Eval conformance](phase-07-eval-conformance.md)
|
||||
- [Verification commands](phase-07-verification-commands.md)
|
||||
- [Phase 7F explorer verification](../knowledge/evidence/2026-08-09-phase-07f-explorer-verification.md)
|
||||
</content>
|
||||
@@ -0,0 +1,87 @@
|
||||
# Phase 7 Semantic Tool Catalog
|
||||
|
||||
The semantic tool catalog is the transport-neutral surface an agent may call. It
|
||||
is a frozen set of JSON-schema descriptors — no HTTP method, path, or provider
|
||||
detail. Provider-specific shapes are produced only by *bindings*, so every
|
||||
provider sees the same operation set.
|
||||
|
||||
Sources: `src/tools/phase7-semantic-tool-catalog.ts`,
|
||||
`src/tools/phase7-semantic-tool-types.ts`, `src/tools/phase7-tool-bindings.ts`,
|
||||
barrel `src/tools/index.ts`.
|
||||
|
||||
## Two dispositions, plus a separate control-plane list
|
||||
|
||||
A tool descriptor's `disposition` is either **`always_agent_tool`** or
|
||||
**`optional_agent_tool`**. Control-plane-owned operations are a separate frozen
|
||||
list, not a disposition — they are never exposed as tools. See
|
||||
[capability disposition](phase-07-capability-disposition.md) and
|
||||
[authorization and exposure](phase-07-authorization-and-exposure.md).
|
||||
|
||||
The catalog holds **36 tools**: 14 always-agent tools and 22 optional tools
|
||||
across 10 groups.
|
||||
|
||||
### Always-agent tools (14)
|
||||
|
||||
`get_task_context`, `get_task_history`, `list_documents`, `read_document`,
|
||||
`list_document_revisions`, `report_progress`, `answer_status_question`,
|
||||
`finish_task`, `block_task`, `request_review`, `write_document`,
|
||||
`request_human_input`, `register_deliverable`, `inspect_operation_result`.
|
||||
|
||||
### Optional tools (22), by group
|
||||
|
||||
| Group | Tools |
|
||||
| --- | --- |
|
||||
| discovery | `search_tasks`, `list_agents`, `list_projects`, `list_goals` |
|
||||
| delegation_dependencies | `create_task`, `set_dependencies` |
|
||||
| governance | `list_approvals`, `request_approval`, `decide_approval`, `comment_on_approval` |
|
||||
| cases | `list_cases`, `upsert_case` |
|
||||
| workspace_runtime | `get_workspace_runtime`, `control_workspace_service` |
|
||||
| routines | `list_routines`, `manage_routine` |
|
||||
| company_skills | `list_company_skills`, `sync_company_skills` |
|
||||
| secrets | `list_secret_metadata`, `read_secret_value` |
|
||||
| portability_admin | `export_company`, `administer_company` |
|
||||
| test_escape_hatch | `generic_api_request` |
|
||||
|
||||
## Descriptor shape
|
||||
|
||||
Each descriptor carries `operationId`, `version` (1), `title`, `description`,
|
||||
input/output JSON schemas, `disposition`, an optional `optionalGroup`,
|
||||
`requiredClaims`, and optionally `allowedRoles`, `taskModes`, a `sideEffectClass`
|
||||
(`read`, `task_write`, `company_write`, `governance`, `workspace_control`,
|
||||
`secret_read`, `admin`, `test_escape_hatch`), an `idempotency` level, `redaction`
|
||||
rules, and an abstract `mockCommandMapping`.
|
||||
|
||||
The `mockCommandMapping` is one of `context_read`, `snapshot_read`,
|
||||
`semantic_command`, `operation_result`, or `mock_extension` — describing *what*
|
||||
the tool does against the mock, never *how* a transport would carry it.
|
||||
|
||||
## Transport neutrality
|
||||
|
||||
Bindings, not descriptors, produce provider shapes, both derived from the same
|
||||
`visibleTools.tools` array:
|
||||
|
||||
- `Phase7FakeAgentToolBinding` emits `{operationId, description, inputSchema,
|
||||
outputSchema}`.
|
||||
- `Phase7CodexToolBinding` emits `{type: "function", name, description, strict:
|
||||
true, parameters}`.
|
||||
|
||||
Because both bindings derive from one array, the fake-agent and Codex operation
|
||||
surfaces are byte-identical. The [conformance suite](phase-07-eval-conformance.md)
|
||||
asserts this parity (the 18/18 fake-agent/Codex operation matrix).
|
||||
|
||||
## Running the tests
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
|
||||
src/tools/phase7-semantic-tools.test.ts
|
||||
```
|
||||
|
||||
The "catalog" describe asserts a unique, versioned, provider-neutral catalog and
|
||||
that every optional group is present.
|
||||
|
||||
## Related
|
||||
|
||||
- [Authorization and exposure](phase-07-authorization-and-exposure.md)
|
||||
- [Mock ControlPlanePort](phase-07-mock-control-plane-port.md)
|
||||
- [Eval conformance](phase-07-eval-conformance.md)
|
||||
</content>
|
||||
@@ -0,0 +1,42 @@
|
||||
# Phase 7 Verification Commands
|
||||
|
||||
Every command here runs from the repository root, is offline and deterministic,
|
||||
starts no Paperclip service, and holds no credential. All are prefixed
|
||||
`pnpm --filter @paperclipai/paperclip-runner`. The observed results for the
|
||||
acceptance record are in
|
||||
[the Phase 7 verification evidence](../knowledge/evidence/2026-08-09-phase-07-verification.md).
|
||||
|
||||
| Surface | Command | Expected result |
|
||||
| --- | --- | --- |
|
||||
| Capability contract completeness and drift | `check:phase7-inventory` | `Phase 7 inventory completeness and generated-output checks passed.` |
|
||||
| Contract validator negatives | `test:phase7-inventory` | 4 tests pass |
|
||||
| Mock control plane + shared port | `exec vitest run src/conformance/control-plane-port.test.ts src/mock-core/phase7-mock-control-plane-adapter.test.ts` | 9 tests pass |
|
||||
| Semantic tools + authorization/redaction | `exec vitest run src/tools/phase7-semantic-tools.test.ts` | 9 tests pass |
|
||||
| 106-case conformance | `test:phase7-evals` | 1 file (all 106 cases) passes |
|
||||
| Fake-agent matrix + bounded Codex + parity report | `report:phase7-evals` | `Phase 7 eval conformance passed: 106 cases across 16 groups.` |
|
||||
| Scenario runtime + explorer + routes | `test:phase7` | 45 tests pass |
|
||||
| Browser IA, accessibility, determinism, boundary | `test:browser:phase7` | 19 Playwright tests pass |
|
||||
| Screenshot acceptance set | `record:phase7` | 24 deterministic images |
|
||||
| Doc links + OKF bundle | `docs:validate` | 62 files, 26 OKF concepts, 5 indexes |
|
||||
|
||||
## Notes
|
||||
|
||||
- **No Rust toolchain is needed** for any Phase 7 command. The full
|
||||
`verify` target still builds the Rust workspace, but nothing above does.
|
||||
- **Determinism.** Fake-mode runs render fixture time only, so repeat runs and
|
||||
repeat screenshots are byte-identical.
|
||||
- **The parity report is generated on demand.** `report:phase7-evals` writes
|
||||
`knowledge/evidence/phase-07/eval-parity-report.{json,md}`, which are not
|
||||
committed. Because the `.md` carries no OKF frontmatter, run `docs:validate`
|
||||
**before** generating the report, or delete the report first. This dirty-tree
|
||||
interaction is tracked for the Phase 7H QA gate (`PAP-16907`).
|
||||
- **Browser libraries.** On minimal or rootless hosts, extract Playwright's
|
||||
browser libraries first; the package's `verify:rootless` target shows the
|
||||
pattern.
|
||||
|
||||
## Related
|
||||
|
||||
- [Clean-start tutorial](tutorials/phase-07-scenario-explorer.md)
|
||||
- [Eval conformance](phase-07-eval-conformance.md)
|
||||
- [Scenario explorer](phase-07-scenario-explorer.md)
|
||||
</content>
|
||||
@@ -0,0 +1,188 @@
|
||||
# Phase 7 Clean-Start Tutorial: Capability Contract, Mock Control Plane, and Scenario Explorer
|
||||
|
||||
**Time to first success: about 5 minutes.** Two commands take you from a clean
|
||||
checkout to 106 passing conformance cases and a browser you can click through.
|
||||
The whole tutorial runs from the repository root. It starts no Paperclip
|
||||
service, contacts no Paperclip control plane, clones no external eval
|
||||
repository, and holds no provider credential. Everything it needs is checked in
|
||||
under `packages/paperclip-runner/`.
|
||||
|
||||
Phase 7 does not integrate the runner into Paperclip. It builds a package-local
|
||||
model of what a native Paperclip run *would* do — a deterministic mock control
|
||||
plane, a transport-neutral semantic tool catalog, an authorization engine, a
|
||||
106-case conformance suite derived from the Paperclip Evals corpus, and a
|
||||
read-only browser explorer over all of it. Real integration is Phase 8 and
|
||||
requires separate approval. See
|
||||
[the future binding boundary reference](../phase-07-future-binding-boundary.md).
|
||||
|
||||
## What you need
|
||||
|
||||
- Node.js 20 or newer and pnpm 9 or newer. This tutorial was verified with Node
|
||||
22.22.2 and pnpm 9.15.4.
|
||||
- No Rust toolchain. Every command in this tutorial is TypeScript/Node only. (A
|
||||
full `verify` still builds Rust, but nothing here does.)
|
||||
- No network access after `pnpm install`.
|
||||
|
||||
Install the package workspace from the repository root:
|
||||
|
||||
```sh
|
||||
pnpm install --filter @paperclipai/paperclip-runner --lockfile=false --offline --ignore-scripts --dev
|
||||
```
|
||||
|
||||
## 1. Prove the 106-case conformance suite (about 1 minute)
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7-evals
|
||||
```
|
||||
|
||||
This runs the eval-derived conformance suite entirely in-process against the
|
||||
mock control plane. Expected final line:
|
||||
|
||||
```
|
||||
Test Files 1 passed (1)
|
||||
Tests 1 passed (1)
|
||||
```
|
||||
|
||||
The single test file drives all 106 cases across the 16 eval groups. To see the
|
||||
per-group counts, assertion classes, the fake-agent operation matrix, and the
|
||||
bounded Codex binding sample, generate the parity report:
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/paperclip-runner report:phase7-evals
|
||||
```
|
||||
|
||||
Expected final line:
|
||||
|
||||
```
|
||||
Phase 7 eval conformance passed: 106 cases across 16 groups.
|
||||
```
|
||||
|
||||
The report is written to
|
||||
`knowledge/evidence/phase-07/eval-parity-report.{json,md}`. It is a
|
||||
generated-on-demand artifact, not a committed file; a clean checkout does not
|
||||
contain it. Delete it before you run `pnpm --filter @paperclipai/paperclip-runner docs:validate`
|
||||
(the report carries no OKF frontmatter and would otherwise fail the knowledge
|
||||
bundle check — see [Known gaps](#known-gaps-and-boundaries)).
|
||||
|
||||
The 16 groups and case counts are fixed by the capability contract:
|
||||
|
||||
| Group | Cases | Group | Cases | Group | Cases | Group | Cases |
|
||||
| --- | ---: | --- | ---: | --- | ---: | --- | ---: |
|
||||
| hb | 5 | co | 6 | st | 8 | cm | 6 |
|
||||
| se | 4 | su | 4 | bl | 5 | dp | 3 |
|
||||
| ix | 9 | ap | 6 | ar | 4 | er | 9 |
|
||||
| rf | 22 | mh | 4 | rs | 3 | wk | 8 |
|
||||
|
||||
## 2. Open the scenario explorer (about 2 minutes)
|
||||
|
||||
```sh
|
||||
pnpm --filter @paperclipai/paperclip-runner demo:phase7
|
||||
```
|
||||
|
||||
Open `http://127.0.0.1:4183/phase7-explorer/`. The explorer is read-only. It
|
||||
runs each scenario in the browser against the same mock control plane and
|
||||
renders the run artifact; it never re-judges parity and never leaves its own
|
||||
origin. Walk this path:
|
||||
|
||||
1. **Home.** Confirm 16 group facets whose counts sum to 106.
|
||||
2. **Pick a scenario.** Filter to `ap` and open `ap-mcp-gate-01`. The
|
||||
authorization view shows a deny row and a "1 deny" tab chip; the typed
|
||||
denial card carries no task data.
|
||||
3. **Read a heartbeat.** Open `hb-context-01`. The control-plane checkout entry
|
||||
is labelled "no agent tool exists for this" — it is control-plane-owned, not
|
||||
a semantic tool.
|
||||
4. **Inspect a manager scenario.** Open `rf-api-mgr-heartbeat-01`. Optional
|
||||
tools are listed with the grant that unlocked them, alongside the
|
||||
control-plane "no tool" list and a traceability panel.
|
||||
5. **Confirm the credential boundary.** Codex mode is a disabled option with a
|
||||
stated reason. `localStorage` holds no run artifact or grant, and the page
|
||||
makes no request beyond its own assets.
|
||||
|
||||
The explorer imports the frozen `0.1.2` SDK through the package-local
|
||||
`@paperclip-runner-local/phase7` alias — deliberately not the published package
|
||||
name. See [the browser explorer reference](../phase-07-scenario-explorer.md).
|
||||
|
||||
## 3. Read what each surface guarantees
|
||||
|
||||
The explorer renders four surfaces produced by the runtime. Each has a
|
||||
package-local reference page:
|
||||
|
||||
- [Capability disposition](../phase-07-capability-disposition.md) — how every
|
||||
skill/reference behavior and eval case is classified as control-plane-owned,
|
||||
always-agent-tool, or optional-agent-tool, and how the generated contract is
|
||||
produced and checked.
|
||||
- [Mock ControlPlanePort](../phase-07-mock-control-plane-port.md) — the
|
||||
deterministic in-memory adapter, its entity domains, and its boundary.
|
||||
- [Semantic tool catalog](../phase-07-semantic-tools.md) — the
|
||||
transport-neutral tools an agent may call.
|
||||
- [Authorization and exposure](../phase-07-authorization-and-exposure.md) —
|
||||
grants, typed denials, and secret redaction.
|
||||
- [Eval conformance](../phase-07-eval-conformance.md) — how the 106 cases are
|
||||
derived from the checked-in traceability derivative with no external repo.
|
||||
|
||||
## 4. Run the full focused verification set
|
||||
|
||||
Every command below is offline, deterministic, and package-local. The
|
||||
[verification commands reference](../phase-07-verification-commands.md) lists
|
||||
each one with its purpose and expected result.
|
||||
|
||||
```sh
|
||||
# Capability contract: completeness, uniqueness, one-to-one MCP folds, no drift.
|
||||
pnpm --filter @paperclipai/paperclip-runner check:phase7-inventory
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7-inventory
|
||||
|
||||
# Mock control plane adapter and shared ControlPlanePort conformance.
|
||||
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
|
||||
src/conformance/control-plane-port.test.ts \
|
||||
src/mock-core/phase7-mock-control-plane-adapter.test.ts
|
||||
|
||||
# Semantic tool catalog and authorization/redaction engine.
|
||||
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
|
||||
src/tools/phase7-semantic-tools.test.ts
|
||||
|
||||
# 106-case conformance and the parity/fake-agent/bounded-Codex report.
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7-evals
|
||||
pnpm --filter @paperclipai/paperclip-runner report:phase7-evals
|
||||
|
||||
# Scenario runtime, explorer components, and route determinism (45 tests).
|
||||
pnpm --filter @paperclipai/paperclip-runner test:phase7
|
||||
|
||||
# Browser information architecture, accessibility, determinism, and boundary.
|
||||
pnpm --filter @paperclipai/paperclip-runner test:browser:phase7
|
||||
|
||||
# Deterministic 24-image screenshot acceptance set.
|
||||
pnpm --filter @paperclipai/paperclip-runner record:phase7
|
||||
|
||||
# Documentation links and the OKF knowledge bundle (run with no generated
|
||||
# report present).
|
||||
pnpm --filter @paperclipai/paperclip-runner docs:validate
|
||||
```
|
||||
|
||||
The exact results captured for the acceptance record are in the
|
||||
[Phase 7 verification evidence](../../knowledge/evidence/2026-08-09-phase-07-verification.md).
|
||||
|
||||
## Known gaps and boundaries
|
||||
|
||||
- **The parity report breaks the OKF bundle if left in place.**
|
||||
`report:phase7-evals` writes `eval-parity-report.md` into the knowledge
|
||||
bundle without OKF frontmatter, so `docs:validate` fails while it is present.
|
||||
Delete it before validating. A clean checkout never contains it. This is
|
||||
recorded for the Phase 7H QA gate.
|
||||
- **Codex mode is disabled.** The explorer offers a bounded real-Codex binding
|
||||
sample only through `report:phase7-evals`; the browser holds no credential
|
||||
and the in-page Codex mode is inert with a stated reason.
|
||||
- **No Paperclip integration.** Nothing here touches a real control plane,
|
||||
database, or provider. Phase 8 (ACPX) binds the real adapter behind the same
|
||||
seam and requires separate CTO approval.
|
||||
|
||||
## Where this sits
|
||||
|
||||
- Plan: Phase 7 plan, issue `PAP-16897` (see its `#document-plan` document).
|
||||
- Child issues: 7A capability inventory (PAP-16898), 7B UX map (PAP-16899),
|
||||
7C mock adapter (PAP-16900), 7D semantic tools and authorization (PAP-16901,
|
||||
security gate PAP-16902, remediation PAP-16909), 7E conformance suite
|
||||
(PAP-16903), 7F browser explorer (PAP-16904, UX gate PAP-16905), 7G docs and
|
||||
evidence (PAP-16906), 7H clean-room QA (PAP-16907), Phase 7 checkpoint
|
||||
(PAP-16908).
|
||||
</content>
|
||||
</invoke>
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
type: Verification Evidence
|
||||
title: Phase 7 documentation, tutorial, and evidence manifest
|
||||
description: Package-local acceptance manifest cross-linking every focused Phase 7 verification command, its observed result, the parity/screenshot artifacts, and the security, UX, and QA gate issues.
|
||||
tags: [native-runner, phase-7, verification, conformance, evidence, manifest]
|
||||
status: stable
|
||||
generated: { by: anthropic/claude-opus-4-8, at: 2026-08-09T13:42:50Z }
|
||||
---
|
||||
|
||||
# Scope
|
||||
|
||||
This record is the Phase 7G evidence manifest (PAP-16906). It links each
|
||||
package-local, offline, deterministic verification command to the result
|
||||
observed on 2026-08-09, and to the reference pages, tutorial, parity report,
|
||||
and screenshot set that describe the Phase 7 surfaces.
|
||||
|
||||
Phase 7 contacts no Paperclip service, ACPX session, database, or provider
|
||||
credential. Every command below runs from the repository root against
|
||||
checked-in fixtures and the in-process mock control plane.
|
||||
|
||||
# Environment
|
||||
|
||||
- Node.js 22.22.2, pnpm 9.15.4. No Rust toolchain is required for any Phase 7
|
||||
focused command.
|
||||
- vitest 4.1.10; Playwright with its bundled Chromium.
|
||||
- No network access after `pnpm install`.
|
||||
|
||||
# Verification matrix
|
||||
|
||||
All commands are prefixed `pnpm --filter @paperclipai/paperclip-runner`.
|
||||
Results were captured on 2026-08-09 between 13:37Z and 13:39Z.
|
||||
|
||||
| Surface | Command | Observed result |
|
||||
| --- | --- | --- |
|
||||
| Capability contract completeness and drift | `check:phase7-inventory` | `Phase 7 inventory completeness and generated-output checks passed.` (exit 0) |
|
||||
| Contract validator negatives | `test:phase7-inventory` | 4 tests, 4 pass, 0 fail |
|
||||
| Mock control plane + shared port | `exec vitest run src/conformance/control-plane-port.test.ts src/mock-core/phase7-mock-control-plane-adapter.test.ts` | 2 files, 9 tests pass |
|
||||
| Semantic tools + authorization/redaction | `exec vitest run src/tools/phase7-semantic-tools.test.ts` | 1 file, 9 tests pass |
|
||||
| 106-case conformance | `test:phase7-evals` | 1 file, 1 test (drives all 106 cases) pass |
|
||||
| Fake-agent matrix + bounded Codex sample + parity report | `report:phase7-evals` | `Phase 7 eval conformance passed: 106 cases across 16 groups.` |
|
||||
| Scenario runtime + explorer components + routes | `test:phase7` | 3 files, 45 tests pass |
|
||||
| Browser IA, accessibility, determinism, boundary | `test:browser:phase7` | 19 Playwright tests pass (6.8s) |
|
||||
| Screenshot acceptance set | `record:phase7` | 24 deterministic images (12 routes × 2 viewports) |
|
||||
| Doc links + OKF bundle | `docs:validate` | `Documentation link validation passed (62 files).` and `OKF v0.2 validation passed (26 concepts, 5 indexes).` |
|
||||
|
||||
# Conformance shape
|
||||
|
||||
- 106 cases across 16 groups: hb 5, co 6, st 8, cm 6, se 4, su 4, bl 5, dp 3,
|
||||
ix 9, ap 6, ar 4, er 9, rf 22, mh 4, rs 3, wk 8.
|
||||
- Assertion classes: `agent_tool_contract`, `authorization_policy`,
|
||||
`combined_multi_hop`, `control_plane_invariant`, `restraint_no_call`.
|
||||
- Fake-agent/Codex operations: 18/18 executed.
|
||||
- Capability contract baseline: 152 skill/reference headings, 106 eval cases,
|
||||
258 normative rows, 41 legacy MCP aliases folded one-to-one into normative
|
||||
rows. See [the generated contract](../../docs/phase-07-capability-contract.md).
|
||||
|
||||
# Artifacts
|
||||
|
||||
- Parity report: generated on demand by `report:phase7-evals` into
|
||||
`knowledge/evidence/phase-07/eval-parity-report.{json,md}`. It is not a
|
||||
committed file; a clean checkout does not contain it, and the explorer renders
|
||||
"Not run" when it is absent.
|
||||
- Screenshots: the committed 24-image acceptance set and its manifest are under
|
||||
[Phase 7 evidence](phase-07/index.md).
|
||||
- Prior surface evidence:
|
||||
[Phase 7F browser scenario explorer verification](2026-08-09-phase-07f-explorer-verification.md).
|
||||
|
||||
# Documentation delivered
|
||||
|
||||
- Tutorial: [Phase 7 clean-start tutorial](../../docs/tutorials/phase-07-scenario-explorer.md).
|
||||
- Reference pages:
|
||||
[capability disposition](../../docs/phase-07-capability-disposition.md),
|
||||
[mock ControlPlanePort](../../docs/phase-07-mock-control-plane-port.md),
|
||||
[semantic tools](../../docs/phase-07-semantic-tools.md),
|
||||
[authorization and exposure](../../docs/phase-07-authorization-and-exposure.md),
|
||||
[eval conformance](../../docs/phase-07-eval-conformance.md),
|
||||
[scenario explorer](../../docs/phase-07-scenario-explorer.md),
|
||||
[future binding boundary](../../docs/phase-07-future-binding-boundary.md),
|
||||
[verification commands](../../docs/phase-07-verification-commands.md).
|
||||
|
||||
# Gate issues
|
||||
|
||||
- Security gate: `PAP-16902` (authorization and credential isolation review),
|
||||
with remediation `PAP-16909` (redact secret invocation results before trace
|
||||
or browser exposure). The `rs-secret-redaction` case and the
|
||||
`src/tools/phase7-semantic-tools.test.ts` redaction assertions exercise the
|
||||
remediated path.
|
||||
- UX gate: `PAP-16905` (explorer acceptance and screenshots), against the
|
||||
approved interaction map §9 twelve-route matrix.
|
||||
- QA gate: `PAP-16907` (clean-room QA and human checkpoint), which consumes this
|
||||
manifest and the tutorial from a clean checkout.
|
||||
|
||||
# Known gaps
|
||||
|
||||
- `report:phase7-evals` writes `eval-parity-report.md` into the knowledge bundle
|
||||
without OKF frontmatter, and `evidence/phase-07/index.md` does not link it, so
|
||||
`docs:validate` fails while the generated report is present. Delete the report
|
||||
before validating, or run validation first. Recorded for the Phase 7H QA gate
|
||||
as a candidate to relocate the generated report outside the bundle or exclude
|
||||
it from the OKF scan.
|
||||
- Codex mode in the explorer is disabled with a stated reason; the only real
|
||||
provider exercise is the bounded binding sample in `report:phase7-evals`.
|
||||
- No Paperclip control plane, database, or provider is contacted anywhere in
|
||||
Phase 7. Real integration is Phase 8 (ACPX) and requires separate approval.
|
||||
</content>
|
||||
@@ -14,4 +14,5 @@
|
||||
* [Phase 5 browser SDK verification](2026-08-08-phase-05-verification.md) - Targeted and package-acceptance evidence for the versioned SDK, both public consumers, fake/real drivers, accessibility, reconnect, replay, and screenshots.
|
||||
* [Phase 6 thin Paperclip adapter verification](2026-08-09-phase-06-verification.md) - Mock and database-backed conformance, authoritative completion, restart recovery, migration repair, atomic liveness, internal canary, and deterministic post-kill-switch legacy evidence.
|
||||
* [Phase 7F browser scenario explorer verification](2026-08-09-phase-07f-explorer-verification.md) - Scenario index, run artifacts, exposure, authorization, state diff, parity, route determinism, accessibility, and the 24-image screenshot acceptance set.
|
||||
* [Phase 7 documentation, tutorial, and evidence manifest](2026-08-09-phase-07-verification.md) - Command→result matrix for every focused Phase 7 verification, artifact links, and the security, UX, and QA gate issues.
|
||||
* [Phase 7 evidence](phase-07/) - Screenshot acceptance captures and the eval conformance parity report.
|
||||
@@ -0,0 +1,102 @@
|
||||
---
|
||||
type: Engineering Journal Entry
|
||||
title: Phase 7 reference docs, tutorial, evidence, and OKF
|
||||
description: Decisions, observed facts, gaps, and readiness evidence for the package-local Phase 7 documentation set and clean-start tutorial.
|
||||
tags: [native-runner, phase-7, documentation, tutorial, evidence, okf]
|
||||
status: stable
|
||||
generated: { by: anthropic/claude-opus-4-8, at: 2026-08-09T13:42:50Z }
|
||||
entry_kind: phase
|
||||
phase: "7"
|
||||
---
|
||||
|
||||
# Context
|
||||
|
||||
Phase 7 (PAP-16897) built a package-local model of a native Paperclip run:
|
||||
a generated capability contract (7A), a UX interaction map (7B), a deterministic
|
||||
mock `ControlPlanePort` (7C), a transport-neutral semantic tool catalog and
|
||||
authorization engine (7D, security gate 7D/PAP-16902, remediation PAP-16909),
|
||||
an eval-derived 106-case conformance suite (7E), and a read-only browser
|
||||
scenario explorer (7F, UX gate PAP-16905). Phase 7G (PAP-16906) produces the
|
||||
reference documentation, the clean-start tutorial, the evidence manifest, and
|
||||
this OKF entry. It was blocked until 7E and 7F were done because the tutorial
|
||||
and evidence must describe and exercise the final conformance and browser
|
||||
surfaces.
|
||||
|
||||
# Decisions
|
||||
|
||||
1. Keep the reference set flat under `docs/phase-07-*.md`, matching the existing
|
||||
per-phase reference convention, rather than introducing a new folder. Eight
|
||||
reference pages: capability disposition, mock control plane, semantic tools,
|
||||
authorization/exposure, eval conformance, scenario explorer, future binding
|
||||
boundary, and verification commands.
|
||||
2. Treat the hand-written capability-disposition page as a reader's guide to the
|
||||
generated `docs/phase-07-capability-contract.md`; never restate the generated
|
||||
table, which carries a "DO NOT EDIT" header and is rewritten by
|
||||
`generate:phase7-inventory`.
|
||||
3. Write the tutorial to run from the repository root with two commands to first
|
||||
success, no Rust toolchain, no Paperclip service, and no external eval
|
||||
repository. State time-to-first-success and exact expected output.
|
||||
4. Make the verification evidence file the single manifest: a command→result
|
||||
matrix, artifact links, and the security/UX/QA gate issues. Capture real
|
||||
observed output rather than asserting expected output.
|
||||
5. Do not commit the generated parity report. It is an on-demand artifact; a
|
||||
clean checkout does not contain it, and the tracked
|
||||
`evidence/phase-07/index.md` already documents it as generated.
|
||||
6. Update `docs/index.md`, `docs/architecture.md`, `README.md`, and the two
|
||||
knowledge indexes additively. Keep ACPX as Phase 8 and state that real
|
||||
Paperclip integration requires separate approval.
|
||||
|
||||
# Observed facts
|
||||
|
||||
All commands ran from the repository root, offline, on Node 22.22.2 / pnpm
|
||||
9.15.4, between 13:37Z and 13:39Z on 2026-08-09. Full matrix in
|
||||
[the verification evidence](../evidence/2026-08-09-phase-07-verification.md).
|
||||
|
||||
- `check:phase7-inventory` and `test:phase7-inventory` pass; the contract holds
|
||||
152 skill/reference headings, 106 eval cases, 258 normative rows, and 41
|
||||
legacy MCP aliases folded one-to-one.
|
||||
- Mock control plane (9 tests), semantic tools and authorization/redaction
|
||||
(9 tests), scenario runtime and explorer (45 tests), and the 106-case
|
||||
conformance suite all pass in-process.
|
||||
- `report:phase7-evals` reports 106 cases across 16 groups, 18/18 fake-agent
|
||||
operations, and the bounded Codex binding sample.
|
||||
- `test:browser:phase7` passes 19 Playwright checks (6.8s) for information
|
||||
architecture, route determinism, evidence routes, the credential boundary,
|
||||
accessibility, and responsive layout.
|
||||
- On a clean tree `docs:validate` passes 62 documentation files and an OKF
|
||||
bundle of 26 concepts across 5 indexes.
|
||||
|
||||
# Evidence
|
||||
|
||||
- [Phase 7 verification manifest](../evidence/2026-08-09-phase-07-verification.md)
|
||||
- [Phase 7F explorer verification](../evidence/2026-08-09-phase-07f-explorer-verification.md)
|
||||
- [Clean-start tutorial](../../docs/tutorials/phase-07-scenario-explorer.md)
|
||||
- [Verification commands reference](../../docs/phase-07-verification-commands.md)
|
||||
|
||||
# Failures
|
||||
|
||||
- With the generated parity report present, `validate-okf` fails:
|
||||
`eval-parity-report.md` has no OKF frontmatter and is not linked from
|
||||
`evidence/phase-07/index.md`. Deleting the on-demand report restored a clean
|
||||
pass. This is a real dirty-tree gap, not a documentation error; it is recorded
|
||||
below and handed to the QA gate.
|
||||
|
||||
# Known gaps
|
||||
|
||||
- The generated parity report is incompatible with the OKF bundle scan when left
|
||||
in the knowledge tree. Candidate fixes for Phase 7H (PAP-16907): relocate the
|
||||
generated report outside `knowledge/`, add OKF frontmatter plus an index link
|
||||
during generation, or exclude generated reports from the OKF scan.
|
||||
- Codex mode is disabled in the explorer; the only real provider exercise is the
|
||||
bounded binding sample in `report:phase7-evals`.
|
||||
- Phase 7 adds no real Paperclip integration. Phase 8 (ACPX) binds the real
|
||||
adapter behind the preserved seam and requires separate CTO approval
|
||||
(PAP-16908).
|
||||
|
||||
# Follow-up questions
|
||||
|
||||
- Should the generated parity report move under `knowledge/evidence/phase-07/`
|
||||
with generator-authored OKF frontmatter, or out of the bundle entirely?
|
||||
- Which of the eight reference pages should the Phase 8 binding work extend
|
||||
first when the real adapter lands behind the seam?
|
||||
</content>
|
||||
@@ -16,3 +16,4 @@
|
||||
* [Phase 6 thin Paperclip adapter design](2026-08-09-phase-06-design.md) - Proposed public-port seam, feature flag, kill switch, company/auth/governance boundary, durable replay, conformance matrix, and tutorial contract.
|
||||
* [Phase 6 feature-flagged Paperclip adapter implementation](2026-08-09-phase-06-implementation.md) - Persisted selection, production port, public-session task, canonical event allocation, server-owned finalization, cancellation, and kill-switch evidence.
|
||||
* [Phase 6 thin Paperclip adapter implementation](2026-08-09-phase-06-implementation.md) - Default-off native selection, package-owned sessions, canonical persistence/replay, server-owned finalization, recovery, and compatibility evidence.
|
||||
* [Phase 7 reference docs, tutorial, evidence, and OKF](2026-08-09-phase-07.md) - Documentation decisions, observed verification facts across every focused command, the parity-report OKF gap, and the deferred Phase 8 binding boundary.
|
||||
Reference in new issue
Block a user