The live matrix injected `idempotencyKey` into every case's dictated JSON
input, but only mutating operations declare that key -- read operations close
their schema with `additionalProperties: false`. `search_tasks` therefore
rejected the input the prompt told the model to send.
Nothing caught it. A rejected call still emits a `tool_call` entry, so the
observed-vs-expected call comparison matched, and a rejected read leaves the
control plane `unchanged` exactly like a successful one, so the semantic
outcome matched by coincidence. The lone signal was an authorization mismatch,
and the hard-invariant gate only fires on escalation (denied -> allowed), never
on the reverse. se-get-issue-01 scored 1.0 on all five dimensions while its
call was failing.
Fixed on three levels:
- Root cause: derive the retry key from the catalog descriptor so the dictated
input always satisfies the operation's own schema.
- Harness gate: fail any live case whose typed result came back `ok: false`,
so a denial can never pass on matching calls and state alone.
- Scorer: a call the case declared allowed but that was denied now fails
`semantic_outcome` -- a rejected call cannot have produced its effect.
Trajectory stays green, since the model did choose the right tool.
Negative control: reintroducing the injection now aborts the matrix with
`deniedCalls: ["search_tasks"]` instead of reporting a silent pass.
Live matrix re-run 16/16 with zero authorization mismatches; the fault matrix
is unchanged (8 green / 8 red, mean 0.691), confirming the scorer change only
catches genuine denials.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The committed matrix was captured while fontconfig resolved the sans stack
to the DejaVu Sans fallback; ambient user-font churn then made Inter (the
stack's first choice) resolve, drifting all 24 shots at once. Re-recorded
under Chromium 151.0.7922.34 with Inter active, taught the recorder to
refuse to record or compare when Inter does not resolve distinctly, and
wrote the recording environment into the generated evidence index.
PAP-17001
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both findings from the exact-build security review (PAP-16983) against
`b73524ec2ac5`.
Sensitive information disclosure: every projected view carried raw
`provider_event` records and unredacted tool results, so one ordinary turn
handed an admitted browser provider protocol traffic, the echoed prompt,
provider thread/turn/item identifiers, model and token metadata, and complete
tool payloads. Redaction now happens where evidence is recorded, not where it
is rendered — `redactPhase7EvidenceData` is the only way an entry enters a live
session, so a raw payload never reaches the snapshot, the store, a frame, or a
log. `toPhase7PublicThreadView` then publishes an allowlisted DTO on every
response path, aliases provider-authored identifiers consistently, and scrubs
withheld provider identity. A streamed failure answers with a code and fixed
operator copy instead of the underlying message.
Cross-session BOLA: the only capability was minted once per gateway process, so
possession of a session id authorized reads and mutations of it. Each surface
now mints a per-browser capability, stores only its SHA-256 with the session,
and compares it in constant time on every session-scoped route. A valid id
without its capability is answered 404, exactly like an id that never existed.
`New chat`, a scenario POST, and reset rotate the capability after revoking the
caller's prior bindings.
Also normalises the stream frame cap to encoded UTF-8 bytes rather than UTF-16
code units, which the review flagged as low-severity hardening.
Each new test was observed failing against the reinstated rejected behaviour
(19 failed, 18 pre-existing passed) before passing against the fix.
Refs PAP-16991.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The clean-room chat revealed a finished reply instead of writing one: the
live session accumulated agent deltas in its turn waiter, the package
session server awaited the whole turn and answered with one JSON body, and
the browser waited on `response.json()`. Nothing between the provider and
the page could show work in progress, so the board saw the answer appear
all at once.
A turn now streams:
- `Phase7LiveSession` publishes bounded, ordered live turn events and
mirrors the waiter's assembled text into the transcript entry the reply
will finally occupy, so one rendered card grows and the settled text is
still assembled in exactly one place.
- `src/phase7/turn-stream.ts` defines the NDJSON frame codec and reader.
The decoder refuses a replayed, reordered, oversized, late, or truncated
frame rather than applying a stale view.
- `POST /api/phase7/ui/message` flushes headers, then writes one frame per
event-loop tick of provider activity. Admission checks still answer with
a JSON body and their own status code, because they are decided before
the first frame. Client disconnect interrupts the provider turn.
- The browser client consumes frames and renders each projection in place;
the settled payload is applied last and remains the authority. Stop,
reset, and `New chat` abandon the stream, and frames from an abandoned
turn generation are dropped.
Measured against real Codex through real runnerd: headers at 265 ms, 62
growing assistant states from the first character to a 301-character reply
at 3.1 s, the last interim state identical to the settled one, and a single
assistant card in the settled thread.
The scenario explorer and every fixture, preset, and replay route are
unchanged and stay deterministic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds a second primary path beside the preset scenario explorer. `#/chat`
opens a blank thread on a freshly minted mock tenant — one company, one
agent, one blank issue — and drives real Codex through real runnerd. The
scenario explorer and its deterministic deep links are unchanged.
The clean room is live-only by construction. The route pins `mode=live`
and drops `shot`/`at`, the package server has no scripted path for it and
refuses any non-live projection, and a failure to start runnerd or Codex
is reported with a retry rather than replaced by a fixture.
`createPhase7CleanRoomSessionInput` seeds the minimum a runner session
needs and fails closed if anything else is present, so "starts blank" is
enforced at the seam rather than assumed. The exposure profile is fixed
and inspectable while the calls stay conversation-driven; three grants
(`governance:approvals:decide`, `workspace:control`,
`test:generic_api_request`) stay withheld so a typed denial is reachable
in a chat nobody scripted.
`Reset` and `New chat` retire the previous session's authority, drop its
workspace, and mint new mock identities; the old session id answers 404
afterwards. The projection now publishes the real-API block as a
`network-guard-<sessionId>` control-plane record, so "no request reached
a real Paperclip API" is inspectable in the drawer on both surfaces.
Also fixes two things the clean room surfaced: the unanchored `sk-`
credential heuristic matched mock ids of the form `task-cleanroom-<token>`,
and a hash change left one committed frame reporting `settled` for the
surface just navigated away from.
The 24-image screenshot matrix is re-recorded: the surface rail carrying
the clean-room entry is a real, always-visible element, and it reproduces
byte-for-byte against Chromium 151.0.7922.34.
Verified: test:phase7 159/159; test:browser:issue-thread 58/58 with axe
clean; check:phase7:ui 24/24 byte-identical; docs:validate 93 files, 34
OKF concepts; smoke:phase7:cleanroom all 15 assertions true against real
Codex 0.132.0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Document the final Phase 7 build 865d1a72 for clean-room QA and 7L review:
add the live process topology to architecture, an execution-modes reference
that names Real Codex / Real runnerd / Mock Paperclip and the fake-vs-live
eligibility rules, a canonical clean-start issue-thread tutorial (offline fake
mode plus optional real Codex), the authoritative Revision 3/4 evidence
manifest, the screenshot-index eligibility note, and the OKF journal entry.
Mark the Revision 2 preview (build da0d32d74a and its URL) ineligible for
final acceptance. Offline commands re-run on this tree; docs:validate green
(89 files, 33 OKF concepts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Revisions 12-14 verified on the re-captured chat-streaming shots (byte-identical on re-record). Revision 15 accepts the post-capture composer disclaimer and isolated-mock wording from the demo-hardening commits and refreshes the five captures they staled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hold staged replay on a partial model message, limit activity to revealed records, and visibly explain disabled controls. Re-record the two streaming acceptance shots.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Adds a chat surface to the Phase 7 browser explorer where the board sends
prompts to a scenario and sees exactly how the mock Paperclip control plane
was used on every turn — the point of the phase is that the activity stays
visible rather than hidden behind the conversation (PAP-16916).
Runtime owns the turn model, so the UI derives nothing:
- `Phase7ChatSession` keeps one long-lived mock core across turns, so the
state a turn changes is still there for the next one. Turn 0 is the
control-plane seed (checkout, wake routing, exposure) that exists before
anyone can type.
- Timeline entries gain a `turn`; the artifact gains `turns[]` with per-turn
exposure, authorization records, state diff, reconciliation events, counts,
and a turn-scoped verdict. A new `user_message` kind joins the five
existing ones.
- `phase7TurnParity` judges a turn on what that turn did. A session-level
expectation about work nobody has asked for yet would read as a failure of
turn 1 rather than as work still to come.
- Recording machinery moves to `scenario-execution.ts`, shared with the 7F
single-shot runner so a scenario reads the same either way.
Two scenarios carry authored multi-turn scripts: `ap-mcp-gate-01` (allowed
turn, then a denied one) and `ix-checkbox-01` (open an interaction, then the
board answers it — a control-plane-owned action with no agent tool, which is
what makes the control plane schedule the continuation wake). Every other
corpus scenario falls back to its 7F plan as one scripted turn.
The surface implements the approved 7I interaction map: turn strips bridging
each turn to its evidence, the six-section activity stream, the mobile
Chat/Activity segments with a deny count chip, reset and scenario-switch
confirmation, and deterministic deep links including a staged streaming hold.
Deviations taken during implementation are recorded in §11a of the map rather
than left for a reviewer to find.
No ACPX implementation, no real control-plane integration, no credential in
the browser: Codex mode stays disabled with a named reason and scripted mode
drives the same in-page mock core offline.
Tests: 30 new runtime/component tests and 18 browser tests; 79 phase-7 unit
and 43 phase-7 browser tests pass. Ten §11 screenshots recorded by
`record:phase7i`, which fails the run on any horizontal page scroll.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Remove trailing generation-wrapper markup (</content>, plus </invoke> in the
tutorial) that had leaked at EOF into eleven committed Phase 7 reference,
tutorial, evidence, and OKF-journal files.
Sync the stated suite counts in the current-state docs (tutorial,
scenario-explorer reference, verification-commands reference, and OKF journal)
to the verified results: test:phase7 49, test:browser:phase7 25,
docs:validate 73 files / 28 OKF concepts / 5 indexes. Drop a brittle wall-clock
timing from the journal. The dated 7F verification evidence and log.md milestone
entry keep their point-in-time 45/19 figures on purpose.
Add a wrapper-markup guard to docs:validate (via scripts/lib/doc-wrappers.mjs)
that fails with the exact file and line when generator envelope tags reappear,
plus a node:test regression suite wired into the package test script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 7G (PAP-16906): package-local reference pages for capability
disposition, mock ControlPlanePort, semantic tools, authorization and
exposure, eval-derived conformance, browser explorer, the future Phase 8
binding boundary, and the focused verification commands; a clean-start
tutorial that runs from the repo root with no Paperclip service or external
eval repo; an OKF evidence manifest cross-linking every focused command to
its observed result and the security/UX/QA gate issues (PAP-16902, PAP-16905,
PAP-16907); and the Phase 7 OKF journal entry. Additive updates to the docs
index, architecture boundary, and knowledge indexes. Keeps ACPX as Phase 8.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the standalone, package-local Phase 7 scenario explorer against the
approved interaction map (docs/design/phase-7-scenario-explorer-ux.md).
Scenario runtime (src/phase7, no DOM, no network):
- Scenario index over all 106 eval cases and 16 groups, with actor role, task
mode, fixture seed, and catalog claims declared in a profile table rather
than inferred from case prose. Claims that cannot unlock an operation in a
scenario's task mode or for its actor role are dropped, so no scenario shows
a grant that unlocks nothing.
- Deterministic fixture seeds and fake-agent plans: twelve authored plans for
the UX acceptance scenarios, derived plans for the rest. Derived plans only
invoke operations the run actually exposes, so any denial in a derived
transcript is a real policy result, not a scripting artefact.
- A run-artifact builder that executes each plan against the Phase 7C mock
control plane through the Phase 7D authorization engine and records the
ordered timeline (both channels), tool exposure, authorization records, an
immutable ten-domain state diff, and a parity verdict. Descriptor redaction
rules are applied at the artifact boundary so no raw secret can reach a
rendered record.
- Parity is judged in the runtime, never in a component. Phase 7E's per-case
result is carried through verbatim in its own labelled block and renders
"Not run" when no report is bundled.
Explorer (examples/phase7-explorer):
- Read-only surface over the frozen SDK: hash routing, scenario picker with
faceted filters, agent and control-plane transcript channels, and Context,
Authorization, State diff, and Parity inspector tabs. Tokens only; the
0.1.2 export surface is unchanged and the runtime is reached through the
deliberately package-local @paperclip-runner-local/phase7 alias.
- Checked-in inputs arrive as declared virtual modules, so the example
consumer keeps its boundary and fake mode does no network I/O.
Verification: 45 unit tests (index, plans, all-scenario runs and parity,
components, routes), 19 Playwright tests (route determinism, evidence routes,
credential boundary, keyboard paths, responsive), and the 24-image screenshot
acceptance set.
Fixes found while building: SDK badges overflowed at 390px inside disclosure
summaries; SDK badge capitalisation broke the label vocabulary; hash-change
deep links left the wrong mobile segment; screenshot capture leaked run state
between shots. All recorded as revisions in the interaction map.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Interaction map, screenshot acceptance matrix, and annotated mockup for
the Phase 7 browser scenario explorer (PAP-16899). Binds picker facets,
transcript channels, inspector tabs, deterministic routes, and the
twelve-shot acceptance set to the 7A capability contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Route corpus operations through cancellation, reconciliation, attention, and rollout entrypoints, materialize effects through owning services, and document the live-versus-policy proof boundary.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Exercise all atomic-liveness failpoints, make native cancellation retry-safe, and align the allowlist and verification evidence with the final database-backed matrix.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Records the QA pass for PAP-16837: clean-start tutorial executed in a real
browser, deterministic demos, a real Codex end-to-end turn, full package
`verify` acceptance (exit 0), and curated screenshots.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Five protocol corrections from the CTO review of the Phase 4b work.
Canonical runtime request outcome. The real driver emitted `resolution`
while the scripted driver and browser read `action`, so a real resolved
card reported an unknown outcome. `harnessRuntimeRequestOutcome` is now
the one payload every driver emits for a terminal `runtime_request.*`
fact, carrying `requestId`, `requestKind`, `turnId`, and `itemId` in the
payload as well as the event binding.
Decoded request identities. Path segments are split before they are
decoded, so a percent-encoded reserved character stays inside the segment
the browser meant it for; malformed encoding fails with 400.
Resolution shapes validated by request kind.
`parseHarnessRuntimeRequestResolution` requires `answers` for a
`user_input` submit and `content` for an elicitation submit, and rejects
a mismatched body instead of accepting it as an empty response. Both
drivers and the browser transport run it, and the request card now sends
elicitation answers as `content`.
One terminal fact per reconnect. The demo server snapshots after the
close rather than before it, so requests the close already cancelled are
not replayed into the driver's stale-request recovery path and reported
terminal a second time. Reading afterwards also carries the closing
events' source sequence into the resumed session.
Goal support retired only on a provider denial. A transport or protocol
failure during `thread/goal/get` said nothing about what the build
supports but still set `goals: false`. `CodexRpcError` now carries the
JSON-RPC code, and the capability is retired only when the provider
answers that it will not serve the method.
Three defects surfaced while verifying the above are fixed here too: a
same-origin `EventSource` GET sends no `Origin`, so requiring one on
`/stream` rejected the console's own subscription and no live-console
test could connect; the browser payload redactor covered only
`.paperclip` paths and leaked the app-server's `~/.codex/memories`; and
the browser suite shares one server across 19 sessions against an
interactive cap of 16, now sized through `PAPERCLIP_PHASE4B_MAX_SESSIONS`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add the package-local browser tracer over the Phase 4b protocol boundary as a
new Live console mode in the standalone devtool.
Surfaces derive only from canonical PRP events and the shared reducer: live
transcript and composer, same-turn steering with stale-turn recovery, three
distinct interrupt races, inline runtime request cards, capability-gated goal
controls, parent/child lineage, reconnect, refresh, replay, and a redacted
protocol inspector.
Add a deterministic scripted driver and eleven demo chats behind the same demo
server routes the real Codex driver uses, so every protocol state is reachable
without a provider account and PAPERCLIP_PHASE4B_DRIVER=codex swaps in the real
session.
Adapt shadcn/ui and Vercel AI Elements source with zero new runtime
dependencies, and extend the boundary check to keep it that way.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
UX gate PAP-16832: live-console interaction map for all spec 29.1
surfaces, shadcn/AI Elements adapt/reject decision record with a
zero-new-runtime-dependency rule, keyboard/accessibility acceptance
criteria, token additions, annotated 1440x900 and 390x844 mockups,
and OKF journal/index updates.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
