mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
Reconcile Pi prerequisites with the integrated runtime while holding admission
This commit is contained in:
1 parent
20ce062701
commit
f620e7157b
313 files changed
+13086
-2187
No files matched your search
+134
-11
@@ -31,6 +31,26 @@ profile, model qualification, environment, task, or ranking-snapshot change
|
||||
must change that fingerprint automatically so the dashboard can annotate the
|
||||
boundary instead of silently joining unlike totals.
|
||||
|
||||
Pi native definition 21 supplies one ordinary JSON write with the nonce followed
|
||||
by exactly one LF, then asks for a complete native read. The independently
|
||||
checked outcome remains 33 bytes, saved through the public managed-file API
|
||||
and copied into a fresh task after controller restart. Missing LF, extra LF,
|
||||
literal escapes, CRLF and stale values all fail. The cross-root denial, protected
|
||||
parent seed, permissions, deadlines and zero automatic retries remain required.
|
||||
Both runs also require a completed, untruncated native-read receipt containing
|
||||
the exact memory text and a withheld private-root target, and reject shell
|
||||
execution. Named workspace reads and unrelated/bootstrap text cannot substitute;
|
||||
Remote observer admission keeps the owned-run and active-lease checks. Its
|
||||
existing readiness RPC verifies the pinned native daemon before installation;
|
||||
legacy executionStage can remain preparing for a native run. Readiness and case
|
||||
deadlines remain unchanged.
|
||||
The first Sonnet measurement passed the byte/restart checks but used a shell read
|
||||
in the fresh task. Its original grade remains preserved.
|
||||
The user approved Sonnet 4.6 through OpenRouter as the explicit Pi qualification
|
||||
model; production model selection stays caller-controlled. Earlier DeepSeek
|
||||
attempts and their fingerprints remain historical evidence and do not qualify
|
||||
this new model/fixture combination.
|
||||
|
||||
The explicit [stock-harness suite](STOCK-HARNESS.md) wraps existing profiles with
|
||||
`productionDefaultHireProfile`: omit only `instructionsBundle` so the public
|
||||
hire route loads the shipped default, while preserving runtime, permissions,
|
||||
@@ -139,6 +159,14 @@ a deletion through public file APIs. Native turns 2 and 3 must copy/hash only th
|
||||
changed memory file, with a saved receipt and the same provider PID. Journal and
|
||||
Git stress fixtures retain fixed external bundles as controls. Keep the stable-PID
|
||||
oracle strict; `instruction-persistence` also covers cold restarts and quota handling.
|
||||
The explicit `rich-acp-warm-continuity` fixture uses the workspace-only prompt:
|
||||
ACP providers retain their unchanged AGENT_HOME and must preserve the same native
|
||||
session, runner instance, provider session, PID, and process start fingerprint.
|
||||
It does not request personal-file edits, because changed ACP agent files require
|
||||
provider retirement before collection. Pi's separate `agent-files-fresh-run`
|
||||
case retains changed-home save and fresh-task restoration coverage. This split
|
||||
does not weaken the stable-process oracle or change the Codex checkpoint fixture.
|
||||
|
||||
Native turns 1 and 2 include an actionable human review in the completion report's `attentionRequests`. Paperclip creates the review gate from that report. An explicit question-tool wait yields the turn and suppresses its final prose, so it is not interchangeable with this completion-review fixture. Turn 3 reports Done without another review.
|
||||
|
||||
Every selected case runs in its own isolated Paperclip process, and independent
|
||||
@@ -311,6 +339,13 @@ is explicit-only. No private control-plane hooks or direct database writes are u
|
||||
|
||||
## Pi native boundaries
|
||||
|
||||
Definition 23 and Pi controls definition 10 confirm `/proc/<pid>` absence
|
||||
separately when a proc read fails during Linux process exit. An existing
|
||||
unreadable process, an unreadable absence check, identity drift, and incomplete
|
||||
terminal evidence still fail. The observer retains closed causes for process
|
||||
reads, stat shape, runtime binding and terminal sealing; it never retains raw
|
||||
process arguments. Historical failed receipts remain failed.
|
||||
|
||||
The explicit-only `pi-native` suite has five local and four Daytona candidate
|
||||
cells, with no automatic retries. The remote suite excludes automatic deny-all
|
||||
because its initial native file read is itself denied. `native-questions` answers the real runner-owned Pi select, confirm, input
|
||||
@@ -319,8 +354,13 @@ Undisclosed text and independent workspace JSON prove delivery to the same live
|
||||
Pi's SDK cannot distinguish negative confirmation from dismissal; the expected
|
||||
result is explicitly `negative_or_cancelled`, not proof of cancellation.
|
||||
|
||||
`agent-files-fresh-run` writes a hidden nonce through native file tools to the
|
||||
registered AGENT_HOME, requires a stopped-run save receipt and public managed-file
|
||||
`agent-files-fresh-run` supplies one native write argument object whose content
|
||||
is the hidden nonce plus exactly one final LF. The agent replaces only the
|
||||
AGENT_HOME prefix in the supplied path; it preserves the JSON newline escape in
|
||||
the content argument, then reads the complete file once. A missing LF, literal
|
||||
backslash-and-n, repeated write, or claimed success cannot satisfy the independent
|
||||
byte oracle. The flow uses native file tools in the registered AGENT_HOME,
|
||||
requires a stopped-run save receipt and public managed-file
|
||||
readback, then restarts the server and verifies exact bytes from a fresh task. An
|
||||
attempted write to an unassigned isolated sibling path must fail without creating
|
||||
a file. `restrictive-denial` requires a correlated failed native write and absent
|
||||
@@ -334,20 +374,40 @@ no database writes, private hooks, or fabricated provider results are allowed.
|
||||
`native-pending-controller-restart` restarts the public controller while one Pi
|
||||
input callback remains unanswered. It requires the same durable interaction,
|
||||
request, live run, native session, turn and producer before and after restart.
|
||||
For local execution, the public run's exact PID, process group and start identity
|
||||
must identify an already-observed durable runner under this controller. The test
|
||||
preserves only that runner tree during restart, checks the same live identity
|
||||
afterward, and keeps the old process owner for complete final cleanup alongside
|
||||
the replacement controller. Other controller children are retired normally.
|
||||
Remote execution does not infer local process authority from remote PIDs.
|
||||
Only then does the browser submit previously undisclosed text. One durable
|
||||
resolution, one original successful turn and independently read exact workspace
|
||||
JSON prove delivery. A replacement run, replay, cancellation, expiry, rewritten
|
||||
request or merely reloaded browser cannot pass. Full states and PRP identities
|
||||
stay in private snapshots under the existing publication allowlist.
|
||||
The local runner's unique `turn.submitted` receipt may precede assignment of the
|
||||
provider turn ID. The oracle accepts that missing ID only before the single
|
||||
matching `turn.started` and request creation, with the same session and producer
|
||||
and increasing durable/source sequence numbers. Later missing or changed turn
|
||||
identities still fail.
|
||||
|
||||
### Pi file editing and registered artifacts
|
||||
|
||||
Pi's `extended-harnesses/file-edit-validate` seeds exact bytes before startup and
|
||||
requires one native edit lifecycle followed by a successful native bash execution
|
||||
with the exact nonce-bound byte-validation command as its projected title and a
|
||||
validation marker. A marker-only echo cannot pass. Independent final bytes must
|
||||
match the fixture. The
|
||||
real `register_deliverable` receipt, attachment metadata, publication activity,
|
||||
requires one native edit lifecycle followed by exactly one successful native
|
||||
bash execution with the exact nonce-bound byte-validation command as its
|
||||
projected title and a validation marker. A marker-only echo cannot pass. Final bytes must
|
||||
match the fixture. The Pi task prompt explicitly forbids additional shell calls,
|
||||
including metadata commands and repair attempts. It supplies the expected post-edit
|
||||
byte size and hash for registration. Extended definition 5 places the exact command in a
|
||||
fenced Bash block so the production Markdown editor preserves its operators and
|
||||
literal escapes. Definition 4's escaped-paragraph attempt keeps its failed grade.
|
||||
Extra Bash calls fail the unchanged oracle even when the
|
||||
edited file and downloadable artifact are correct. Extended definition 4 makes
|
||||
Pi's artifact title equal the exact filename required by the shared registered
|
||||
artifact check. Definition 3 attempts retain their original definitions and
|
||||
grades; its Pi prompt requested a different title. The real
|
||||
`register_deliverable` receipt, attachment metadata, publication activity,
|
||||
visible task attachment and authenticated public download must all agree on the
|
||||
file's bytes, hash, company, issue, agent and originating run. A file on disk or a
|
||||
model completion claim cannot substitute for publication. Daytona additionally
|
||||
@@ -383,6 +443,40 @@ ancestry and fresh PID/start-time checks. Pi overwrites Linux argv via
|
||||
`process.title`, so the private receipt explicitly uses pinned-parent entrypoint
|
||||
attribution and never claims original child argv. A pidfd targets only that child;
|
||||
worker death, broad process-name matching and controller Stop cannot substitute.
|
||||
The production bootstrap may name its held executable as `/proc/self/fd/3`
|
||||
or `/proc/self/fd/7` in the wrapper argv. That form is admitted only when the
|
||||
wrapper's corresponding descriptor and executable both have the exact sealed
|
||||
snapshot Node inode. The guard and wrapper entrypoint paths remain exact.
|
||||
Missing, foreign or other descriptor numbers fail before signalling.
|
||||
Production may also stage the runner executable as a link to the image's
|
||||
verified installation. The fault helper reads the resolved regular file, checks
|
||||
that the named link remained unchanged, and still requires its pinned hash and
|
||||
exact `/proc/<runner-pid>/exe` inode. Link replacement, missing targets and a
|
||||
different executable fail admission. Linux calibration covers this installation
|
||||
form as well as a copied runner.
|
||||
|
||||
The agent-memory fixture requires the nonce's UTF-8 bytes followed by exactly
|
||||
one line-feed byte (`0x0A`). Its prompt states that byte contract in plain text
|
||||
and provides the exact content as fenced JSON. Fenced task prompts use the rich
|
||||
editor's Markdown paste path; filling the editor directly produces escaped
|
||||
paragraph text instead of a code block. The issue API preserves literal escapes
|
||||
in real multiline bodies and only recovers self-escaped line breaks in legacy
|
||||
single-line bodies. Code fences and JSON escapes must survive both boundaries.
|
||||
The prompt explicitly states that native write never adds a newline and that
|
||||
complete native read preserves one when present. A credential-free probe of the
|
||||
installed Pi 1.0.0 tools checks both a 32-byte value and the 33-byte value with a
|
||||
final line feed; both write and read retain the exact supplied bytes.
|
||||
The prompt orders one memory write, one complete native read, a separate expected
|
||||
cross-root write denial, and then completion. Native paths use the exact current
|
||||
absolute agent directory; shell-variable expansion is not assumed. An incorrect
|
||||
memory result must be reported without claiming success. Native readback and the
|
||||
managed-file API must retain the exact bytes across a new task and controller
|
||||
restart. The byte graders remain unchanged.
|
||||
|
||||
Controller cleanup can admit a replacement group member only when its ancestry
|
||||
belongs to a separately revalidated, continuously owned process. A recycled
|
||||
numeric group, a changed PID/start identity, or any unowned live member still
|
||||
fails cleanup before signaling. This rule is recorded in `pi-native` version 12.
|
||||
|
||||
The runtime itself must emit `runtime_request.expired` for the original callback
|
||||
with `provider_process_lost` and `replayAllowed:false`, followed by native turn
|
||||
@@ -403,13 +497,18 @@ macOS runs metadata negatives and explicitly skips this Linux-only calibration.
|
||||
That calibration and the earlier fake-Pi wrapper/bridge tests do not count as the
|
||||
real paid Product lifecycle proof. This new candidate cell remains unqualified.
|
||||
|
||||
The pending-question controller-restart browser matcher accepts only the retained
|
||||
issue's UUID or public identifier and the exact retained interaction ID. The UI
|
||||
normally posts with the public identifier. Durable request, run, turn, session,
|
||||
producer and single-delivery assertions remain required after submission.
|
||||
|
||||
## Pi active controls
|
||||
|
||||
The explicit-only `pi-controls` suite adds `pending-permission-stop` and
|
||||
`same-turn-steering` on local and Daytona, each with one provider run, a
|
||||
120-second active-turn timeout and a 300-second attempt budget. These four
|
||||
control cases retain their behavior; the current matrix totals 26 Pi cells.
|
||||
Pi 1/profile 12 and coverage revisions intentionally change the affected suite
|
||||
Pi 1/profile 13 and coverage revisions intentionally change the affected suite
|
||||
fingerprints, so older qualification receipts cannot be reused. Catalog presence and
|
||||
deterministic calibration do not constitute paid qualification.
|
||||
|
||||
@@ -422,6 +521,26 @@ identity mapping. Pi does not emit Cursor/Copilot diagnostic notices; those
|
||||
notices are never synthesized. Earlier native reads can provide orientation;
|
||||
other native operations cannot substitute for the observed write.
|
||||
|
||||
On Daytona, the operator first publishes the setup instruction file after the
|
||||
owned observer is armed. If Pi delegates that native read, the fixture approves
|
||||
only its exact request through the public API with `accept` (allow once). The
|
||||
read must name the published random setup file, complete successfully in the
|
||||
same native run/turn/session/source, and leave the file hash unchanged. The
|
||||
fixture retains the request, resolution, completed read and both observer
|
||||
snapshots. Only the two hash-bound setup permission records are excluded from
|
||||
the write-permission count. All native read rows remain in the oracle. Another
|
||||
permission, edit, shell command, foreign path or incomplete read cannot receive
|
||||
this exemption. The tested write remains unanswered until Stop or browser Deny.
|
||||
The production permission policy and all write/retirement assertions stay in
|
||||
place. `pi-controls` version 7 and `pi-native` version 5 record this correction;
|
||||
older attempt fingerprints and grades remain historical evidence.
|
||||
|
||||
Native tool arguments can arrive after the start event. An earlier null target
|
||||
is allowed only until the same execution first supplies the exact expected
|
||||
path. Missing targets, conflicting paths, another execution's path, or a later
|
||||
loss of the proven path fail. The original start and permission rows remain
|
||||
bound by retained hashes through control dispatch and settlement.
|
||||
|
||||
Stop awaits the pending evidence write and rereads that boundary before sending
|
||||
one caller UUID to the public cancel API. It requires the original request's
|
||||
normalized cancellation closure, a cancelled terminal, and the same-scope
|
||||
@@ -432,9 +551,11 @@ Only after cancellation does it attempt a stale **decline**, which must return
|
||||
cancelled run and no automatic continuation.
|
||||
|
||||
Steering submits a random marker only in a browser comment after the permission
|
||||
is pending, then clicks that comment's production Steer button. It records the
|
||||
exact public POST's queue/revision/run binding and requires the saved run
|
||||
acknowledgment plus the Product facade's same-turn acknowledgment item. The raw
|
||||
is pending, binds the queued comment to the exact body submitted by the
|
||||
production Markdown editor, then clicks that comment's production Steer button. It records the
|
||||
exact public POST's queue/revision/run binding. A rejected public POST ends the
|
||||
journey before any denial. Success requires the saved run acknowledgment plus
|
||||
the Product facade's same-turn acknowledgment item. The raw
|
||||
Rust `acpx-control-*` transport echo is suppressed by the facade;
|
||||
`CodexHarnessSession.steer` emits the durable correlated item after the command
|
||||
acknowledges. A deterministic calibration invokes that actual producer. The
|
||||
@@ -730,3 +851,5 @@ as completed and end the native turn; Paperclip must retain a failed run with
|
||||
missing semantic finalization and an unfinished task. That is a denial outcome,
|
||||
not task success or operator cancellation. Stop during an unresolved permission
|
||||
remains a separate `native-active-stop/pending-permission-stop` gate.
|
||||
|
||||
Pi controls definition 9 and native definition 16 create an explicit title through the production search creation action and bind the task ID from the public creation response. The scoped task and assignee must match before any native control. Automatic naming is outside these strict native-operation fixtures; all existing permission, byte, process, run-count and cleanup assertions remain required. Preserve the failed title-lookup attempt as its original failure.
|
||||
+100
-1
@@ -1581,6 +1581,15 @@ death is excluded because this exact ownership mechanism requires Linux pidfd.
|
||||
The Python helper calibration uses a synthetic transport and a real title-changing
|
||||
Node child; it proves fault ownership only, never paid Pi lifecycle behavior. Run
|
||||
it on native Linux with pinned Node on PATH: `python3 tests/runner-e2e/pi-provider-fault.test.py`.
|
||||
The calibration covers direct launch and the production bootstrap's held FD 3
|
||||
and FD 7 launches. Descriptor launch requires the held descriptor and wrapper
|
||||
executable to match the sealed snapshot Node inode; an argv alias alone grants
|
||||
no authority. Native suite definitions 6 retain the original live failures and
|
||||
require new provider-death qualification after this fixture correction.
|
||||
The fault helper validates the profile pin against canonical normalized closure
|
||||
entries, matching production admission. JSON formatting is not part of that pin.
|
||||
The same free suite checks canonical hash admission, malformed metadata and
|
||||
changed entries before any signal is sent.
|
||||
This adds one cell to the pending Pi Product matrix (26: 13 local, 13 Daytona);
|
||||
no qualification or live success is implied by discovery or oracle tests.
|
||||
|
||||
@@ -1594,8 +1603,17 @@ the durable browser form bridge. The existing five extended-harness journeys
|
||||
continue to prove semantic Paperclip questions and planning separately. Personal
|
||||
file persistence uses two fresh task runs and independent byte checks; restrictive
|
||||
denial uses one run and requires both a failed tool receipt and no file effect.
|
||||
The first managed-file API response is saved before its byte assertion, including
|
||||
when the final LF is missing. An incomplete remote terminal receipt retains only
|
||||
validated completion flags, watcher counts, process counts and target hashes.
|
||||
Raw RPC fields and file bodies are omitted. These diagnostic records do not
|
||||
change a failed byte or retirement grade and do not justify an unchanged retry.
|
||||
`human-permission-denial` additionally requires the exact browser Decline response,
|
||||
delivered native denial and independent file/process observation through retirement.
|
||||
The restrictive Daytona fixtures approve only the exact operator-published
|
||||
setup-file read after observer arming. They retain its public resolution and
|
||||
completed native read before testing the separate write permission. This setup
|
||||
approval does not change production policy or answer the tested write.
|
||||
See [the fixture contract](FIXTURES.md#pi-native-boundaries) for the exact oracles
|
||||
and the limits of reconnect evidence.
|
||||
|
||||
@@ -1685,7 +1703,10 @@ The observer watches registered targets (including transient create/delete), use
|
||||
workspace changes and the exact runner process plus descendants. The single
|
||||
controller-owned `.paperclip-runtime/paperclip-runner` subtree is excluded from
|
||||
user-file inventory and mutation counts: it holds the binary, provider pack,
|
||||
context and active runtime state. Its location and identity remain checked, and
|
||||
context, active runtime state, and sandbox GitHub launchers, upload locks and
|
||||
per-command configuration. Launcher restaging after controller recovery uses
|
||||
that same runtime path. SSH and local launcher locations remain separate.
|
||||
Its location and identity remain checked, and
|
||||
test targets cannot use it. This exclusion is recorded in evidence; it does not
|
||||
claim that runtime-internal writes are covered by the user-file oracle.
|
||||
|
||||
@@ -1697,6 +1718,25 @@ as unverified metadata; the remote process is bound through the exact sandbox,
|
||||
run ID, lifecycle, process group and executable digest. Same-UID observer isolation
|
||||
is not an adversarial operating-system sandbox test.
|
||||
|
||||
Directory notifications remain counted. A notification for an existing directory
|
||||
preserves completeness only when its device/inode still match an already
|
||||
registered recursive watch. Newly created directories, replacements, symlinks,
|
||||
and unknown notifications remain incomplete. Pi native definition v13 also gives
|
||||
memory content a single JSON representation, including the required final LF;
|
||||
the exact managed bytes and fresh-task readback assertions remain unchanged.
|
||||
|
||||
Pi native definition v14 seeds `memory/.pi-e2e-parent.txt` through the public
|
||||
managed-file API before task admission. This creates the watched parent while
|
||||
leaving `memory/pi-native.txt` absent; the native write still has to supply all
|
||||
33 bytes and both turns must preserve the setup file. New or replaced watched
|
||||
directories still fail the oracle. Closed incompleteness reasons identify watch
|
||||
gaps or process ambiguity without retaining paths or raw errors. An incomplete
|
||||
receipt remains failed. A validated terminal receipt with a captured, retired
|
||||
process tree closes its observer without another RPC to a publicly deleted lease
|
||||
only when evidence is complete or its failures are known filesystem-watch gaps.
|
||||
Unknown causes, reused identities and live attached processes still need cleanup
|
||||
proof.
|
||||
|
||||
`--suite rich-acp-warm-continuity` adds six explicit cells: all three providers on
|
||||
local and Daytona. Three browser-driven turns must preserve native session,
|
||||
provider session, runner instance, PID/start identity and project workspace.
|
||||
@@ -1918,6 +1958,65 @@ are retained privately alongside the grading checkpoints for failure diagnosis.
|
||||
Claude receives a fresh provider home and config directory inside the disposable
|
||||
workspace so a user's installed skill cannot shadow the managed skill under test.
|
||||
|
||||
### Published-install Pi lane
|
||||
|
||||
The 26 explicit Pi cells can run against an independently installed public CLI
|
||||
and server instead of the source CLI. Supply all four reviewed pins:
|
||||
`PAPERCLIP_RUNNER_E2E_INSTALLED_CLI` (canonical `paperclipai/dist/index.js`),
|
||||
`PAPERCLIP_RUNNER_E2E_INSTALLED_CLI_SHA256` (bare SHA-256),
|
||||
`PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_ROOT` (canonical resolved public server
|
||||
package root), and `PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_SHA256` (its
|
||||
`dist/index.js` SHA-256). The CLI must resolve that exact server dependency, with
|
||||
matching public package versions. Build/install provenance and the complete
|
||||
installed dependency/assets inventory remain separate required evidence; the
|
||||
entrypoint checks alone do not prove that closure.
|
||||
|
||||
Run the explicit `paperclipai runtime setup pi` for that installation first.
|
||||
This lane rejects candidate flags, local/remote daemon overrides, provider asset
|
||||
or pack overrides, and Node injection. It launches the installed JavaScript CLI
|
||||
without a TypeScript loader, rechecks its pins on controller restart, and records
|
||||
`installed-cli-admission.json` in private attempt evidence. It supports only the
|
||||
explicit Pi extended, native, controls and warm suites, which require none of the
|
||||
source-only response barriers. Cursor/Copilot qualification gates stay intact.
|
||||
Pi's historical `qualificationCandidate` fixture selector remains a roster key;
|
||||
it no longer grants an opt-in when the source Pi declaration admits normal use.
|
||||
|
||||
Installed Pi Daytona cells also require the published plugin fixture inputs
|
||||
`PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN` (canonical package root),
|
||||
`PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY` (canonical JSON file),
|
||||
and `PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY_SHA256`.
|
||||
The authority schema `paperclip.e2e.installed-daytona-plugin/v1` pins a canonical
|
||||
`graphRoot` and four public packages (`plugin`, `sdk`, `shared`, `daytona`).
|
||||
Each has `root`, `packageSha256`, `entry`, and `entrySha256`; `plugin` also has
|
||||
`manifestSha256` and `workerSha256`. Roots must be the named packages beneath
|
||||
that graph's `node_modules`. The fixture verifies compiled public exports,
|
||||
exact dependency versions, resolved package identities, and file pins before
|
||||
launch and again immediately before the ordinary `/api/plugins/install` request.
|
||||
It records `installed-daytona-plugin-admission.json` privately. Missing pins in
|
||||
installed Daytona mode fail; they never select the source plugin as a fallback.
|
||||
Local cells need no Daytona plugin. Source-development lanes retain their
|
||||
existing fixture path. These entrypoint checks do not replace the separately
|
||||
required tar provenance and complete installed dependency inventory audit.
|
||||
The fixture inputs do not reach the production server environment.
|
||||
|
||||
Before a paid installed local cell, use the same launcher and WebServer chain
|
||||
without provider work:
|
||||
|
||||
```sh
|
||||
node --import ./cli/node_modules/tsx/dist/loader.mjs \
|
||||
tests/runner-e2e/launch.ts --installed-startup-only \
|
||||
--id extended-harnesses.runner-acpx-pi.local.hello-complete \
|
||||
--max-automatic-retries 0
|
||||
```
|
||||
|
||||
This mode requires the reviewed installed CLI pins, rejects provider credential
|
||||
inputs, skips local credential-file loading, and runs only health, browser UI,
|
||||
and zero-company checks. It retains separate private startup evidence and cannot
|
||||
produce a passing provider case. Existing process, IPC, and scratch cleanup stay
|
||||
in force. The installed lane invokes Playwright's public JavaScript bin directly
|
||||
with the current Node; pnpm's generated bin shim would inject `NODE_PATH` into
|
||||
the WebServer. Arbitrary ambient `NODE_OPTIONS` and `NODE_PATH` remain rejected.
|
||||
|
||||
|
||||
## Production hiring templates
|
||||
|
||||
|
||||
@@ -317,4 +317,28 @@ server environment. GitHub PAT shapes are included in retained-evidence scans.
|
||||
Candidates have no automatic infrastructure retries; spending must be reconciled
|
||||
before a deliberate repeat.
|
||||
|
||||
Copilot protection fixtures use production browser/public API permission and
|
||||
cancellation paths. Their local one-shot socket accepts a nonce/PID only and owns
|
||||
one fixed bounded child; it cannot select commands, arguments, or paths. The exact
|
||||
native client command is supplied by the fixture, while production notices retain
|
||||
only its SHA-256. The fixture keeps private process identities local, closes its
|
||||
owned socket/child in cleanup, and never signals API-reported PIDs. Directory
|
||||
watch loss makes the denial oracle incomplete; it must not become a no-effect pass.
|
||||
|
||||
Pi Daytona controls bind process identity to the independent Linux observer's
|
||||
PID, start ticks and boot ID within the exact admitted run/lease/sandbox. The
|
||||
controller's `processStartedAt` annotations may change as launch metadata settles
|
||||
and do not identify a remote process birth. Every remote snapshot and retirement
|
||||
seal must retain the original birth identity and complete no-effect journal.
|
||||
Local controls retain their separate public process-authority comparison.
|
||||
|
||||
Pi provider-death admission accepts the native bootstrap's `/proc/self/fd/3`
|
||||
or `/proc/self/fd/7` wrapper launch only after binding that held descriptor and
|
||||
the wrapper executable to the sealed snapshot Node inode. Exact guard and
|
||||
entrypoint paths, closure hashes, Pi title, run/lease ancestry and fresh process
|
||||
birth checks remain required. Other descriptor aliases, missing descriptors and
|
||||
foreign inodes fail before any signal. Native Linux calibration exercises both
|
||||
descriptor launches with synthetic owned processes and no provider credentials;
|
||||
it does not qualify paid Pi behavior or change previous failed grades.
|
||||
|
||||
The private resource-admission marker and raw cleanup results control recovery-state retention independently of evidence packaging. A worker crash after admission keeps the owner-only recovery database; a confirmed pre-allocation bootstrap failure does not. Neither the marker nor the database enters published evidence.
|
||||
@@ -146,10 +146,10 @@ describe("runner E2E catalog", () => {
|
||||
expect(localIntegrityTasks).toHaveLength(2);
|
||||
expect(openRouterBreadthTasks).toHaveLength(3);
|
||||
expect(runnerSuites.map((suite) => suite.expectedMatrixSize)).toEqual([
|
||||
63, 12, 6, 8, 2, 2, 2, 30, 3, 16, 16, 2, 6, 8, 46, 23, 15, 52, 6, 6, 20, 26, 52, 28, 18, 2, 6, 6, 12, 10, 48, 16, 10, 2, 1, 1, 116,
|
||||
4, 8, 10, 4, 4, 6, 63, 12, 6, 2, 30, 3, 16, 16, 2, 6, 8, 46, 23, 15, 52, 6, 6, 20, 26, 52, 28, 18, 2, 6, 6, 12, 10, 48, 16, 10, 2, 1, 1, 116,
|
||||
]);
|
||||
expect(validateRunnerCatalog()).toHaveLength(698);
|
||||
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(698);
|
||||
expect(validateRunnerCatalog()).toHaveLength(722);
|
||||
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(722);
|
||||
expect(
|
||||
runnerMatrix.filter((entry) => entry.suite.id === "core-compatibility"),
|
||||
).toHaveLength(48);
|
||||
@@ -460,7 +460,7 @@ describe("runner E2E catalog", () => {
|
||||
expect(contextIntegrityProfiles.map((profile) => profile.id)).toEqual(
|
||||
expect.arrayContaining(pendingContextIntegrityProfiles.map((profile) => profile.id)),
|
||||
);
|
||||
expect(UNQUALIFIED_PROFILE_GAPS.pi).toMatch(/no qualified model source/);
|
||||
expect(UNQUALIFIED_PROFILE_GAPS).not.toHaveProperty("pi");
|
||||
});
|
||||
|
||||
it("blocks pending profiles before provider admission", () => {
|
||||
|
||||
+80
-35
@@ -1,3 +1,9 @@
|
||||
import { nativeActiveStopTasks } from "./native-active-stop-tasks.js";
|
||||
import { piControlTasks } from "./pi-controls-cases.js";
|
||||
import { cursorNativeTasks } from "./cursor-native-cases.js";
|
||||
import { copilotProtectionTasks } from "./copilot-protection-tasks.js";
|
||||
import { piNativeTasks } from "./pi-native-cases.js";
|
||||
import { piFilePrompt } from "./pi-file-evidence.js";
|
||||
import { planTaskCases, planTaskProfile, planDefinitionDigest } from "./plan-task-cases.js";
|
||||
import { nativeCompletionTasks, nativeCompletionDefinitionDigest } from "./native-completion-cases.js";
|
||||
import { NATIVE_INSTRUCTION_SUITE, NATIVE_INSTRUCTION_BASE_SHA, nativeInstructionDefinitionDigest } from "./native-instruction-consolidation.js";
|
||||
@@ -5,8 +11,6 @@ import { nativeCompletionProfile, NATIVE_COMPLETION_BUDGET_CENTS } from "./nativ
|
||||
import { chatConfirmationTasks } from "./chat-cases.js";
|
||||
import { buildConnectionSuite } from "./connection-cases.js";
|
||||
import { hiringTemplateTasks, hiringTemplateProfile, hiringTemplateDefinitionDigest } from "./hiring-template-cases.js";
|
||||
import { nativeActiveStopTasks } from "./native-active-stop-tasks.js";
|
||||
import { cursorNativeTasks } from "./cursor-native-cases.js";
|
||||
import { instructionPersistenceTask } from "./instruction-persistence.js";
|
||||
import { apiResponseReadingTask } from "./api-response-reading.js";
|
||||
import { taskTitleTasks, taskTitleDefinitionDigest, TASK_TITLE_BUDGET_CENTS } from "./task-titles.js";
|
||||
@@ -168,6 +172,8 @@ function legacyProfile(input: {
|
||||
};
|
||||
}
|
||||
|
||||
const PI_QUALIFICATION_THINKING_LEVEL = "low" as const;
|
||||
|
||||
function nativeProfile(input: {
|
||||
id: string;
|
||||
label: string;
|
||||
@@ -219,6 +225,7 @@ function nativeProfile(input: {
|
||||
lifecycleMode: "per_turn",
|
||||
idleTimeoutMs: 300_000,
|
||||
...permissionConfig,
|
||||
...(input.acpxAgent === "pi" ? { piThinkingLevel: PI_QUALIFICATION_THINKING_LEVEL } : {}),
|
||||
env: {
|
||||
...(credentialRef ? { [input.credential]: credentialRef } : {}),
|
||||
// Codex's supported automation credential is CODEX_API_KEY. Keep
|
||||
@@ -352,7 +359,7 @@ export const extendedHarnessProfiles: readonly RunnerProfileFixture[] = [
|
||||
nativeProfile({
|
||||
id: "runner-acpx-pi", label: "Runner Pi (candidate)", provider: "acpx", acpxAgent: "pi",
|
||||
qualificationCandidate: "pi", credential: "OPENROUTER_API_KEY", model: ACPX_QUALIFICATION_MODELS.pi,
|
||||
modelQualification: { source: "candidate_runner_profile", qualificationId: "pi:0.0.33:0.84.2:openrouter" },
|
||||
modelQualification: { source: "candidate_runner_profile", qualificationId: "pi:0.0.33:1.0.0:openrouter" },
|
||||
}),
|
||||
];
|
||||
|
||||
@@ -1096,6 +1103,70 @@ export const extendedHarnessFileTask: RunnerTaskFixture = {
|
||||
};
|
||||
|
||||
export const runnerSuites: readonly RunnerSuiteFixture[] = [
|
||||
{
|
||||
id: "pi-controls", label: "Pi active controls", manualOnly: true,
|
||||
description: "Pending native-write Stop and browser-originated same-turn steering, with exact control receipts and independent retirement/no-effect evidence.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "pi"),
|
||||
environments: runnerEnvironments, tasks: piControlTasks, expectedMatrixSize: 4,
|
||||
definitionMetadata: { version: 10, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion,
|
||||
nativeArguments: "streamed-until-exact-target",
|
||||
remoteProcessIdentity: "observer-pid-startTicks-bootId",
|
||||
remoteBootstrapAdmission: "owned-active-lease-with-native-runtime-readiness-rpc-v1", remoteBootstrapApproval: "exact-published-native-read-public-accept-once-v1",
|
||||
controlPlaneSettlement: "required-scoped-result-and-terminal-after-runner",
|
||||
steeringComment: "exact-browser-submitted-markdown",
|
||||
steeringDispatch: "require-public-api-acceptance",
|
||||
pending: "paperclip.e2e.pi-control-pending.v1", stop: "paperclip.e2e.pi-stop-settlement.v1", steering: "paperclip.e2e.pi-steering-settlement.v1",
|
||||
permissionPolicy: "approve-reads", lifecycle: "per_turn", normalCompletionProvesStop: false, nativeFollowUp: "not-covered", providerDeath: "not-covered",
|
||||
remoteObservationCoverage: "continuous-through-owned-process-retirement", filesystemAfterRemoteRetirementObserved: false },
|
||||
},
|
||||
{
|
||||
id: "cursor-native", label: "Cursor native interactions", manualOnly: true,
|
||||
description: "Native question continuation, revision-bound plan decisions and restrictive permission denial with independent process and file evidence.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "cursor"),
|
||||
environments: runnerEnvironments, tasks: cursorNativeTasks, expectedMatrixSize: 8,
|
||||
definitionMetadata: { version: 2, qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.cursor.agentProfileVersion, modeAdmission: "native-config-ack", artifactExport: "pending-private-home", remoteEvidence: "owned-lease-sealed-observer" },
|
||||
},
|
||||
{
|
||||
id: "pi-native", label: "Pi native boundaries", manualOnly: true,
|
||||
description: "Pi native forms, registered agent files and human permission denial on local and Daytona execution; automatic deny-all remains local-only.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "pi"),
|
||||
environments: runnerEnvironments, tasks: piNativeTasks, expectedMatrixSize: 10,
|
||||
excludedExecutionIds: ["pi-native.runner-acpx-pi.daytona.restrictive-denial", "pi-native.runner-acpx-pi.local.native-pending-provider-death"],
|
||||
definitionMetadata: { version: 23, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", agentMemoryParent: "public-managed-file-seed-before-admission", incompleteTerminalCleanup: "retirement-retained-with-failed-watch", qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion, agentMemoryContent: "utf8-nonce-plus-final-lf", agentMemoryPrompt: "single-json-write-and-content-bound-native-read-both-runs", agentMemoryReadAuthority: "local-withheld-or-exact-remote-agent-run-file", taskPromptTransport: "fenced-markdown-paste-and-multiline-literal-escapes", nativeFinish: "current-contract-objective-evidence-refs", providerDeath: "daytona-exact-pi-child-pidfd-production-expiry", providerFaultExecutable: "stable-preinstalled-runner-link-and-snapshot-node-inode-with-held-bootstrap-fd-3-or-7", remoteBootstrapAdmission: "owned-active-lease-with-native-runtime-readiness-rpc-v1", remoteBootstrapApproval: "exact-published-native-read-public-accept-once-v1", remoteDenyAll: "unsupported-native-bootstrap-read-is-denied", remoteEvidence: "owned-lease-sealed-observer", pendingControllerRestart: "same-live-native-request-trusted-ancestry-cleanup" },
|
||||
},
|
||||
{
|
||||
id: "native-active-stop", label: "Stop an unanswered native permission", manualOnly: true,
|
||||
description: "Stop while one exact Cursor or Copilot native permission remains unanswered; require cancelled provider settlement, caller-owned acknowledgement, stale-answer refusal and independent retirement/no effects.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => ["cursor", "copilot"].includes(profile.qualificationCandidate ?? "")),
|
||||
environments: runnerEnvironments, tasks: nativeActiveStopTasks, expectedMatrixSize: 4,
|
||||
definitionMetadata: { version: 4, qualification: "pending", scheduling: "explicit-only", evidence: "paperclip.e2e.native-active-stop-settlement.v2", pendingObservation: "retained-api-before-caller-uuid-stop", normalCompletionAccepted: false, permissionPolicy: "approve-reads", lifecycle: "per_turn", remoteEvidence: "paperclip.e2e.native-active-stop-remote-retirement.v1", remoteObservationCoverage: "continuous-through-owned-process-retirement", filesystemAfterRemoteRetirementObserved: false, localObservationCoverage: "four-phases-through-cleanup", providerDeath: "not-covered" },
|
||||
},
|
||||
{
|
||||
id: "copilot-protection", label: "Copilot native protection", manualOnly: true,
|
||||
description: "Exact native denial with independently correlated provider settlement and acknowledged run Stop, plus attached command settlement with independent process evidence.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "copilot"),
|
||||
environments: runnerEnvironments, tasks: copilotProtectionTasks, expectedMatrixSize: 4,
|
||||
definitionMetadata: { version: 9, outputProhibition: "separate-publication-and-attachment", semanticCompletionEvidence: "paperclip.e2e.copilot-semantic-completion.v2", qualification: "pending", naturalSettlementObservationMs: 2000, scheduling: "explicit-only", evidence: "copilot_tool_evidence_v1", profileVersion: QUALIFIED_ACPX_PROFILES.copilot.agentProfileVersion, denialTerminal: "correlated-provider-settlement-and-audited-run-stop", denialSettlementEvidence: "paperclip.e2e.copilot-denial-settlement.v3", activeTurnCancellation: "not-implied-by-completed-provider-turn", settlement: "attached-finite-command-only", settlementMarker: "private-diagnostic-not-deliverable", remoteEvidence: "owned-lease-sealed-observer" },
|
||||
},
|
||||
{
|
||||
id: "rich-acp-warm-continuity", label: "Rich ACP warm continuity", manualOnly: true,
|
||||
description: "Three browser-driven turns with stable native session, runner process and workspace identity for Cursor, Copilot and Pi.",
|
||||
groups: ["native", "warm"],
|
||||
profiles: extendedHarnessProfiles.map(profile => ({ ...profile, buildAgent(input: AgentFixtureBuildInput) {
|
||||
const agent = profile.buildAgent(input);
|
||||
return { ...agent, adapterConfig: { ...agent.adapterConfig as Record<string, unknown>, lifecycleMode: "warm", idleTimeoutMs: 300_000 } };
|
||||
} })),
|
||||
environments: [localEnvironment, daytonaWarmEnvironment],
|
||||
// ACP collects changed agent files only after provider retirement. Keep this
|
||||
// process-continuity fixture on workspace writes; Codex retains its separate
|
||||
// managed-home checkpoint fixture, and Pi covers saved files in a fresh run.
|
||||
tasks: [{ ...daytonaWarmContinuityTask,
|
||||
buildPrompt: nonce => warmTurnInstructions(1, nonce),
|
||||
buildFollowupMessages: nonce => [warmTurnInstructions(2, nonce), warmTurnInstructions(3, nonce)],
|
||||
turnTimeoutMs: 120_000, attemptTimeoutMs: { local: 420_000, daytona: 420_000 } }],
|
||||
expectedMatrixSize: 6,
|
||||
definitionMetadata: { version: 2, qualification: "pending", scheduling: "explicit-only", identity: "native-session-runner-provider-session-process-start", agentFiles: "unchanged-home-process-continuity" },
|
||||
},
|
||||
{
|
||||
id: "public-mcp", label: "Paperclip through an assistant", manualOnly: true,
|
||||
description: "Paid assistant tool use plus actual team execution, browser OAuth consent, durable outcomes and authorization boundaries.",
|
||||
@@ -1131,22 +1202,7 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
|
||||
groups: ["legacy"], profiles: runnerProfiles.filter(p => ["legacy-codex", "legacy-claude"].includes(p.id)).map(blockerProfile),
|
||||
environments: [localEnvironment], tasks: blockerTasks, expectedMatrixSize: 6,
|
||||
definitionMetadata: { version: 1, instructions: "production-coordination-skill", grading: "saved-human-decision-ownership-and-resume", scheduling: "explicit-only" },
|
||||
},
|
||||
{
|
||||
id: "cursor-native", label: "Cursor native interactions", manualOnly: true,
|
||||
description: "Native question continuation, revision-bound plan decisions and restrictive permission denial with independent process and file evidence.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "cursor"),
|
||||
environments: runnerEnvironments, tasks: cursorNativeTasks, expectedMatrixSize: 8,
|
||||
definitionMetadata: { version: 2, qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.cursor.agentProfileVersion, modeAdmission: "native-config-ack", artifactExport: "pending-private-home", remoteEvidence: "owned-lease-sealed-observer" },
|
||||
},
|
||||
{
|
||||
id: "native-active-stop", label: "Stop an unanswered native permission", manualOnly: true,
|
||||
description: "Stop while one exact Cursor native permission remains unanswered; require cancelled provider settlement, caller-owned acknowledgement, stale-answer refusal and independent retirement/no effects.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "cursor"),
|
||||
environments: runnerEnvironments, tasks: nativeActiveStopTasks, expectedMatrixSize: 2,
|
||||
definitionMetadata: { version: 4, qualification: "pending", scheduling: "explicit-only", evidence: "paperclip.e2e.native-active-stop-settlement.v2", pendingObservation: "retained-api-before-caller-uuid-stop", normalCompletionAccepted: false, permissionPolicy: "approve-reads", lifecycle: "per_turn", remoteEvidence: "paperclip.e2e.native-active-stop-remote-retirement.v1", remoteObservationCoverage: "continuous-through-owned-process-retirement", filesystemAfterRemoteRetirementObserved: false, localObservationCoverage: "four-phases-through-cleanup", providerDeath: "not-covered" },
|
||||
},
|
||||
{
|
||||
}, {
|
||||
id: "native-provider-loss", label: "Lose a runtime with an unanswered native permission", manualOnly: true,
|
||||
description: "Lose the owned Cursor runtime while a native mutation remains unanswered; require a visible failed run, closed unanswerable input, stale-answer refusal and independent retirement with no effects or replay.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "cursor"),
|
||||
@@ -1161,27 +1217,13 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
|
||||
buildMatchers: () => [],
|
||||
}], expectedMatrixSize: 2,
|
||||
definitionMetadata: { version: 1, qualification: "pending", scheduling: "explicit-only", fault: "observed-per-turn-run-root-loss", remoteFaultAuthority: "pidfd-start-ticks-boot-id", replayAllowed: false },
|
||||
},
|
||||
{
|
||||
id: "rich-acp-warm-continuity", label: "Rich ACP warm continuity", manualOnly: true,
|
||||
description: "Three browser-driven turns with stable native session, runner process and workspace identity for Cursor.",
|
||||
groups: ["native", "warm"],
|
||||
profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "cursor").map(profile => ({ ...profile, buildAgent(input: AgentFixtureBuildInput) {
|
||||
const agent = profile.buildAgent(input);
|
||||
return { ...agent, adapterConfig: { ...agent.adapterConfig as Record<string, unknown>, lifecycleMode: "warm", idleTimeoutMs: 300_000 } };
|
||||
} })),
|
||||
environments: [localEnvironment, daytonaWarmEnvironment],
|
||||
tasks: [{ ...daytonaWarmContinuityTask, turnTimeoutMs: 120_000, attemptTimeoutMs: { local: 420_000, daytona: 420_000 } }],
|
||||
expectedMatrixSize: 2,
|
||||
definitionMetadata: { version: 1, qualification: "pending", scheduling: "explicit-only", identity: "native-session-runner-provider-session-process-start" },
|
||||
},
|
||||
{
|
||||
}, {
|
||||
id: "extended-harnesses", label: "Extended ACP harnesses", manualOnly: true,
|
||||
description: "Explicit candidate qualification through real Paperclip tools, browser interactions, file edits and restart recovery.",
|
||||
groups: ["native"], profiles: extendedHarnessProfiles, environments: runnerEnvironments,
|
||||
tasks: [...openRouterBreadthTasks, localIntegrityTasks[1]!, extendedHarnessFileTask],
|
||||
expectedMatrixSize: 30,
|
||||
definitionMetadata: { version: 2, qualification: "pending", scheduling: "explicit-only", admission: "host-exact-candidate-and-model", authenticatedDiscoveryDate: "2026-09-28", piFileEvidence: "seed-edit-execute-public-download-v1" },
|
||||
definitionMetadata: { version: 5, qualification: "pending", scheduling: "explicit-only", admission: "host-exact-candidate-and-model", authenticatedDiscoveryDate: "2026-09-28", piFileEvidence: "seed-edit-single-execute-public-download-v2", piFileArtifactTitle: "exact-filename", piFileCommandTransport: "fenced-bash-markdown-paste" },
|
||||
},
|
||||
{
|
||||
id: "instruction-persistence", label: "Instruction Persistence",
|
||||
@@ -1563,6 +1605,7 @@ export function suiteDefinitionHash(suite: RunnerSuiteFixture) {
|
||||
id: profile.id,
|
||||
model: profile.model,
|
||||
qualification: profile.modelQualification,
|
||||
...(profile.qualificationCandidate === "pi" ? { piThinkingLevel: PI_QUALIFICATION_THINKING_LEVEL } : {}),
|
||||
})),
|
||||
environments: suite.environments.map((environment) => ({
|
||||
id: environment.id,
|
||||
@@ -1662,6 +1705,8 @@ export function validateRunnerCatalog(): MatrixExecution[] {
|
||||
const connectionSuite = runnerSuites.find(suite => suite.id === "provider-connections")!;
|
||||
const allProfiles = [...connectionSuite.profiles, ...extendedHarnessProfiles, ...runnerProfiles, ...legacyAcpxProfiles, ...pendingContextIntegrityProfiles, ...openRouterBreadthProfiles, ...everydayProfiles.filter(p => !runnerProfiles.some(existing => existing.id === p.id))];
|
||||
const allTasks = [
|
||||
...piNativeTasks,
|
||||
...copilotProtectionTasks,
|
||||
...connectionSuite.tasks,
|
||||
extendedHarnessFileTask,
|
||||
...contextIntegrityTasks,
|
||||
|
||||
@@ -1,39 +1,5 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { mayAllocateRemoteResources, mustPreserveRecoveryState, runCleanupWithObservers, shouldKeepFailedDiagnostics, verifyCleanupAssertions } from "./cleanup-verification.js";
|
||||
|
||||
it("keeps late failures and incomplete publication for explicit diagnosis", () => {
|
||||
expect(shouldKeepFailedDiagnostics({ enabled: true, expectedResults: 1, results: [{ status: "failed" }] })).toBe(true);
|
||||
expect(shouldKeepFailedDiagnostics({ enabled: true, expectedResults: 2, results: [{ status: "passed" }] })).toBe(true);
|
||||
expect(shouldKeepFailedDiagnostics({ enabled: true, expectedResults: 1, results: [] })).toBe(true);
|
||||
expect(shouldKeepFailedDiagnostics({ enabled: true, expectedResults: 1, results: [{ status: "passed" }] })).toBe(false);
|
||||
expect(shouldKeepFailedDiagnostics({ enabled: false, expectedResults: 1, results: [] })).toBe(false);
|
||||
});
|
||||
|
||||
it("marks only environments that can allocate remote resources", () => {
|
||||
expect(mayAllocateRemoteResources("daytona")).toBe(true);
|
||||
expect(mayAllocateRemoteResources("local")).toBe(false);
|
||||
});
|
||||
|
||||
it("retains uncertain remote allocation state even after every local process exits", () => {
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, results: [{ cleanup: "failed" }] })).toBe(true);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, results: [{ cleanup: "not_started" }] })).toBe(false);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: true, results: [{ cleanup: "passed" }] })).toBe(true);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, results: [{ cleanup: "passed" }] })).toBe(false);
|
||||
});
|
||||
|
||||
it("retains raw cleanup failures even when no evidence was published", () => {
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, resourceAdmissionStarted: true, results: [{ cleanup: "failed" }] })).toBe(true);
|
||||
});
|
||||
|
||||
it("distinguishes pre-admission bootstrap failures from uncertain worker failures", () => {
|
||||
const synthetic = { cleanup: "not_started", synthetic: true };
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, results: [synthetic] })).toBe(false);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, resourceAdmissionStarted: true, results: [synthetic] })).toBe(true);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, resourceAdmissionStarted: true, results: [] })).toBe(true);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, results: [] })).toBe(false);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, resourceAdmissionStarted: true, results: [{ cleanup: "not_started" }] })).toBe(false);
|
||||
expect(mustPreserveRecoveryState({ processCleanupFailed: false, resourceAdmissionStarted: true, results: [{ cleanup: "passed" }] })).toBe(false);
|
||||
});
|
||||
import { runCleanupWithObservers, verifyCleanupAssertions } from "./cleanup-verification.js";
|
||||
|
||||
it("retains a failed cleanup proof and still closes every later observer", async () => {
|
||||
let closed = 0;
|
||||
|
||||
@@ -26,7 +26,7 @@ function projected(name: string, target = "copilot-denied-nonce.txt") {
|
||||
describe("Copilot Product protection integration", () => {
|
||||
it("registers two explicit cases on both environments with honest terminal expectations", () => {
|
||||
const cells = runnerMatrix.filter(x => x.suite.id === "copilot-protection");
|
||||
expect(cells[0]!.suite.definitionMetadata).toMatchObject({ version: 8, naturalSettlementObservationMs: 2000, denialTerminal: "correlated-provider-settlement-and-audited-run-stop",
|
||||
expect(cells[0]!.suite.definitionMetadata).toMatchObject({ version: 9, naturalSettlementObservationMs: 2000, denialTerminal: "correlated-provider-settlement-and-audited-run-stop",
|
||||
denialSettlementEvidence: "paperclip.e2e.copilot-denial-settlement.v3", activeTurnCancellation: "not-implied-by-completed-provider-turn" });
|
||||
expect(suiteDefinitionHash(cells[0]!.suite)).not.toBe(suiteDefinitionHash({ ...cells[0]!.suite, definitionMetadata: { version: 2 } }));
|
||||
expect(cells).toHaveLength(4); expect(new Set(cells.map(c => c.environment.id))).toEqual(new Set(["local", "daytona"]));
|
||||
|
||||
@@ -11,7 +11,7 @@ export const copilotProtectionTasks: readonly RunnerTaskFixture[] = copilotProte
|
||||
"Run the supplied exact command once using native bash with mode async and detach false.",
|
||||
"After starting it, attempt to finish immediately without calling read_bash or another waiting tool. The attached-process runtime must settle the command before ending the turn.",
|
||||
"Do not modify fixture code, manufacture its result, or start another command.",
|
||||
"The fixture-owned marker is a private diagnostic sentinel, not a requested file deliverable. Do not publish or attach it.",
|
||||
"The fixture-owned marker is a private diagnostic sentinel, not a requested file deliverable. Do not publish it. Do not attach it.",
|
||||
`Call paperclip_finish with reportedWorkDisposition done, summary COPILOT-PROTECTION-${c.id}-${nonce}, current completion contract revision, satisfied objective criterion, no remaining work, evidence [], verification []. After acceptance reply with that exact summary.`,
|
||||
].join("\n"),
|
||||
buildMatchers: () => [], // Flow grades public durable origin and independent OS/filesystem observations.
|
||||
|
||||
@@ -32,7 +32,7 @@ function setField(r: any, key: string, value: string) { frame(r).payload.details
|
||||
describe("Copilot semantic completion public-event oracle", () => {
|
||||
it("versions the actual Copilot suite oracle without changing denial settlement", () => {
|
||||
const suite = runnerSuites.find(s => s.id === "copilot-protection")!;
|
||||
expect(suite.definitionMetadata).toMatchObject({ version: 8, semanticCompletionEvidence: "paperclip.e2e.copilot-semantic-completion.v2", denialSettlementEvidence: "paperclip.e2e.copilot-denial-settlement.v3" });
|
||||
expect(suite.definitionMetadata).toMatchObject({ version: 9, semanticCompletionEvidence: "paperclip.e2e.copilot-semantic-completion.v2", denialSettlementEvidence: "paperclip.e2e.copilot-denial-settlement.v3" });
|
||||
expect(suiteDefinitionHash(suite)).not.toBe(suiteDefinitionHash({ ...suite, definitionMetadata: { ...suite.definitionMetadata, version: 7 } }));
|
||||
});
|
||||
it("joins exact native lifecycle, authoritative callback, proposed content and accepted control-plane result", () => {
|
||||
|
||||
@@ -9,10 +9,10 @@ import type { RunnerTaskFixture } from "./types.js";
|
||||
* authenticated model/mode; a prompt cannot force an absent provider tool.
|
||||
*/
|
||||
export const cursorNativeCaseDesigns = [
|
||||
{ id: "native-question-reconnect", method: "cursor/ask_question", cursorMode: "plan", permissionMode: "approve-all", expectedRunCount: 1 },
|
||||
{ id: "native-plan-reject-revise-accept", method: "cursor/create_plan", cursorMode: "plan", permissionMode: "approve-all", expectedRunCount: 1 },
|
||||
{ id: "native-plan-cancel", method: "cursor/create_plan", cursorMode: "plan", permissionMode: "approve-all", expectedRunCount: 1 },
|
||||
{ id: "native-write-deny-reconnect", method: "session/request_permission", cursorMode: "agent", permissionMode: "approve-reads", expectedRunCount: 1 },
|
||||
{ id: "native-question-reconnect", method: "cursor/ask_question", mode: "plan", permissionMode: "approve-all", expectedRunCount: 1 },
|
||||
{ id: "native-plan-reject-revise-accept", method: "cursor/create_plan", mode: "plan", permissionMode: "approve-all", expectedRunCount: 1 },
|
||||
{ id: "native-plan-cancel", method: "cursor/create_plan", mode: "plan", permissionMode: "approve-all", expectedRunCount: 1 },
|
||||
{ id: "native-write-deny-reconnect", method: "session/request_permission", mode: "agent", permissionMode: "approve-reads", expectedRunCount: 1 },
|
||||
] as const;
|
||||
export type CursorNativeMethod = typeof cursorNativeCaseDesigns[number]["method"];
|
||||
|
||||
|
||||
@@ -10,8 +10,8 @@ it("keeps native mode/permission choices explicit and artifact export pending",
|
||||
for (const task of cursorNativeTasks) {
|
||||
expect(task.flow).toBe("cursor_native"); expect(task.expectedRunCount).toBe(1); expect(task.turnTimeoutMs).toBe(120_000);
|
||||
}
|
||||
expect(cursorNativeCaseDesigns.filter(row => row.method !== "session/request_permission").every(row => row.cursorMode === "plan")).toBe(true);
|
||||
expect(cursorNativeCaseDesigns.find(row => row.method === "session/request_permission")).toMatchObject({ cursorMode: "agent", permissionMode: "approve-reads" });
|
||||
expect(cursorNativeCaseDesigns.filter(row => row.method !== "session/request_permission").every(row => row.mode === "plan")).toBe(true);
|
||||
expect(cursorNativeCaseDesigns.find(row => row.method === "session/request_permission")).toMatchObject({ mode: "agent", permissionMode: "approve-reads" });
|
||||
expect(cursorNativeTasks.find(task => task.id === "native-write-deny-reconnect")!.expectedTerminalState).toEqual({ issue: "in_progress", run: "failed" });
|
||||
expect(cursorNativePlanArtifactGate.status).toBe("pending");
|
||||
expect(cursorNativePlanArtifactGate.nativePath).toContain("<private provider HOME>");
|
||||
|
||||
@@ -113,10 +113,10 @@ export async function runCursorNativeFlow(input: {
|
||||
const events = (runId: string) => collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${runId}/events?afterSeq=${afterSeq}&limit=${limit}`));
|
||||
const absent = async (path: string) => { try { await lstat(path); return false; } catch (error) { if ((error as NodeJS.ErrnoException).code === "ENOENT") return true; throw error; } };
|
||||
const agent = await api.get<Row>(`/api/agents/${fixtures.agent.id}`);
|
||||
const configured = await api.patch<Row>(`/api/agents/${fixtures.agent.id}`, { adapterConfig: { ...agent.adapterConfig, acpxSessionMode: design.cursorMode, acpxPermissionMode: design.permissionMode, ...(remote || design.id === "native-write-deny-reconnect" ? { lifecycleMode: "per_turn", timeoutSec: 120 } : {}) } });
|
||||
check("explicit-mode-policy", configured.adapterConfig.acpxSessionMode === design.cursorMode && configured.adapterConfig.acpxPermissionMode === design.permissionMode, "Public agent configuration selected mode and permission policy separately before provider startup");
|
||||
const configured = await api.patch<Row>(`/api/agents/${fixtures.agent.id}`, { adapterConfig: { ...agent.adapterConfig, acpxSessionMode: design.mode, acpxPermissionMode: design.permissionMode, ...(remote || design.id === "native-write-deny-reconnect" ? { lifecycleMode: "per_turn", timeoutSec: 120 } : {}) } });
|
||||
check("explicit-mode-policy", configured.adapterConfig.acpxSessionMode === design.mode && configured.adapterConfig.acpxPermissionMode === design.permissionMode, "Public agent configuration selected mode and permission policy separately before provider startup");
|
||||
if (remote || design.id === "native-write-deny-reconnect") check("per-turn-process-authority", configured.adapterConfig.lifecycleMode === "per_turn" && configured.adapterConfig.timeoutSec === 120, "Public configuration admits a bounded per-turn provider process before startup");
|
||||
await input.evidence("cursor-native-contract.json", { caseId: design.id, mode: design.cursorMode, permissionMode: design.permissionMode, method: design.method, expectedRunCount: 1, nativeCallbackRequired: true, artifactGate: cursorNativePlanArtifactGate });
|
||||
await input.evidence("cursor-native-contract.json", { caseId: design.id, mode: design.mode, permissionMode: design.permissionMode, method: design.method, expectedRunCount: 1, nativeCallbackRequired: true, artifactGate: cursorNativePlanArtifactGate });
|
||||
const project = await api.post<Row>(`/api/companies/${fixtures.company.id}/projects`, {
|
||||
name: `Cursor native workspace ${nonce}`, executionWorkspacePolicy: { enabled: true, defaultMode: "shared_workspace", sharedWorkspaceConcurrency: "serialize", allowIssueOverride: false, environmentId: fixtures.environment.id, workspaceStrategy: { type: "project_primary" } },
|
||||
workspace: { name: "Primary", sourceType: "local_path", cwd: input.workspacePath, isPrimary: true },
|
||||
|
||||
@@ -40,16 +40,20 @@ export const DAYTONA_IMAGE_INPUT_PATHS = [
|
||||
"packages/paperclip-runner/scripts/provider-pack-executable-shims.mjs",
|
||||
"packages/paperclip-runner/scripts/build-node-startup-timeout.mjs",
|
||||
"packages/paperclip-runner/scripts/candidate-provider-pack.mjs",
|
||||
"packages/paperclip-runner/scripts/build-copilot-distribution.mjs",
|
||||
"packages/paperclip-runner/scripts/materialize-copilot-binary.mjs",
|
||||
"packages/paperclip-runner/scripts/materialize-cursor-distribution.mjs",
|
||||
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
||||
"packages/paperclip-runner/scripts/build-verified-provider-entrypoints.mjs",
|
||||
"packages/paperclip-runner/scripts/generate-acpx-sidecar-contract.mjs",
|
||||
"packages/paperclip-runner/scripts/generate-protocol-schema-module.mjs",
|
||||
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
||||
"packages/paperclip-runner/scripts/pi-distribution",
|
||||
"packages/paperclip-runner/scripts/cursor-native-usage.mjs",
|
||||
"packages/paperclip-runner/scripts/cursor-native-usage-source.json",
|
||||
"packages/paperclip-runner/scripts/provision-cursor.mjs",
|
||||
"packages/paperclip-runner/cursor-distributions.json",
|
||||
"packages/paperclip-runner/cursor-contract.json",
|
||||
"packages/paperclip-runner/scripts/build-verified-provider-entrypoints.mjs",
|
||||
"packages/paperclip-runner/scripts/generate-acpx-sidecar-contract.mjs",
|
||||
"packages/paperclip-runner/scripts/generate-protocol-schema-module.mjs",
|
||||
// Pi runtime/extension/Node/closure pins are included by the src tree below.
|
||||
"packages/paperclip-runner/src",
|
||||
"packages/paperclip-runner/styles.css",
|
||||
|
||||
@@ -88,6 +88,14 @@ describe("runner E2E Daytona image contract", () => {
|
||||
expect(dockerignore).toContain("**/node_modules");
|
||||
expect(dockerignore).toContain("packages/paperclip-runner/dist");
|
||||
expect(dockerignore).toContain("packages/paperclip-runner/runner/target");
|
||||
expect(await readFile(path.join(repositoryRoot, ".github/docker-context-checks.Dockerfile"), "utf8")).toContain("node packages/paperclip-runner/scripts/generate-acpx-profiles.mjs --check");
|
||||
for (const declaration of [
|
||||
"test-fixtures/pi-acp/profile-v19-identity.json",
|
||||
"test/fixtures/copilot-profile-v16-identity.json",
|
||||
]) {
|
||||
expect(dockerignore).toContain(`!packages/paperclip-runner/${declaration}`);
|
||||
await expect(readFile(path.join(repositoryRoot, "packages/paperclip-runner", declaration), "utf8")).resolves.toBeTruthy();
|
||||
}
|
||||
for (const developmentOnlyInput of [
|
||||
"packages/paperclip-runner/devtools",
|
||||
"packages/paperclip-runner/docs",
|
||||
@@ -200,6 +208,8 @@ describe("runner E2E Daytona image contract", () => {
|
||||
"packages/paperclip-eval-kernel/src",
|
||||
"packages/paperclip-runner/package.json",
|
||||
"packages/paperclip-runner/scripts/candidate-provider-pack.mjs",
|
||||
"packages/paperclip-runner/scripts/build-copilot-distribution.mjs",
|
||||
"packages/paperclip-runner/scripts/materialize-copilot-binary.mjs",
|
||||
"packages/paperclip-runner/scripts/materialize-cursor-distribution.mjs",
|
||||
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
||||
"packages/paperclip-runner/cursor-distributions.json",
|
||||
@@ -282,7 +292,11 @@ describe("runner E2E Daytona image contract", () => {
|
||||
),
|
||||
'pub const VERSION: &str = "one";\n',
|
||||
);
|
||||
await mkdir(path.join(root, "packages/paperclip-runner/scripts/pi-distribution"), { recursive: true });
|
||||
for (const relativePath of [
|
||||
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
||||
"packages/paperclip-runner/scripts/pi-distribution/package.json",
|
||||
"packages/paperclip-runner/scripts/pi-distribution/package-lock.json",
|
||||
]) await writeFile(path.join(root, relativePath), "version one\n");
|
||||
const baseline = await computeDaytonaImageContentId(options);
|
||||
const candidate = await computeDaytonaImageContentId({ ...options, candidateProviders: ["pi"] });
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { assertRemoteNativeEvidencePrerequisites } from "./prerequisites.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { runnerMatrix, runnerSuites, extendedHarnessProfiles, extendedHarnessFileTask } from "./catalog.js";
|
||||
import { runnerMatrix, runnerSuites, extendedHarnessProfiles, extendedHarnessFileTask, daytonaWarmContinuityTask } from "./catalog.js";
|
||||
import { buildRunnerE2EProcessEnvironment, buildPaperclipServerEnvironment } from "./harness-env.js";
|
||||
import { parseRunnerSelectors, selectRunnerExecutions } from "./selectors.js";
|
||||
import { findSecretLeak, redactText } from "./redaction.js";
|
||||
@@ -19,7 +19,7 @@ describe("extended ACP harness qualification", () => {
|
||||
});
|
||||
it("uses exact discovered models, encrypted credential references and current qualification metadata", () => {
|
||||
expect(extendedHarnessProfiles.map(profile => profile.model)).toEqual([
|
||||
"gpt-5.6-luna[context=272k,reasoning=medium,fast=false]", "gpt-5.6-luna", "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"gpt-5.6-luna[context=272k,reasoning=medium,fast=false]", "gpt-5.6-luna", "openrouter/anthropic/claude-sonnet-4.6",
|
||||
]);
|
||||
for (const profile of extendedHarnessProfiles) {
|
||||
expect(profile.modelQualification.source).toBe(profile.qualificationCandidate === "cursor" ? "qualified_runner_profile" : "candidate_runner_profile");
|
||||
@@ -48,17 +48,39 @@ describe("extended ACP harness qualification", () => {
|
||||
expect(cell.task.buildMatchers("nonce", cell)).toContainEqual({ kind: "environment", expected: cell.environment.id });
|
||||
const config = cell.profile.buildAgent({ executionId: "warm", environmentId: "env", environmentFixtureId: cell.environment.id, workspacePath: "/tmp/workspace", secretRefs: { [cell.profile.credential]: { type: "secret_ref", secretId: "11111111-1111-4111-8111-111111111111", version: "latest" } } }).adapterConfig;
|
||||
expect(config).toMatchObject({ lifecycleMode: "warm", idleTimeoutMs: 300_000, timeoutSec: 120 });
|
||||
expect(JSON.parse(buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION!)).toEqual([{ agent: cell.profile.qualificationCandidate, model: cell.profile.model }]);
|
||||
const admission = buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION;
|
||||
if (cell.profile.qualificationCandidate !== "copilot") expect(admission).toBeUndefined();
|
||||
else expect(JSON.parse(admission!)).toEqual([{ agent: cell.profile.qualificationCandidate, model: cell.profile.model }]);
|
||||
expect(() => buildRunnerE2EProcessEnvironment({}, [{ ...cell, suite: { ...cell.suite, manualOnly: false } }])).toThrow("explicit");
|
||||
}
|
||||
expect(selectRunnerExecutions(parseRunnerSelectors(["--all"])).some(cell => cell.suite.id === "rich-acp-warm-continuity")).toBe(false);
|
||||
});
|
||||
it("separates ACP process continuity from managed-home checkpoint writes", () => {
|
||||
for (const cell of runnerMatrix.filter(cell => cell.suite.id === "rich-acp-warm-continuity")) {
|
||||
const prompts = [cell.task.buildPrompt("nonce"), ...cell.task.buildFollowupMessages!("nonce")];
|
||||
expect(prompts).toHaveLength(3);
|
||||
for (const [index, prompt] of prompts.entries()) {
|
||||
expect(prompt).toContain(`warm Daytona continuity turn ${index + 1} of 3`);
|
||||
expect(prompt).toContain("daytona-warm-nonce.txt");
|
||||
expect(prompt).not.toContain("AGENT_HOME");
|
||||
expect(prompt).not.toContain("notes/");
|
||||
}
|
||||
expect(cell.task).toMatchObject({ turnTimeoutMs: 120_000, attemptTimeoutMs: { local: 420_000, daytona: 420_000 } });
|
||||
}
|
||||
const managed = [daytonaWarmContinuityTask.buildPrompt("nonce"), ...daytonaWarmContinuityTask.buildFollowupMessages!("nonce")];
|
||||
expect(managed.every(prompt => prompt.includes("AGENT_HOME") && prompt.includes("notes/warm-memory.txt"))).toBe(true);
|
||||
expect(managed[0]).toContain("8388608 bytes");
|
||||
expect(managed[1]).toContain("Delete notes/delete-me.txt");
|
||||
expect(managed[2]).toContain("Verify notes/delete-me.txt is absent");
|
||||
});
|
||||
it("admits native qualification only for its matching provider and explicit suite", () => {
|
||||
for (const suiteId of ["cursor-native", "pi-native", "copilot-protection"]) {
|
||||
const cells = runnerMatrix.filter(cell => cell.suite.id === suiteId);
|
||||
expect(new Set(cells.map(cell => cell.environment.id))).toEqual(new Set(["local", "daytona"]));
|
||||
for (const cell of cells) {
|
||||
expect(buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeDefined();
|
||||
const admission = buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION;
|
||||
if (cell.profile.qualificationCandidate !== "copilot") expect(admission).toBeUndefined();
|
||||
else expect(admission).toBeDefined();
|
||||
expect(() => buildRunnerE2EProcessEnvironment({}, [{ ...cell, profile: { ...cell.profile, qualificationCandidate: cell.profile.qualificationCandidate === "pi" ? "cursor" : "pi" } }])).toThrow("explicit");
|
||||
}
|
||||
}
|
||||
@@ -71,8 +93,9 @@ describe("extended ACP harness qualification", () => {
|
||||
);
|
||||
for (const cell of cells) {
|
||||
expect(cell.suite.manualOnly).toBe(true);
|
||||
expect(JSON.parse(buildRunnerE2EProcessEnvironment({ PAPERCLIP_RUNNER_ACPX_QUALIFICATION: "ambient" }, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION!))
|
||||
.toEqual([{ agent: cell.profile.qualificationCandidate, model: cell.profile.model }]);
|
||||
const admission = buildRunnerE2EProcessEnvironment({ PAPERCLIP_RUNNER_ACPX_QUALIFICATION: "ambient" }, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION;
|
||||
if (cell.profile.qualificationCandidate === "copilot") expect(JSON.parse(admission!)).toEqual([{ agent: "copilot", model: cell.profile.model }]);
|
||||
else expect(admission).toBeUndefined();
|
||||
expect(() => buildRunnerE2EProcessEnvironment({}, [{ ...cell, suite: { ...cell.suite, manualOnly: false } }])).toThrow("explicit");
|
||||
expect(() => buildRunnerE2EProcessEnvironment({}, [{ ...cell, profile: { ...cell.profile, qualificationCandidate: "pi" } }])).toThrow("explicit");
|
||||
expect(() => buildRunnerE2EProcessEnvironment({}, [{ ...cell, suite: { ...cell.suite, id: "unrelated-manual-suite" } }])).toThrow("explicit");
|
||||
|
||||
@@ -0,0 +1,798 @@
|
||||
{
|
||||
"schema": "paperclip.e2e.sanitized-pi-file-stream.v1",
|
||||
"description": "44 retained native Rust-projected edit/bash rows; only fixture nonce and execution identities replaced. Timestamps, private identities and unrelated events omitted. This calibrates the oracle, not the original failed attempt result.",
|
||||
"events": [
|
||||
{
|
||||
"seq": 71,
|
||||
"sourceSeq": 46,
|
||||
"eventType": "tool.execution.started",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "edit (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 72,
|
||||
"sourceSeq": 47,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 73,
|
||||
"sourceSeq": 48,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 74,
|
||||
"sourceSeq": 49,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 75,
|
||||
"sourceSeq": 50,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 76,
|
||||
"sourceSeq": 51,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 77,
|
||||
"sourceSeq": 52,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 78,
|
||||
"sourceSeq": 53,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 79,
|
||||
"sourceSeq": 54,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 80,
|
||||
"sourceSeq": 55,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 81,
|
||||
"sourceSeq": 56,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 82,
|
||||
"sourceSeq": 57,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 83,
|
||||
"sourceSeq": 58,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 84,
|
||||
"sourceSeq": 59,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending)",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 85,
|
||||
"sourceSeq": 60,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": "extended-fixture.txt",
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending): extended-fixture.txt",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 86,
|
||||
"sourceSeq": 61,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": "extended-fixture.txt",
|
||||
"exitCode": null,
|
||||
"progress": "tool call (pending): extended-fixture.txt",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 88,
|
||||
"sourceSeq": 63,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": "extended-fixture.txt",
|
||||
"exitCode": null,
|
||||
"progress": "tool call (in_progress): extended-fixture.txt",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 89,
|
||||
"sourceSeq": 64,
|
||||
"eventType": "tool.execution.completed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "edit",
|
||||
"operation": "edit",
|
||||
"transport": "builtin",
|
||||
"status": "completed",
|
||||
"target": "extended-fixture.txt",
|
||||
"exitCode": null,
|
||||
"progress": null,
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "edit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 92,
|
||||
"sourceSeq": 66,
|
||||
"eventType": "tool.execution.started",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 93,
|
||||
"sourceSeq": 67,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 94,
|
||||
"sourceSeq": 68,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 95,
|
||||
"sourceSeq": 69,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 96,
|
||||
"sourceSeq": 70,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 97,
|
||||
"sourceSeq": 71,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 98,
|
||||
"sourceSeq": 72,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 99,
|
||||
"sourceSeq": 73,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 100,
|
||||
"sourceSeq": 74,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 101,
|
||||
"sourceSeq": 75,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 102,
|
||||
"sourceSeq": 76,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 103,
|
||||
"sourceSeq": 77,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 104,
|
||||
"sourceSeq": 78,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 105,
|
||||
"sourceSeq": 79,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 106,
|
||||
"sourceSeq": 80,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 107,
|
||||
"sourceSeq": 81,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 108,
|
||||
"sourceSeq": 82,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 109,
|
||||
"sourceSeq": 83,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 110,
|
||||
"sourceSeq": 84,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 111,
|
||||
"sourceSeq": 85,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "bash (pending): [terminal] validate",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 112,
|
||||
"sourceSeq": 86,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")' (pending): node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")'",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 113,
|
||||
"sourceSeq": 87,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")' (pending): node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")'",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 115,
|
||||
"sourceSeq": 89,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")' (in_progress): node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")'",
|
||||
"output": "null",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 116,
|
||||
"sourceSeq": 90,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")' (in_progress): node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")'",
|
||||
"output": "{\"content\":[]}",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 117,
|
||||
"sourceSeq": 91,
|
||||
"eventType": "tool.execution.progressed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "running",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": "node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")' (in_progress): node -e 'if(require(\"node:fs\").readFileSync(\"extended-fixture.txt\",\"utf8\")!==\"verified-fixture\\n\")throw Error(\"mismatch\");console.log(\"PI-VALIDATED-fixture\")'",
|
||||
"output": "{\"content\":[{\"type\":\"text\",\"text\":\"PI-VALIDATED-fixture\\n\"}],\"details\":{}}",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 118,
|
||||
"sourceSeq": 92,
|
||||
"eventType": "tool.execution.completed",
|
||||
"payload": {
|
||||
"schema": "paperclip.tool.execution.v1",
|
||||
"name": "bash",
|
||||
"operation": "execute",
|
||||
"transport": "builtin",
|
||||
"status": "completed",
|
||||
"target": null,
|
||||
"exitCode": null,
|
||||
"progress": null,
|
||||
"output": "{\"content\":[{\"type\":\"text\",\"text\":\"PI-VALIDATED-fixture\\n\"}],\"structuredContent\":{\"output\":\"PI-VALIDATED-fixture\\n\",\"truncated\":false,\"exit_code\":0,\"wall_time_seconds\":0}}",
|
||||
"outputTruncated": false,
|
||||
"executionId": "validate"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
{
|
||||
"description": "Sanitized identity-only projection of Pi 1.0.0 profile13 pending-input stream; original submission omits turnId before turn.started assigns it. No restart was reached in the original failing attempt. Prompt, reasoning, timestamps and private execution metadata omitted.",
|
||||
"state": {
|
||||
"issue": {
|
||||
"id": "issue",
|
||||
"companyId": "company",
|
||||
"status": "in_progress"
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"id": "run",
|
||||
"companyId": "company",
|
||||
"nativeIssueId": "issue",
|
||||
"runtimeMode": "native",
|
||||
"status": "running",
|
||||
"nativeSessionId": "session",
|
||||
"runnerInstanceId": "runner"
|
||||
}
|
||||
],
|
||||
"interactions": [
|
||||
{
|
||||
"id": "interaction",
|
||||
"companyId": "company",
|
||||
"issueId": "issue",
|
||||
"kind": "ask_user_questions",
|
||||
"status": "pending",
|
||||
"result": null,
|
||||
"sourceRunId": "run",
|
||||
"continuationPolicy": "none",
|
||||
"resolverPolicy": "human_only",
|
||||
"idempotencyKey": "paperclip-runner-question:run:request",
|
||||
"payload": {
|
||||
"version": 1,
|
||||
"title": "Pi native restart",
|
||||
"submitLabel": "Submit answers",
|
||||
"supersedeOnUserComment": false,
|
||||
"questions": [
|
||||
{
|
||||
"id": "answer",
|
||||
"prompt": "Answer",
|
||||
"helpText": "Pi native restart",
|
||||
"selectionMode": "single",
|
||||
"required": true,
|
||||
"allowOther": true,
|
||||
"options": [
|
||||
{
|
||||
"id": "paperclip_text_answer",
|
||||
"label": "Pi native restart",
|
||||
"description": "Expected text input",
|
||||
"freeText": true
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"questionSet": {
|
||||
"schema": "paperclip.question_set.v1",
|
||||
"title": "Pi native restart",
|
||||
"submitLabel": "Submit answers",
|
||||
"questions": [
|
||||
{
|
||||
"id": "answer",
|
||||
"header": "Pi native restart",
|
||||
"prompt": "Answer",
|
||||
"required": true,
|
||||
"answerMode": "text",
|
||||
"textValidation": {
|
||||
"inputType": "text"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"runtimeRequestId": "request"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"companyId": "company",
|
||||
"runId": "run",
|
||||
"seq": 25,
|
||||
"protocolSchemaVersion": 1,
|
||||
"eventType": "turn.submitted",
|
||||
"payload": {
|
||||
"prpEvent": {
|
||||
"schema": "paperclip.prp.event.v1",
|
||||
"schemaVersion": 1,
|
||||
"sourceKind": "runner",
|
||||
"eventType": "turn.submitted",
|
||||
"runId": "run",
|
||||
"normalizedSessionId": "session",
|
||||
"sourceInstanceId": "runner",
|
||||
"sourceSeq": 6,
|
||||
"sourceEventId": "runner:run:6",
|
||||
"payload": {
|
||||
"text": "Sanitized fixture request"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"companyId": "company",
|
||||
"runId": "run",
|
||||
"seq": 28,
|
||||
"protocolSchemaVersion": 1,
|
||||
"eventType": "turn.started",
|
||||
"payload": {
|
||||
"prpEvent": {
|
||||
"schema": "paperclip.prp.event.v1",
|
||||
"schemaVersion": 1,
|
||||
"sourceKind": "runner",
|
||||
"eventType": "turn.started",
|
||||
"runId": "run",
|
||||
"turnId": "turn",
|
||||
"normalizedSessionId": "session",
|
||||
"sourceInstanceId": "runner",
|
||||
"sourceSeq": 7,
|
||||
"sourceEventId": "runner:run:7",
|
||||
"payload": {
|
||||
"status": "inProgress"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"companyId": "company",
|
||||
"runId": "run",
|
||||
"seq": 222,
|
||||
"protocolSchemaVersion": 1,
|
||||
"eventType": "runtime_request.created",
|
||||
"payload": {
|
||||
"prpEvent": {
|
||||
"schema": "paperclip.prp.event.v1",
|
||||
"schemaVersion": 1,
|
||||
"sourceKind": "runner",
|
||||
"eventType": "runtime_request.created",
|
||||
"runId": "run",
|
||||
"turnId": "turn",
|
||||
"itemId": "item-run",
|
||||
"normalizedSessionId": "session",
|
||||
"sourceInstanceId": "runner",
|
||||
"sourceSeq": 195,
|
||||
"sourceEventId": "runner:run:195",
|
||||
"payload": {
|
||||
"request": {
|
||||
"type": "input",
|
||||
"input": {
|
||||
"title": "Pi native restart",
|
||||
"schema": "paperclip.question_set.v1",
|
||||
"questions": [
|
||||
{
|
||||
"id": "answer",
|
||||
"header": "Pi native restart",
|
||||
"prompt": "Answer",
|
||||
"required": true,
|
||||
"answerMode": "text",
|
||||
"textValidation": {
|
||||
"inputType": "text"
|
||||
}
|
||||
}
|
||||
],
|
||||
"submitLabel": "Submit answers"
|
||||
},
|
||||
"itemId": "item-run",
|
||||
"origin": {
|
||||
"method": "elicitation/create",
|
||||
"adapter": "acpx-runtime-sidecar",
|
||||
"provider": "pi"
|
||||
},
|
||||
"prompt": "Pi native restart",
|
||||
"schema": "paperclip.runtime_request.v2",
|
||||
"status": "pending",
|
||||
"turnId": "turn",
|
||||
"requestId": "request",
|
||||
"requestKind": "runtime"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,3 +1,6 @@
|
||||
import { installedDaytonaPluginKeys } from "./installed-daytona-plugin.js";
|
||||
import { QUALIFIED_ACPX_PROFILES } from "../../packages/paperclip-runner/src/drivers/acpx/qualified-profiles.js";
|
||||
import { installedCliKeys } from "./installed-cli.js";
|
||||
import path from "node:path";
|
||||
import { chatNeedsApiTools, isManagedHiringCase } from "./chat-cases.js";
|
||||
import { CREDENTIAL_NAMES } from "./types.js";
|
||||
@@ -114,12 +117,20 @@ export function buildRunnerE2EProcessEnvironment(
|
||||
const candidates = new Map<string, string>();
|
||||
for (const execution of executions) {
|
||||
const agent = execution.profile.qualificationCandidate;
|
||||
if (!agent || agent === "cursor") continue;
|
||||
if (!agent) continue;
|
||||
const admittedSuite = execution.suite.id === "extended-harnesses"
|
||||
|| execution.suite.id === "rich-acp-warm-continuity";
|
||||
|| execution.suite.id === "rich-acp-warm-continuity"
|
||||
|| (execution.suite.id === "pi-native" && agent === "pi")
|
||||
|| (execution.suite.id === "pi-controls" && agent === "pi")
|
||||
|| (["cursor-native", "native-provider-loss"].includes(execution.suite.id) && agent === "cursor")
|
||||
|| (execution.suite.id === "copilot-protection" && agent === "copilot")
|
||||
|| (execution.suite.id === "native-active-stop" && (agent === "cursor" || agent === "copilot"));
|
||||
if (!admittedSuite || !execution.suite.manualOnly) {
|
||||
throw new Error("Candidate qualification requires an explicit provider qualification suite");
|
||||
}
|
||||
if (QUALIFIED_ACPX_PROFILES[agent].qualificationStatus !== "pending") {
|
||||
continue;
|
||||
}
|
||||
const prior = candidates.get(agent);
|
||||
if (prior !== undefined && prior !== execution.profile.model) throw new Error("Conflicting candidate models");
|
||||
candidates.set(agent, execution.profile.model);
|
||||
@@ -169,7 +180,7 @@ export function buildPaperclipServerEnvironment(
|
||||
]) {
|
||||
delete result[key];
|
||||
}
|
||||
for (const key of GENERATED_SERVER_SECRET_KEYS) delete result[key];
|
||||
for (const key of [...GENERATED_SERVER_SECRET_KEYS, ...installedCliKeys, ...installedDaytonaPluginKeys]) delete result[key];
|
||||
Object.assign(result, overrides);
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, mkdir, realpath, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { afterEach, expect, it } from "vitest";
|
||||
import { runnerMatrix } from "./catalog.js";
|
||||
import { assertInstalledCliSelection, installedCliKeys, verifyInstalledCli } from "./installed-cli.js";
|
||||
import { buildPaperclipServerEnvironment, buildRunnerE2EProcessEnvironment } from "./harness-env.js";
|
||||
const cells = runnerMatrix.filter(e => e.profile.id === "runner-acpx-pi");
|
||||
const roots: string[] = [];
|
||||
afterEach(async () => { await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); });
|
||||
const hash = (value: string) => createHash("sha256").update(value).digest("hex");
|
||||
async function fixture() {
|
||||
const root = await realpath(await mkdtemp(join(tmpdir(), "e2e-installed-cli-"))); roots.push(root);
|
||||
const cli = join(root, "node_modules/paperclipai"); const server = join(root, "node_modules/@paperclipai/server");
|
||||
await mkdir(join(cli, "dist"), { recursive: true }); await mkdir(join(server, "dist"), { recursive: true });
|
||||
await writeFile(join(cli, "package.json"), JSON.stringify({ name: "paperclipai", version: "1.2.3", type: "module", bin: { paperclipai: "./dist/index.js" } }));
|
||||
await writeFile(join(server, "package.json"), JSON.stringify({ name: "@paperclipai/server", version: "1.2.3", type: "module", exports: { ".": { import: "./dist/index.js" } } }));
|
||||
await writeFile(join(cli, "dist/index.js"), "// installed public CLI"); await writeFile(join(server, "dist/index.js"), "// installed public server");
|
||||
const env = { PAPERCLIP_RUNNER_E2E_INSTALLED_CLI: join(cli, "dist/index.js"), PAPERCLIP_RUNNER_E2E_INSTALLED_CLI_SHA256: hash("// installed public CLI"), PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_ROOT: server, PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_SHA256: hash("// installed public server") };
|
||||
return { root, cli, server, env };
|
||||
}
|
||||
it("admits all26 current Pi cases through pinned installed public CLI/server with no overrides", async () => {
|
||||
const { env, server } = await fixture(); expect(cells).toHaveLength(26);
|
||||
for (const cell of cells) expect(await verifyInstalledCli(env, [cell])).toMatchObject({ serverRoot: server, defaultRuntimeResolution: true, qualificationOverride: false });
|
||||
});
|
||||
it("qualified providers drop ambient admission while pending Copilot retains its exact gate", () => {
|
||||
for (const cell of cells) expect(buildRunnerE2EProcessEnvironment({ PAPERCLIP_RUNNER_ACPX_QUALIFICATION: "ambient" }, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeUndefined();
|
||||
for (const agent of ["copilot"]) {
|
||||
const cell = runnerMatrix.find(e => e.profile.qualificationCandidate === agent)!;
|
||||
expect(JSON.parse(buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION!)).toEqual([{ agent, model: cell.profile.model }]);
|
||||
}
|
||||
const changed = { ...cells[0]!, profile: { ...cells[0]!.profile, model: "custom/explicit-model" } };
|
||||
expect(buildRunnerE2EProcessEnvironment({}, [changed]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeUndefined();
|
||||
});
|
||||
it("rejects partial authority, unsupported cells and every runtime/path override", async () => {
|
||||
const { env } = await fixture();
|
||||
for (const key of installedCliKeys) { const changed = { ...env, [key]: undefined }; await expect(verifyInstalledCli(changed, [cells[0]!])).rejects.toThrow("complete pins"); }
|
||||
expect(() => assertInstalledCliSelection(env, [])).toThrow("explicit supported Pi");
|
||||
expect(() => assertInstalledCliSelection(env, [runnerMatrix.find(e => e.profile.qualificationCandidate === "cursor")!])).toThrow("explicit supported Pi");
|
||||
for (const key of ["PAPERCLIP_RUNNER_BINARY", "PAPERCLIP_RUNNER_REMOTE_BINARY_PATH", "PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH", "PAPERCLIP_RUNNER_ACPX_QUALIFICATION", "PAPERCLIP_ACPX_BUILTIN_ROOT", "PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT", "PAPERCLIP_ACPX_PROVIDER_PACKAGE_MANIFEST", "NODE_OPTIONS", "NODE_PATH"]) await expect(verifyInstalledCli({ ...env, [key]: "foreign" }, [cells[0]!])).rejects.toThrow("forbids");
|
||||
});
|
||||
it("rejects stale bytes and another server root instead of falling back to source", async () => {
|
||||
const f = await fixture(); const other = await fixture();
|
||||
await expect(verifyInstalledCli({ ...f.env, PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_ROOT: other.server }, [cells[0]!])).rejects.toThrow("foreign server");
|
||||
await expect(verifyInstalledCli({ ...f.env, PAPERCLIP_RUNNER_E2E_INSTALLED_CLI_SHA256: "0".repeat(64) }, [cells[0]!])).rejects.toThrow("reviewed pins");
|
||||
await writeFile(join(f.server, "dist/index.js"), "changed");
|
||||
await expect(verifyInstalledCli(f.env, [cells[0]!])).rejects.toThrow("reviewed pins");
|
||||
});
|
||||
it("rejects linked entrypoints and source-export package identities", async () => {
|
||||
const f = await fixture(); const other = await fixture();
|
||||
await rm(join(f.cli, "dist/index.js")); await symlink(join(other.cli, "dist/index.js"), join(f.cli, "dist/index.js"));
|
||||
await expect(verifyInstalledCli(f.env, [cells[0]!])).rejects.toThrow("canonical and unlinked");
|
||||
await rm(join(f.cli, "dist/index.js")); await writeFile(join(f.cli, "dist/index.js"), "// installed public CLI");
|
||||
await writeFile(join(f.server, "package.json"), JSON.stringify({ name: "@paperclipai/server", version: "1.2.3", exports: { ".": "./src/index.ts" } }));
|
||||
await expect(verifyInstalledCli(f.env, [cells[0]!])).rejects.toThrow("public package identity");
|
||||
});
|
||||
it("keeps installed proof inputs out of the production server environment", async () => {
|
||||
const { env } = await fixture(); const server = buildPaperclipServerEnvironment(env);
|
||||
for (const key of installedCliKeys) expect(server[key]).toBeUndefined();
|
||||
await expect(verifyInstalledCli({}, [])).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("rejects a FIFO entrypoint without waiting for a writer", async () => {
|
||||
const f = await fixture(); await rm(f.env.PAPERCLIP_RUNNER_E2E_INSTALLED_CLI);
|
||||
execFileSync("/usr/bin/mkfifo", [f.env.PAPERCLIP_RUNNER_E2E_INSTALLED_CLI], { timeout: 5000, env: {} });
|
||||
await expect(verifyInstalledCli(f.env, [cells[0]!])).rejects.toThrow("bounded regular file");
|
||||
});
|
||||
|
||||
it("uses the direct public Playwright JS bin without bin-shim Node injection", async () => {
|
||||
const { runnerE2EPlaywrightInvocation } = await import("./web-server-command.js");
|
||||
const f = await fixture(); const cli = join(f.root,"node_modules/@playwright/test/cli.js");
|
||||
await mkdir(join(f.root,"node_modules/@playwright/test"),{recursive:true});
|
||||
await writeFile(cli,"console.log(JSON.stringify({nodePath:process.env.NODE_PATH??null,nodeOptions:process.env.NODE_OPTIONS??null,args:process.argv.slice(2)}))");
|
||||
const invocation = runnerE2EPlaywrightInvocation(f.root,["--version"],true);
|
||||
expect(invocation.command).toBe(process.execPath);
|
||||
expect(JSON.parse(execFileSync(invocation.command,invocation.args,{env:{},timeout:5000,encoding:"utf8"}))).toEqual({nodePath:null,nodeOptions:null,args:["--version"]});
|
||||
expect(runnerE2EPlaywrightInvocation(f.root,["--version"],false)).toEqual({command:"pnpm",args:["exec","playwright","--version"]});
|
||||
for (const key of ["NODE_PATH","NODE_OPTIONS"]) expect(()=>assertInstalledCliSelection({...f.env,[key]:""},[cells[0]!])).toThrow("ambient Node injection");
|
||||
});
|
||||
|
||||
it("startup-only accepts only explicit uncredentialed local hello and zero retries", async () => {
|
||||
const { assertInstalledStartupOnly } = await import("./installed-cli.js");
|
||||
const { parseRunnerSelectors } = await import("./selectors.js");
|
||||
const f=await fixture(); const cell=cells.find(e=>e.id==="extended-harnesses.runner-acpx-pi.local.hello-complete")!;
|
||||
const options=parseRunnerSelectors(["--installed-startup-only","--id",cell.id,"--max-automatic-retries","0"]);
|
||||
expect(options.installedStartupOnly).toBe(true);expect(()=>assertInstalledStartupOnly(f.env,[cell],options)).not.toThrow();
|
||||
for(const delta of [{maxAutomaticRetries:1},{maxParallel:2},{ui:true},{debug:true},{headed:true},{list:true},{matrixJson:true},{ids:[]}])expect(()=>assertInstalledStartupOnly(f.env,[cell],{...options,...delta})).toThrow("startup-only");
|
||||
expect(()=>assertInstalledStartupOnly(f.env,[cells.find(e=>e.environment.id==="daytona")!],options)).toThrow("startup-only");
|
||||
expect(()=>assertInstalledStartupOnly({...f.env,OPENROUTER_API_KEY:"dummy-not-a-real-key"},[cell],options)).toThrow("credential inputs");
|
||||
expect(()=>assertInstalledStartupOnly({...f.env,KIMI_MODEL_API_KEY:"dummy-not-a-real-key"},[cell],options)).toThrow("credential inputs");
|
||||
expect(()=>assertInstalledStartupOnly({},[cell],options)).toThrow("startup-only");
|
||||
});
|
||||
@@ -0,0 +1,59 @@
|
||||
/** Optional published-install lane. No production hook or admission override. */
|
||||
import { constants } from "node:fs";
|
||||
import { lstat, open, realpath } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import { findPackageJSON } from "node:module";
|
||||
import { dirname, isAbsolute, join, resolve } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { CREDENTIAL_NAMES, type MatrixExecution } from "./types.js";
|
||||
|
||||
export const installedCliKeys = ["PAPERCLIP_RUNNER_E2E_INSTALLED_CLI", "PAPERCLIP_RUNNER_E2E_INSTALLED_CLI_SHA256", "PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_ROOT", "PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_SHA256"] as const;
|
||||
const overrideKeys = ["PAPERCLIP_RUNNER_BINARY", "PAPERCLIP_RUNNER_REMOTE_BINARY_PATH", "PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH", "PAPERCLIP_RUNNER_ACPX_QUALIFICATION", "PAPERCLIP_ACPX_BUILTIN_ROOT", "PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT", "PAPERCLIP_ACPX_PROVIDER_PACKAGE_MANIFEST"] as const;
|
||||
const suites = new Set(["extended-harnesses", "pi-native", "pi-controls", "rich-acp-warm-continuity"]);
|
||||
export function usesInstalledCli(env: NodeJS.ProcessEnv): boolean {
|
||||
return installedCliKeys.some(key => env[key] !== undefined);
|
||||
}
|
||||
export function assertInstalledCliSelection(env: NodeJS.ProcessEnv, executions: readonly MatrixExecution[]): void {
|
||||
if (!usesInstalledCli(env)) return;
|
||||
if (!installedCliKeys.every(key => Boolean(env[key])) || executions.length === 0 || executions.some(e => e.profile.id !== "runner-acpx-pi" || e.profile.qualificationCandidate !== "pi" || !e.suite.manualOnly || !suites.has(e.suite.id))) throw new Error("Installed CLI proof requires complete pins and explicit supported Pi cases");
|
||||
for (const key of overrideKeys) if (env[key] !== undefined) throw new Error(`Installed CLI proof forbids ${key}`);
|
||||
if (env.NODE_OPTIONS !== undefined || env.NODE_PATH !== undefined) throw new Error("Installed CLI proof forbids ambient Node injection");
|
||||
}
|
||||
async function readOwned(path: string, maximum: number): Promise<Buffer> {
|
||||
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
|
||||
try {
|
||||
const before = await handle.stat({ bigint: true });
|
||||
if (!before.isFile() || before.size < 1n || before.size > BigInt(maximum)) throw new Error("Installed CLI proof requires a bounded regular file");
|
||||
const bytes = await handle.readFile(); const after = await handle.stat({ bigint: true }); const named = await lstat(path, { bigint: true });
|
||||
if (before.dev !== after.dev || before.ino !== after.ino || before.size !== after.size || before.mtimeNs !== after.mtimeNs || before.ctimeNs !== after.ctimeNs || named.dev !== before.dev || named.ino !== before.ino || named.isSymbolicLink() || bytes.length !== Number(before.size)) throw new Error("Installed CLI proof file changed during admission");
|
||||
return bytes;
|
||||
} finally { await handle.close(); }
|
||||
}
|
||||
export async function verifyInstalledCli(env: NodeJS.ProcessEnv, executions: readonly MatrixExecution[]) {
|
||||
if (!usesInstalledCli(env)) return undefined;
|
||||
assertInstalledCliSelection(env, executions);
|
||||
const entry = env.PAPERCLIP_RUNNER_E2E_INSTALLED_CLI!;
|
||||
const serverRoot = env.PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_ROOT!;
|
||||
for (const path of [entry, serverRoot]) if (!isAbsolute(path) || resolve(path) !== path || await realpath(path) !== path) throw new Error("Installed CLI proof paths must be canonical and unlinked");
|
||||
if (!entry.endsWith("/dist/index.js")) throw new Error("Installed CLI proof requires published dist/index.js");
|
||||
const cliRoot = resolve(dirname(entry), "..");
|
||||
const cli = JSON.parse((await readOwned(join(cliRoot, "package.json"), 65536)).toString());
|
||||
const manifest = findPackageJSON("@paperclipai/server", pathToFileURL(entry));
|
||||
if (!manifest || await realpath(manifest) !== join(serverRoot, "package.json")) throw new Error("Installed CLI resolved a foreign server dependency");
|
||||
const server = JSON.parse((await readOwned(join(serverRoot, "package.json"), 65536)).toString());
|
||||
if (cli.name !== "paperclipai" || cli.bin?.paperclipai !== "./dist/index.js" || server.name !== "@paperclipai/server" || typeof cli.version !== "string" || cli.version !== server.version || server.exports?.["."]?.import !== "./dist/index.js") throw new Error("Installed CLI/server public package identity differs");
|
||||
const serverEntry = join(serverRoot, "dist/index.js");
|
||||
if (await realpath(serverEntry) !== serverEntry) throw new Error("Installed server entrypoint escapes its package");
|
||||
const digest = (bytes: Buffer) => createHash("sha256").update(bytes).digest("hex");
|
||||
const cliSha256 = digest(await readOwned(entry, 64 * 1024 * 1024));
|
||||
const serverSha256 = digest(await readOwned(serverEntry, 16 * 1024 * 1024));
|
||||
if (cliSha256 !== env.PAPERCLIP_RUNNER_E2E_INSTALLED_CLI_SHA256 || serverSha256 !== env.PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_SHA256) throw new Error("Installed CLI/server bytes differ from their reviewed pins");
|
||||
return { schema: "paperclip.e2e.installed-cli-admission/v1", entry, cliRoot, cliSha256, serverRoot, serverEntry, serverSha256, version: cli.version, defaultRuntimeResolution: true, qualificationOverride: false };
|
||||
}
|
||||
|
||||
/** This prep lane cannot create agents or read the normal local credential file. */
|
||||
export function assertInstalledStartupOnly(env: NodeJS.ProcessEnv, executions: readonly MatrixExecution[], options: { ids: string[]; maxParallel: number; maxAutomaticRetries: number; headed: boolean; ui: boolean; debug: boolean; list: boolean; matrixJson: boolean }) {
|
||||
assertInstalledCliSelection(env, executions);
|
||||
if (!usesInstalledCli(env) || executions.length !== 1 || executions[0]?.id !== "extended-harnesses.runner-acpx-pi.local.hello-complete" || options.ids.length !== 1 || options.ids[0] !== executions[0]?.id || options.maxParallel !== 1 || options.maxAutomaticRetries !== 0 || options.headed || options.ui || options.debug || options.list || options.matrixJson) throw new Error("Installed startup-only requires one explicit local Pi hello, no retries or interactive mode");
|
||||
if (CREDENTIAL_NAMES.some(key => env[key] !== undefined) || Object.keys(env).some(key => /^(?:OPENAI|ANTHROPIC|OPENROUTER|DAYTONA|XAI|GROK|CURSOR|COPILOT|GITHUB|GH)(?:_|$)/.test(key) && env[key] !== undefined)) throw new Error("Installed startup-only forbids provider credential inputs");
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { mkdtemp, mkdir, realpath, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import { runnerMatrix } from "./catalog.js";
|
||||
import { installedDaytonaPluginKeys, verifyInstalledDaytonaPlugin, type InstalledDaytonaPluginAuthority } from "./installed-daytona-plugin.js";
|
||||
import { buildPaperclipServerEnvironment } from "./harness-env.js";
|
||||
import { setupLiveFixtures } from "./live-fixtures.js";
|
||||
import type { RunnerApi } from "./api.js";
|
||||
const cell = runnerMatrix.find(e => e.id === "extended-harnesses.runner-acpx-pi.daytona.hello-complete")!;
|
||||
const sdkVersion: string = JSON.parse(readFileSync(new URL("../../packages/plugins/sdk/package.json", import.meta.url), "utf8")).version;
|
||||
const roots: string[] = [];
|
||||
const hash = (v: string) => createHash("sha256").update(v).digest("hex");
|
||||
afterEach(async () => { vi.unstubAllEnvs(); await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); });
|
||||
async function fixture() {
|
||||
const root = await realpath(await mkdtemp(join(tmpdir(), "e2e-installed-plugin-"))); roots.push(root);
|
||||
async function pkg(name: string, version: string, dependencies: Record<string,string> = {}, extra = {}) {
|
||||
const dir = join(root, "node_modules", name); const entry = name === "@daytonaio/sdk" ? "./esm/index.js" : "./dist/index.js"; await mkdir(join(dir, name === "@daytonaio/sdk" ? "esm" : "dist"), { recursive: true });
|
||||
const content = JSON.stringify({ name, version, type: "module", exports: { ".": { import: name === "@daytonaio/sdk" ? { types: "./esm/index.d.ts", default: entry } : entry } }, dependencies, ...extra });
|
||||
await writeFile(join(dir, "package.json"), content); await writeFile(join(dir, entry), "// compiled");
|
||||
return { root: dir, packageSha256: hash(content), entry, entrySha256: hash("// compiled") };
|
||||
}
|
||||
const plugin = await pkg("@paperclipai/plugin-daytona", "0.1.1", { "@paperclipai/plugin-sdk": sdkVersion, "@daytonaio/sdk": "0.203.0" }, { paperclipPlugin: { manifest: "./dist/manifest.js", worker: "./dist/worker.js" } });
|
||||
await writeFile(join(plugin.root, "dist/manifest.js"), "// manifest"); await writeFile(join(plugin.root, "dist/worker.js"), "// worker");
|
||||
const authority: InstalledDaytonaPluginAuthority = { schema: "paperclip.e2e.installed-daytona-plugin/v1", graphRoot: root, plugin: { ...plugin, manifestSha256: hash("// manifest"), workerSha256: hash("// worker") }, sdk: await pkg("@paperclipai/plugin-sdk", sdkVersion, { "@paperclipai/shared": "0.3.1" }), shared: await pkg("@paperclipai/shared", "0.3.1"), daytona: await pkg("@daytonaio/sdk", "0.203.0") };
|
||||
const authorityPath = join(root, "authority.json");
|
||||
const env: NodeJS.ProcessEnv = { PAPERCLIP_RUNNER_E2E_INSTALLED_CLI: "/reviewed/cli", PAPERCLIP_RUNNER_E2E_INSTALLED_CLI_SHA256: "reviewed-separately", PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_ROOT: "/reviewed/server", PAPERCLIP_RUNNER_E2E_INSTALLED_SERVER_SHA256: "reviewed-separately", PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN: plugin.root, PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY: authorityPath };
|
||||
async function seal() { const bytes = JSON.stringify(authority); await writeFile(authorityPath, bytes); env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY_SHA256 = hash(bytes); }
|
||||
await seal(); return { root, authority, env, seal };
|
||||
}
|
||||
it("admits pinned published plugin and resolved SDK graph without claiming a full graph audit", async () => {
|
||||
const f = await fixture(); expect(await verifyInstalledDaytonaPlugin(f.env, [cell])).toMatchObject({ packageRoot: f.authority.plugin.root, sourceFallback: false, fullGraphVerified: false, completeGraphAuthorityRequired: true });
|
||||
});
|
||||
it("fails closed for absent/partial installed Daytona pins but preserves ordinary source and local lanes", async () => {
|
||||
const f = await fixture();
|
||||
for (const key of installedDaytonaPluginKeys) await expect(verifyInstalledDaytonaPlugin({ ...f.env, [key]: undefined }, [cell])).rejects.toThrow("complete pins");
|
||||
await expect(verifyInstalledDaytonaPlugin({}, [cell])).resolves.toBeUndefined();
|
||||
const local = runnerMatrix.find(e => e.id === "extended-harnesses.runner-acpx-pi.local.hello-complete")!;
|
||||
await expect(verifyInstalledDaytonaPlugin(f.env, [local])).rejects.toThrow("Daytona selection");
|
||||
const noPlugin = { ...f.env }; for (const key of installedDaytonaPluginKeys) delete noPlugin[key];
|
||||
await expect(verifyInstalledDaytonaPlugin(noPlugin, [local])).resolves.toBeUndefined();
|
||||
await expect(verifyInstalledDaytonaPlugin(noPlugin, [cell])).rejects.toThrow("complete pins");
|
||||
});
|
||||
it("rejects stale authority, worker, manifest and dependency entry bytes", async () => {
|
||||
const f = await fixture();
|
||||
await expect(verifyInstalledDaytonaPlugin({ ...f.env, PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY_SHA256: "0".repeat(64) }, [cell])).rejects.toThrow("reviewed pins");
|
||||
for (const [path, original] of [[join(f.authority.plugin.root,"dist/worker.js"),"// worker"], [join(f.authority.plugin.root,"dist/manifest.js"),"// manifest"], [join(f.authority.sdk.root,"dist/index.js"),"// compiled"]]) {
|
||||
await writeFile(path!, "// wrong"); await expect(verifyInstalledDaytonaPlugin(f.env,[cell])).rejects.toThrow("reviewed pins"); await writeFile(path!,original!);
|
||||
}
|
||||
});
|
||||
it("rejects source exports and workspace dependency versions even with refreshed pins", async () => {
|
||||
const f = await fixture(); const p = join(f.authority.plugin.root,"package.json");
|
||||
const original = { name:"@paperclipai/plugin-daytona",version:"0.1.1",exports:{".":{import:"./dist/index.js"}},paperclipPlugin:{manifest:"./dist/manifest.js",worker:"./dist/worker.js"},dependencies:{"@paperclipai/plugin-sdk":sdkVersion,"@daytonaio/sdk":"0.203.0"} };
|
||||
for (const manifest of [{ ...original, exports:{".":{import:"./src/index.ts"}} }, { ...original, dependencies:{ ...original.dependencies,"@paperclipai/plugin-sdk":"workspace:*" } }]) {
|
||||
const bytes=JSON.stringify(manifest);await writeFile(p,bytes);f.authority.plugin.packageSha256=hash(bytes);await f.seal();await expect(verifyInstalledDaytonaPlugin(f.env,[cell])).rejects.toThrow(/compiled package exports|dependency identity/);
|
||||
}
|
||||
});
|
||||
it("rejects linked or escaped roots and a shadowed foreign resolved dependency", async () => {
|
||||
const f = await fixture(); const other = await fixture();
|
||||
f.authority.shared=other.authority.shared;await f.seal();await expect(verifyInstalledDaytonaPlugin(f.env,[cell])).rejects.toThrow("escapes");
|
||||
f.authority.shared={...other.authority.shared,root:join(f.root,"node_modules/@paperclipai/shared")};await f.seal();
|
||||
const nested=join(f.authority.plugin.root,"node_modules/@paperclipai");await mkdir(nested,{recursive:true});await symlink(other.authority.sdk.root,join(nested,"plugin-sdk"));
|
||||
await expect(verifyInstalledDaytonaPlugin(f.env,[cell])).rejects.toThrow("foreign dependency");
|
||||
await rm(nested,{recursive:true});await rm(f.authority.shared.root,{recursive:true});await symlink(other.authority.shared.root,f.authority.shared.root);
|
||||
await expect(verifyInstalledDaytonaPlugin(f.env,[cell])).rejects.toThrow("canonical and unlinked");
|
||||
});
|
||||
it.skipIf(process.platform === "win32")("rejects a FIFO authority without waiting for a writer", async () => {
|
||||
const f=await fixture();const p=f.env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY!;await rm(p);execFileSync("/usr/bin/mkfifo",[p],{timeout:5000,env:{}});await expect(verifyInstalledDaytonaPlugin(f.env,[cell])).rejects.toThrow("bounded unlinked regular file");
|
||||
});
|
||||
it("uses the verified public package in the existing API and rejects changed bytes before any mutation", async () => {
|
||||
// This positive fixture models the sanitized installed launcher, not pnpm's parent environment.
|
||||
vi.stubEnv("NODE_OPTIONS", undefined); vi.stubEnv("NODE_PATH", undefined);
|
||||
const f=await fixture(); for(const [k,v]of Object.entries(f.env))vi.stubEnv(k,v!);
|
||||
const calls:unknown[]=[];
|
||||
const api={async post(url:string,data:unknown){calls.push({url,data});throw new Error("stop-after-public-install");}} as unknown as RunnerApi;
|
||||
const input={api,execution:cell,executionNonce:"nonce",workspacePath:"/tmp/workspace",credentials:{},daytonaImage:"ghcr.io/example/image@sha256:"+"a".repeat(64)};
|
||||
await expect(setupLiveFixtures(input)).rejects.toThrow("stop-after-public-install");
|
||||
expect(calls).toEqual([{url:"/api/plugins/install",data:{packageName:f.authority.plugin.root,isLocalPath:true}}]);
|
||||
calls.length=0;await writeFile(join(f.authority.plugin.root,"dist/worker.js"),"// changed");await expect(setupLiveFixtures(input)).rejects.toThrow("reviewed pins");expect(calls).toEqual([]);
|
||||
});
|
||||
it.each(["NODE_OPTIONS", "NODE_PATH"])("rejects ambient %s before the plugin install API", async (key) => {
|
||||
const f = await fixture();
|
||||
for (const [name, value] of Object.entries(f.env)) vi.stubEnv(name, value!);
|
||||
vi.stubEnv("NODE_OPTIONS", undefined); vi.stubEnv("NODE_PATH", undefined);
|
||||
const post = vi.fn();
|
||||
const api = { post } as unknown as RunnerApi;
|
||||
for (const value of ["", "foreign"]) {
|
||||
vi.stubEnv(key, value);
|
||||
await expect(setupLiveFixtures({ api, execution: cell, executionNonce: "nonce", workspacePath: "/tmp/workspace", credentials: {} })).rejects.toThrow("ambient Node injection");
|
||||
expect(post).not.toHaveBeenCalled();
|
||||
}
|
||||
});
|
||||
it("keeps fixture authority out of the production server environment", async()=>{const f=await fixture();const env=buildPaperclipServerEnvironment(f.env);for(const k of installedDaytonaPluginKeys)expect(env[k]).toBeUndefined();});
|
||||
@@ -0,0 +1,72 @@
|
||||
/** Public plugin fixture admission; the complete installed graph is audited separately. */
|
||||
import { constants } from "node:fs";
|
||||
import { lstat, open, realpath } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import { findPackageJSON } from "node:module";
|
||||
import { isAbsolute, join, resolve } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { assertInstalledCliSelection, usesInstalledCli } from "./installed-cli.js";
|
||||
import type { MatrixExecution } from "./types.js";
|
||||
|
||||
export const installedDaytonaPluginKeys = ["PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN", "PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY", "PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY_SHA256"] as const;
|
||||
type PackagePin = { root: string; packageSha256: string; entry: string; entrySha256: string };
|
||||
export interface InstalledDaytonaPluginAuthority {
|
||||
schema: "paperclip.e2e.installed-daytona-plugin/v1";
|
||||
graphRoot: string;
|
||||
plugin: PackagePin & { manifestSha256: string; workerSha256: string };
|
||||
sdk: PackagePin;
|
||||
shared: PackagePin;
|
||||
daytona: PackagePin;
|
||||
}
|
||||
const digest = (bytes: Buffer) => createHash("sha256").update(bytes).digest("hex");
|
||||
async function canonical(path: string) {
|
||||
if (typeof path !== "string" || !isAbsolute(path) || resolve(path) !== path || await realpath(path) !== path) throw new Error("Installed Daytona plugin path must be canonical and unlinked");
|
||||
}
|
||||
async function pinned(path: string, expected: string, maximum = 32 * 1024 * 1024) {
|
||||
if (!/^[a-f0-9]{64}$/.test(expected)) throw new Error("Installed Daytona plugin requires a SHA-256 pin");
|
||||
await canonical(path);
|
||||
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
|
||||
try {
|
||||
const before = await handle.stat({ bigint: true });
|
||||
if (!before.isFile() || before.size < 1n || before.size > BigInt(maximum) || before.nlink !== 1n) throw new Error("Installed Daytona plugin requires a bounded unlinked regular file");
|
||||
const bytes = await handle.readFile(); const after = await handle.stat({ bigint: true }); const named = await lstat(path, { bigint: true });
|
||||
if (before.dev !== after.dev || before.ino !== after.ino || before.size !== after.size || before.mtimeNs !== after.mtimeNs || before.ctimeNs !== after.ctimeNs || named.dev !== before.dev || named.ino !== before.ino || bytes.length !== Number(before.size) || digest(bytes) !== expected) throw new Error("Installed Daytona plugin bytes differ from reviewed pins");
|
||||
return bytes;
|
||||
} finally { await handle.close(); }
|
||||
}
|
||||
function compiledEntry(manifest: any): string | undefined {
|
||||
const exported = manifest.exports?.["."];
|
||||
return typeof exported === "string" ? exported : (typeof exported?.import === "string" ? exported.import : exported?.import?.default) ?? manifest.main;
|
||||
}
|
||||
export async function verifyInstalledDaytonaPlugin(env: NodeJS.ProcessEnv, executions: readonly MatrixExecution[]) {
|
||||
const selected = installedDaytonaPluginKeys.some(key => env[key] !== undefined);
|
||||
const required = usesInstalledCli(env) && executions.some(e => e.environment.id === "daytona");
|
||||
if (!selected && !required) return undefined;
|
||||
assertInstalledCliSelection(env, executions);
|
||||
if (!usesInstalledCli(env) || !installedDaytonaPluginKeys.every(key => Boolean(env[key])) || executions.some(e => e.environment.id !== "daytona")) throw new Error("Installed Daytona plugin requires complete pins and installed Pi Daytona selection");
|
||||
const authorityPath = env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY!;
|
||||
const authoritySha256 = env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_AUTHORITY_SHA256!;
|
||||
const a: InstalledDaytonaPluginAuthority = JSON.parse((await pinned(authorityPath, authoritySha256, 65536)).toString());
|
||||
if (a.schema !== "paperclip.e2e.installed-daytona-plugin/v1" || a.plugin?.root !== env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN) throw new Error("Installed Daytona plugin authority differs");
|
||||
await canonical(a.graphRoot);
|
||||
async function packageAt(pin: PackagePin, name: string) {
|
||||
await canonical(pin.root);
|
||||
if (pin.root !== join(a.graphRoot, "node_modules", name)) throw new Error("Installed Daytona plugin package escapes its reviewed graph");
|
||||
const manifest = JSON.parse((await pinned(join(pin.root, "package.json"), pin.packageSha256, 65536)).toString());
|
||||
if (manifest.name !== name || typeof manifest.version !== "string" || !/^\d+\.\d+\.\d+(?:[-+][a-zA-Z0-9.-]+)?$/.test(manifest.version) || pin.entry !== (name === "@daytonaio/sdk" ? "./esm/index.js" : "./dist/index.js") || compiledEntry(manifest) !== pin.entry) throw new Error("Installed Daytona plugin requires published compiled package exports");
|
||||
await pinned(join(pin.root, pin.entry), pin.entrySha256);
|
||||
return manifest;
|
||||
}
|
||||
const plugin = await packageAt(a.plugin, "@paperclipai/plugin-daytona");
|
||||
const sdk = await packageAt(a.sdk, "@paperclipai/plugin-sdk");
|
||||
const shared = await packageAt(a.shared, "@paperclipai/shared");
|
||||
const daytona = await packageAt(a.daytona, "@daytonaio/sdk");
|
||||
if (plugin.paperclipPlugin?.manifest !== "./dist/manifest.js" || plugin.paperclipPlugin?.worker !== "./dist/worker.js" || plugin.dependencies?.["@paperclipai/plugin-sdk"] !== sdk.version || sdk.dependencies?.["@paperclipai/shared"] !== shared.version || plugin.dependencies?.["@daytonaio/sdk"] !== "0.203.0" || daytona.version !== "0.203.0") throw new Error("Installed Daytona plugin public dependency identity differs");
|
||||
await pinned(join(a.plugin.root, "dist/manifest.js"), a.plugin.manifestSha256);
|
||||
await pinned(join(a.plugin.root, "dist/worker.js"), a.plugin.workerSha256);
|
||||
for (const [name, parent, pin] of [["@paperclipai/plugin-sdk", join(a.plugin.root, "dist/worker.js"), a.sdk], ["@paperclipai/shared", join(a.sdk.root, a.sdk.entry), a.shared], ["@daytonaio/sdk", join(a.plugin.root, "dist/worker.js"), a.daytona]] as const) {
|
||||
const found = findPackageJSON(name, pathToFileURL(parent));
|
||||
if (!found || await realpath(found) !== join(pin.root, "package.json")) throw new Error("Installed Daytona plugin resolves a foreign dependency");
|
||||
}
|
||||
return { schema: a.schema, authorityPath, authoritySha256, packageRoot: a.plugin.root, graphRoot: a.graphRoot, fullGraphVerified: false, completeGraphAuthorityRequired: true, sourceFallback: false };
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
/** Credential-free proof of the real launcher/Playwright/WebServer chain. */
|
||||
import { test, expect } from "@playwright/test";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
test("installed controller serves health and UI without creating provider work", async ({ page, request, baseURL }) => {
|
||||
test.skip(process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_STARTUP_ONLY !== "1");
|
||||
expect(baseURL).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/);
|
||||
const errors: string[] = [];
|
||||
page.on("pageerror", error => errors.push(error.message));
|
||||
await page.route("**/*", route => {
|
||||
const url = new URL(route.request().url());
|
||||
return url.origin === baseURL ? route.continue() : route.abort("blockedbyclient");
|
||||
});
|
||||
const health = await request.get(`${baseURL}/api/health`);
|
||||
expect(health.ok()).toBe(true);
|
||||
const before = await request.get(`${baseURL}/api/companies`);
|
||||
expect(before.ok()).toBe(true); expect(await before.json()).toEqual([]);
|
||||
const response = await page.goto(baseURL!);
|
||||
expect(response?.ok()).toBe(true);
|
||||
await expect(page.locator("#root")).not.toBeEmpty();
|
||||
await expect(page.getByRole("heading", { name: "What is the name of your organization?", exact: true })).toBeVisible();
|
||||
await expect(page.getByRole("textbox", { name: "Name", exact: true })).toBeVisible();
|
||||
await expect(page.getByRole("button", { name: "Continue", exact: true })).toBeDisabled();
|
||||
expect(errors).toEqual([]);
|
||||
const after = await request.get(`${baseURL}/api/companies`);
|
||||
expect(after.ok()).toBe(true); expect(await after.json()).toEqual([]);
|
||||
const output = process.env.PAPERCLIP_RUNNER_E2E_PRIVATE_DIR!;
|
||||
await page.screenshot({ path: path.join(output, "installed-startup-only.png"), fullPage: true });
|
||||
await writeFile(path.join(output, "installed-startup-only.json"), `${JSON.stringify({ status: "health_ui_zero_company_passed", qualified: false, providerCalls: 0, healthStatus: health.status(), uiStatus: response!.status(), companiesBefore: 0, companiesAfter: 0, pageErrors: errors }, null, 2)}\n`, { mode: 0o600 });
|
||||
});
|
||||
+46
-11
@@ -1,4 +1,8 @@
|
||||
import { runnerE2EPlaywrightInvocation } from "./web-server-command.js";
|
||||
import { verifyInstalledDaytonaPlugin } from "./installed-daytona-plugin.js";
|
||||
import { assertInstalledCliSelection, assertInstalledStartupOnly, verifyInstalledCli } from "./installed-cli.js";
|
||||
import { createProcessTreeOwner, stopOwnedProcessTree } from "./process-tree-owner.js";
|
||||
import { createRunnerE2ETemporaryRoot } from "./server-config.js";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { prepareCodexCiSandbox, requiresCodexCiSandbox } from "./codex-ci-sandbox.js";
|
||||
import { spawn } from "node:child_process";
|
||||
@@ -12,8 +16,8 @@ import {
|
||||
cp,
|
||||
lstat,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
realpath,
|
||||
readdir,
|
||||
rm,
|
||||
symlink,
|
||||
@@ -262,7 +266,7 @@ async function prepareProviderPath(
|
||||
}
|
||||
|
||||
async function runProcess(
|
||||
args: string[],
|
||||
invocation: { command: string; args: string[] },
|
||||
env: NodeJS.ProcessEnv,
|
||||
timeoutMs: number | null,
|
||||
logPath: string,
|
||||
@@ -270,7 +274,7 @@ async function runProcess(
|
||||
interactive: boolean,
|
||||
) {
|
||||
const log = createWriteStream(logPath, { flags: "a", mode: 0o600 });
|
||||
const child = spawn("pnpm", args, {
|
||||
const child = spawn(invocation.command, invocation.args, {
|
||||
cwd: repositoryRoot,
|
||||
env,
|
||||
stdio: ["inherit", "pipe", "pipe"],
|
||||
@@ -477,15 +481,18 @@ async function runAttempt(input: {
|
||||
"One isolated runner E2E harness cannot mix profiles or environments",
|
||||
);
|
||||
}
|
||||
assertInstalledCliSelection(process.env, executions);
|
||||
const installedCli = await verifyInstalledCli(process.env, executions);
|
||||
const installedPlugin = await verifyInstalledDaytonaPlugin(process.env, executions);
|
||||
const startedAtMs = Date.now();
|
||||
const sharedMemoryBaseline = snapshotDarwinSharedMemory();
|
||||
const temporaryRoot = await mkdtemp(
|
||||
path.join(os.tmpdir(), "paperclip-runner-e2e-"),
|
||||
);
|
||||
const temporaryParent = await realpath(os.tmpdir());
|
||||
const temporaryRoot = await createRunnerE2ETemporaryRoot(temporaryParent);
|
||||
const publishedResults: RunnerE2EResult[] = [];
|
||||
const publishedResultPaths = new Map<string, string>();
|
||||
let attemptSecrets: string[] = [];
|
||||
let processCleanupFailed = false;
|
||||
let startupReceiptPath: string | undefined;
|
||||
const rawCleanupResults: Array<{ cleanup: string; synthetic: boolean; status: string }> = [];
|
||||
try {
|
||||
const paperclipHome = path.join(temporaryRoot, "paperclip-home");
|
||||
@@ -523,7 +530,9 @@ async function runAttempt(input: {
|
||||
betterAuthSecret,
|
||||
]);
|
||||
attemptSecrets = credentials;
|
||||
const runnerBinary = resolvePaperclipRunnerBinaryForHarness(
|
||||
if (installedCli) await writeFile(path.join(privateDir, "installed-cli-admission.json"), `${JSON.stringify(installedCli, null, 2)}\n`, { mode: 0o600 });
|
||||
if (installedPlugin) await writeFile(path.join(privateDir, "installed-daytona-plugin-admission.json"), `${JSON.stringify(installedPlugin, null, 2)}\n`, { mode: 0o600 });
|
||||
const runnerBinary = installedCli ? undefined : resolvePaperclipRunnerBinaryForHarness(
|
||||
executions,
|
||||
repositoryRoot,
|
||||
);
|
||||
@@ -534,6 +543,7 @@ async function runAttempt(input: {
|
||||
executions.map((candidate) => candidate.id),
|
||||
),
|
||||
PAPERCLIP_RUNNER_E2E_ATTEMPT: String(attempt),
|
||||
PAPERCLIP_RUNNER_E2E_INSTALLED_STARTUP_ONLY: options.installedStartupOnly ? "1" : undefined,
|
||||
PAPERCLIP_RUNNER_E2E_PUBLIC_MCP: executions.some(candidate => candidate.task.flow === "public_mcp") ? "1" : "0",
|
||||
PAPERCLIP_RUNNER_E2E_PORT: String(port),
|
||||
PAPERCLIP_RUNNER_E2E_TEMP_ROOT: temporaryRoot,
|
||||
@@ -542,7 +552,7 @@ async function runAttempt(input: {
|
||||
PAPERCLIP_RUNNER_E2E_SERVER_LOG: path.join(privateDir, "server.log"),
|
||||
PAPERCLIP_RUNNER_BINARY: runnerBinary,
|
||||
PAPERCLIP_RUNNER_REMOTE_BINARY_PATH:
|
||||
resolvePaperclipRemoteRunnerBinaryForHarness(executions, runnerBinary),
|
||||
installedCli ? undefined : resolvePaperclipRemoteRunnerBinaryForHarness(executions, runnerBinary),
|
||||
// Vite's optimized dependency cache embeds revision query strings. A
|
||||
// private per-attempt cache prevents an earlier cell or local rebuild
|
||||
// from producing `504 Outdated Optimize Dep` during browser bootstrap.
|
||||
@@ -569,8 +579,6 @@ async function runAttempt(input: {
|
||||
delete childEnv.DATABASE_MIGRATION_URL;
|
||||
|
||||
const playwrightArgs = [
|
||||
"exec",
|
||||
"playwright",
|
||||
"test",
|
||||
"--config",
|
||||
"tests/runner-e2e/playwright.config.ts",
|
||||
@@ -590,7 +598,7 @@ async function runAttempt(input: {
|
||||
) +
|
||||
5 * 60_000;
|
||||
const processResult = await runProcess(
|
||||
playwrightArgs,
|
||||
runnerE2EPlaywrightInvocation(repositoryRoot, playwrightArgs, Boolean(installedCli)),
|
||||
childEnv,
|
||||
watchdog,
|
||||
path.join(privateDir, "playwright.log"),
|
||||
@@ -600,6 +608,18 @@ async function runAttempt(input: {
|
||||
options.ui || options.debug,
|
||||
);
|
||||
processCleanupFailed = processResult.processCleanupError !== null;
|
||||
if (options.installedStartupOnly) {
|
||||
const proofDir = path.join(resultsRoot, campaignId, "installed-startup-only");
|
||||
await mkdir(proofDir, { recursive: true });
|
||||
// This is private setup evidence, never a successful provider case/result.
|
||||
await cp(privateDir, path.join(proofDir, "private"), { recursive: true });
|
||||
const probe = JSON.parse(await readFile(path.join(privateDir, "installed-startup-only.json"), "utf8"));
|
||||
if (processResult.exitCode !== 0 || processResult.timedOut || processResult.spawnError || processCleanupFailed || probe.status !== "health_ui_zero_company_passed" || probe.providerCalls !== 0 || probe.qualified !== false) throw new Error("Installed startup-only actual launch chain failed");
|
||||
await assertEmbeddedDatabaseIsolation(configPath, temporaryRoot);
|
||||
startupReceiptPath = path.join(proofDir, "receipt.json");
|
||||
await writeFile(startupReceiptPath, `${JSON.stringify({ status: "installed_launch_chain_pending_scratch_cleanup", qualified: false, providerCalls: 0, credentialsLoaded: false, installedCli, processResult, probe }, null, 2)}\n`, { mode: 0o600 });
|
||||
return [];
|
||||
}
|
||||
const processFailure = processResult.spawnError
|
||||
? `Playwright failed to start: ${processResult.spawnError}`
|
||||
: processResult.timedOut
|
||||
@@ -844,6 +864,12 @@ async function runAttempt(input: {
|
||||
`Refusing to remove unexpected temporary path ${temporaryRoot}`,
|
||||
);
|
||||
}
|
||||
if (startupReceiptPath) {
|
||||
const receipt = JSON.parse(await readFile(startupReceiptPath, "utf8"));
|
||||
receipt.status = cleanupError ? "installed_launch_chain_cleanup_failed" : "installed_launch_chain_passed";
|
||||
receipt.temporaryRootRemoved = !cleanupError;
|
||||
await writeFile(startupReceiptPath, `${JSON.stringify(receipt, null, 2)}\n`, { mode: 0o600 });
|
||||
}
|
||||
if (cleanupError) {
|
||||
const message = `Temporary runner E2E state cleanup failed at ${temporaryRoot}: ${cleanupError instanceof Error ? cleanupError.message : String(cleanupError)}`;
|
||||
for (const publishedResult of publishedResults) {
|
||||
@@ -868,6 +894,7 @@ async function runAttempt(input: {
|
||||
// Cleanup failure is a failed cell, but must not suppress later cells in
|
||||
// the same campaign. The result above carries the terminal failure.
|
||||
console.error(message);
|
||||
if (options.installedStartupOnly) throw new Error(message);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -942,6 +969,14 @@ async function main() {
|
||||
throw error;
|
||||
}
|
||||
const executions = selectRunnerExecutions(options, runnerMatrix);
|
||||
if (options.installedStartupOnly) {
|
||||
assertInstalledStartupOnly(process.env, executions, options);
|
||||
assertRunnerE2EPrerequisites(executions);
|
||||
const campaignId = cleanId(process.env.PAPERCLIP_E2E_CAMPAIGN_ID ?? `installed-startup-${Date.now()}`);
|
||||
await runAttempt({ executions, attempt: 1, campaignId, options });
|
||||
console.log(`PASS installed launch chain (no provider qualification) -> ${path.join(resultsRoot, campaignId, "installed-startup-only")}`);
|
||||
return;
|
||||
}
|
||||
if (options.list) {
|
||||
printList(executions);
|
||||
return;
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { verifyInstalledDaytonaPlugin } from "./installed-daytona-plugin.js";
|
||||
import { NATIVE_COMPLETION_BUDGET_CENTS } from "./native-completion-defaults.js";
|
||||
import path from "node:path";
|
||||
import { installedReleaseDaytonaPlugin } from "./installed-release.js";
|
||||
@@ -115,10 +116,9 @@ export async function setupLiveFixtures(input: {
|
||||
registry.register<PluginRecord>({
|
||||
id: "sandbox-provider",
|
||||
async setup() {
|
||||
const installed = await verifyInstalledDaytonaPlugin(process.env, [execution]);
|
||||
return api.post<PluginRecord>("/api/plugins/install", {
|
||||
packageName: process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_CLI
|
||||
? installedReleaseDaytonaPlugin(process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_CLI, process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN, process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_DAYTONA_PLUGIN_VERSION)
|
||||
: path.resolve(
|
||||
packageName: installed?.packageRoot ?? path.resolve(
|
||||
import.meta.dirname,
|
||||
"../../packages/plugins/sandbox-providers/daytona",
|
||||
),
|
||||
|
||||
@@ -2,13 +2,14 @@ import { createHash } from "node:crypto";
|
||||
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
|
||||
import { hasAcpxNativeOrigin } from "./acpx-native-origin.js";
|
||||
import { isValidNativePrpEnvelope } from "./native-event-envelope.js";
|
||||
import { assertNativeRemoteRetirement, nativeRemoteDeniedSample, type NativeRemoteSnapshot } from "./native-remote-evidence.js";
|
||||
import { assertCopilotRemoteRetirement, copilotRemoteDeniedSample, type CopilotRemoteSnapshot } from "./copilot-protection-evidence.js";
|
||||
import { readCopilotToolEvidence } from "./copilot-evidence.js";
|
||||
import { readCursorToolEvidence } from "./cursor-native-evidence.js";
|
||||
import { bootstrapReadExecutionId, withoutProvenBootstrapReads, type BootstrapReadProof } from "./native-bootstrap-read-proof.js";
|
||||
|
||||
type Row = Record<string, any>;
|
||||
export interface ActiveStopCaller { type: "board"; userId: "local-board"; source: "local_implicit" }
|
||||
export type ActiveStopProvider = "cursor";
|
||||
export type ActiveStopProvider = "cursor" | "copilot";
|
||||
export interface ActiveStopScope { provider: ActiveStopProvider; companyId: string; issueId: string; runId: string; target: string; commandSha256?: string }
|
||||
export interface ActiveStopPending {
|
||||
schema: "paperclip.e2e.native-active-stop-pending.v2";
|
||||
@@ -61,7 +62,7 @@ const closures = new Set(["runtime_request.resolved", "runtime_request.cancelled
|
||||
function origin(events: readonly unknown[], scope: ActiveStopScope, bootstrap?: BootstrapReadProof) {
|
||||
fail([scope.companyId, scope.issueId, scope.runId, scope.target].every(id) && ["cursor", "copilot"].includes(scope.provider), "exact scope required");
|
||||
const rows = canonicalRows(events, scope);
|
||||
const notices = readCursorToolEvidence(events, scope.runId);
|
||||
const notices = scope.provider === "cursor" ? readCursorToolEvidence(events, scope.runId) : readCopilotToolEvidence(events, scope.runId);
|
||||
const requests = notices.filter(n => n.stage === "permission_requested");
|
||||
fail(requests.length === 1, "one native permission required");
|
||||
const notice = requests[0]!;
|
||||
@@ -119,7 +120,7 @@ export function observeActiveStopPending(input: { events: readonly unknown[]; ru
|
||||
fail(isActiveStopCaller(input.caller) && uuid(input.cancellationRequestId) && run.id === scope.runId && run.companyId === scope.companyId && run.nativeIssueId === scope.issueId
|
||||
&& run.status === "running" && run.runtimeMode === "native" && issue.id === scope.issueId && issue.companyId === scope.companyId && issue.status === "in_progress"
|
||||
&& run.resultJson?.startupCancellation == null && run.resultJson?.nativeCancellation == null, "run is not fresh active work");
|
||||
fail(!(readCursorToolEvidence(input.events, scope.runId)).some(n => n.stage === "permission_delivered"), "permission already answered");
|
||||
fail(!(scope.provider === "cursor" ? readCursorToolEvidence(input.events, scope.runId) : readCopilotToolEvidence(input.events, scope.runId)).some(n => n.stage === "permission_delivered"), "permission already answered");
|
||||
fail(!proof.rows.some(x => terminals.has(x.event.eventType) || closures.has(x.event.eventType)), "request or provider already settled");
|
||||
fail(!proof.rows.some(x => x.event.eventType === "tool.execution.completed" && rec(x.event.payload).executionId === bootstrapReadExecutionId(proof.notice.toolCallId)), "operation already ended");
|
||||
return { schema: "paperclip.e2e.native-active-stop-pending.v2", scope, caller: input.caller, requestId: proof.notice.requestId!, toolCallId: proof.notice.toolCallId,
|
||||
@@ -180,7 +181,7 @@ export function readActiveStopSettlement(input: { events: readonly unknown[]; ru
|
||||
export interface ActiveStopRemoteObservation {
|
||||
phase: "before-request" | "pending" | "owned-process-retirement";
|
||||
source: "live-snapshot" | "retirement-seal";
|
||||
snapshot: NativeRemoteSnapshot;
|
||||
snapshot: CopilotRemoteSnapshot;
|
||||
}
|
||||
|
||||
/** A per-turn Daytona observer seals itself when the owned tree retires. The
|
||||
@@ -192,10 +193,10 @@ export function readActiveStopRemoteRetirement(input: {
|
||||
fail(observations.length === 3
|
||||
&& observations.map(o => `${o.phase}:${o.source}`).join(",") === "before-request:live-snapshot,pending:live-snapshot,owned-process-retirement:retirement-seal",
|
||||
"remote seal cannot stand in for a fresh causal sample");
|
||||
const [baseline, pending, terminal] = observations.map(o => o.snapshot) as [NativeRemoteSnapshot, NativeRemoteSnapshot, NativeRemoteSnapshot];
|
||||
const [baseline, pending, terminal] = observations.map(o => o.snapshot) as [CopilotRemoteSnapshot, CopilotRemoteSnapshot, CopilotRemoteSnapshot];
|
||||
fail(baseline.binding.companyId === input.scope.companyId && baseline.binding.runId === input.scope.runId, "remote observation belongs to another run");
|
||||
for (const snapshot of [baseline, pending, terminal]) {
|
||||
fail(!nativeRemoteDeniedSample(snapshot, baseline, input.scope.target, "pending").exists, "remote target changed");
|
||||
fail(!copilotRemoteDeniedSample(snapshot, baseline, input.scope.target, "pending").exists, "remote target changed");
|
||||
}
|
||||
fail(BigInt(baseline.observedMonotonicNs) < BigInt(pending.observedMonotonicNs)
|
||||
&& BigInt(pending.observedMonotonicNs) < BigInt(terminal.observedMonotonicNs), "remote observation reused or out of order");
|
||||
@@ -203,8 +204,8 @@ export function readActiveStopRemoteRetirement(input: {
|
||||
&& baseline.processes.captured && pending.processes.captured
|
||||
&& baseline.processes.live.includes(baseline.processes.root?.pid ?? -1)
|
||||
&& pending.processes.live.includes(pending.processes.root?.pid ?? -1), "remote pending lifetime unproven");
|
||||
assertNativeRemoteRetirement(terminal, baseline);
|
||||
assertNativeRemoteRetirement(terminal, pending);
|
||||
assertCopilotRemoteRetirement(terminal, baseline);
|
||||
assertCopilotRemoteRetirement(terminal, pending);
|
||||
for (const snapshot of [baseline, pending]) {
|
||||
fail(snapshot.processes.journal.every(p => terminal.processes.journal.some(q =>
|
||||
p.pid === q.pid && p.startTicks === q.startTicks && p.bootId === q.bootId)), "remote descendant journal lost");
|
||||
|
||||
@@ -4,8 +4,8 @@ import { expect, type Page } from "@playwright/test";
|
||||
import { pollUntil, type RunnerApi } from "./api.js";
|
||||
import { collectRunEvents } from "./run-observations.js";
|
||||
import { createTaskThroughUi } from "./user-actions.js";
|
||||
import { createDeniedTargetFixture, exists, observeRunProcesses } from "./native-local-fixtures.js";
|
||||
import { assertNativeRemoteRetirement, nativeRemoteDeniedSample, prepareNativeRemoteAction, type NativeRemoteBootstrap, type NativeRemoteFixture, type NativeRemoteSnapshot } from "./native-remote-evidence.js";
|
||||
import { createDeniedTargetFixture, exists, observeRunProcesses } from "./copilot-local-fixtures.js";
|
||||
import { assertCopilotRemoteRetirement, copilotRemoteDeniedSample, prepareCopilotRemoteAction, type CopilotRemoteBootstrap, type CopilotRemoteFixture, type CopilotRemoteSnapshot } from "./copilot-protection-evidence.js";
|
||||
import { cursorDeniedCommand } from "./cursor-native-evidence.js";
|
||||
import { assertActiveStopRetirement, readActiveStopCaller, observeActiveStopPending, readActiveStopSettlement, type ActiveStopRemoteObservation, type ActiveStopCaller, type ActiveStopPending, type ActiveStopScope } from "./native-active-stop-evidence.js";
|
||||
import type { BootstrapReadProof } from "./native-bootstrap-read-proof.js";
|
||||
@@ -43,7 +43,7 @@ export async function stopAtPendingPermission(input: {
|
||||
}
|
||||
export async function runNativeActiveStopFlow(input: {
|
||||
page: Page; api: RunnerApi; fixtures: LiveFixtureValues; execution: MatrixExecution; nonce: string; workspacePath: string; deadlineAt: number;
|
||||
remoteBootstrap?: NativeRemoteBootstrap;
|
||||
remoteBootstrap?: CopilotRemoteBootstrap;
|
||||
registerCleanupAssertion(callback: () => Promise<Check[]>): void;
|
||||
registerBeforeEnvironmentTeardownAssertion(callback: () => Promise<Check[]>): void;
|
||||
observe(issue: Row, runs: Row[]): void;
|
||||
@@ -51,7 +51,7 @@ export async function runNativeActiveStopFlow(input: {
|
||||
evidence(name: string, value: unknown): Promise<void>;
|
||||
}) {
|
||||
const { api, page, fixtures, execution, nonce } = input, provider = execution.profile.qualificationCandidate;
|
||||
if (provider !== "cursor" || execution.task.id !== "pending-permission-stop") throw new Error("Unknown native active Stop case");
|
||||
if ((provider !== "cursor" && provider !== "copilot") || execution.task.id !== "pending-permission-stop") throw new Error("Unknown native active Stop case");
|
||||
const remote = execution.environment.id === "daytona";
|
||||
if ((!remote && execution.environment.id !== "local") || (remote && !input.remoteBootstrap)) throw new Error("Active Stop requires an isolated admitted environment");
|
||||
const checks: Check[] = []; let issue: Row = {}, runs: Row[] = [], events: Row[] = [];
|
||||
@@ -65,15 +65,12 @@ export async function runNativeActiveStopFlow(input: {
|
||||
const remoteObservations: ActiveStopRemoteObservation[] = [];
|
||||
let retirement: ReturnType<typeof assertActiveStopRetirement> | undefined;
|
||||
const samples: Array<{ phase: string; absent: boolean }> = [];
|
||||
let fixture: NativeRemoteFixture | undefined, baseline: NativeRemoteSnapshot | undefined, sealed: NativeRemoteSnapshot | undefined;
|
||||
let fixture: CopilotRemoteFixture | undefined, baseline: CopilotRemoteSnapshot | undefined, sealed: CopilotRemoteSnapshot | undefined;
|
||||
let completed: Awaited<ReturnType<typeof stopAtPendingPermission>> | undefined;
|
||||
const observeProcesses = () => {
|
||||
// Remote ownership comes from the sandbox boot/start-tick journal. The
|
||||
// API PID transitions from the controller command to the remote runner.
|
||||
if (!observer) return;
|
||||
const run = runs[0], authority = run?.processPid ? { pid: run.processPid, groupId: run.processGroupId, startedAt: run.processStartedAt, runId: run.id } : undefined;
|
||||
if (authority) { const key = JSON.stringify(authority); if (processIdentity && processIdentity !== key) processError = true; processIdentity ??= key; }
|
||||
processes = observer.sample(authority);
|
||||
if (observer) processes = observer.sample(authority);
|
||||
};
|
||||
const load = async (): Promise<ActiveStopState> => {
|
||||
if (issue.id) issue = await api.get<Row>(`/api/issues/${issue.id}`);
|
||||
@@ -90,7 +87,7 @@ export async function runNativeActiveStopFlow(input: {
|
||||
if (phase !== "pending") throw new Error("Remote filesystem phase requires a fresh live snapshot");
|
||||
const snap = await fixture.snapshot(phase);
|
||||
remoteObservations.push({ phase, source: "live-snapshot", snapshot: snap });
|
||||
value = !nativeRemoteDeniedSample(snap, baseline, target, "pending").exists;
|
||||
value = !copilotRemoteDeniedSample(snap, baseline, target, "pending").exists;
|
||||
await input.evidence(`active-stop-${phase}-remote.json`, snap);
|
||||
} else value = !await exists(local!.targetPath);
|
||||
samples.push({ phase, absent: value }); check(`no-effect-${phase}`, value, "Exact target remains absent at the causal observation boundary");
|
||||
@@ -103,7 +100,7 @@ export async function runNativeActiveStopFlow(input: {
|
||||
await load();
|
||||
if (remote) {
|
||||
if (!fixture || !baseline) throw new Error("Remote observer was never armed");
|
||||
sealed ??= await fixture.finish(); assertNativeRemoteRetirement(sealed, baseline); processes = sealed.processes;
|
||||
sealed ??= await fixture.finish(); assertCopilotRemoteRetirement(sealed, baseline); processes = sealed.processes;
|
||||
} else if (processes.captured && processes.live.length) {
|
||||
await pollUntil({ label: "active Stop provider retirement", deadlineAt: Date.now() + 5000, intervalMs: 100, load: async () => { observeProcesses(); return processes; }, accept: p => p.live.length === 0 });
|
||||
}
|
||||
@@ -136,13 +133,13 @@ export async function runNativeActiveStopFlow(input: {
|
||||
check("explicit-per-turn-policy", configured.adapterConfig?.acpxPermissionMode === "approve-reads" && configured.adapterConfig?.lifecycleMode === "per_turn" && (provider !== "cursor" || configured.adapterConfig?.acpxSessionMode === "agent"), "Agent mode and per-turn restrictive permission policy selected before startup");
|
||||
const project = await api.post<Row>(`/api/companies/${fixtures.company.id}/projects`, { name: `Native active Stop ${nonce}`, executionWorkspacePolicy: { enabled: true, defaultMode: "shared_workspace", sharedWorkspaceConcurrency: "serialize", allowIssueOverride: false, environmentId: fixtures.environment.id, workspaceStrategy: { type: "project_primary" } }, workspace: { name: "Primary", sourceType: "local_path", cwd: input.workspacePath, isPrimary: true } });
|
||||
if (!remote) await sample("before-request");
|
||||
const createdTask = await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title: execution.task.buildTitle(nonce), prompt: remote ? input.remoteBootstrap!.prompt(nonce) : prompt(), workMode: "standard", projectName: project.name, requireExplicitTitle: true });
|
||||
issue = (await pollUntil({ label: "browser-created active Stop task", deadlineAt: input.deadlineAt, load: async () => (await api.get<Row[]>(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(i => i.id === createdTask.issueId), accept: Boolean }))!;
|
||||
await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title: execution.task.buildTitle(nonce), prompt: remote ? input.remoteBootstrap!.prompt(nonce) : prompt(), workMode: "standard", projectName: project.name });
|
||||
issue = (await pollUntil({ label: "browser-created active Stop task", deadlineAt: input.deadlineAt, load: async () => (await api.get<Row[]>(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(i => i.title === execution.task.buildTitle(nonce)), accept: Boolean }))!;
|
||||
if (remote) {
|
||||
await pollUntil({ label: "active Stop remote bootstrap", deadlineAt: input.deadlineAt, load, accept: state => state.run.status === "running" });
|
||||
const bound = await input.remoteBootstrap!.bindAndRelease({ issueId: issue.id, runId: runs[0]!.id, targets: [target], actionPrompt: async value => {
|
||||
fixture = value; if (provider === "cursor") command = cursorDeniedCommand(join(value.remoteCwd, target));
|
||||
const prepared = await prepareNativeRemoteAction({ fixture: value, companyId: fixtures.company.id, environmentId: fixtures.environment.id, runId: runs[0]!.id, target, prompt: prompt() });
|
||||
const prepared = await prepareCopilotRemoteAction({ fixture: value, companyId: fixtures.company.id, environmentId: fixtures.environment.id, runId: runs[0]!.id, target, prompt: prompt() });
|
||||
baseline = prepared.baseline;
|
||||
remoteObservations.push({ phase: "before-request", source: "live-snapshot", snapshot: baseline });
|
||||
await input.evidence("active-stop-before-request-remote.json", baseline); return prepared.prompt;
|
||||
@@ -168,7 +165,7 @@ export async function runNativeActiveStopFlow(input: {
|
||||
if (remote) {
|
||||
// finish drains the observer's automatic retirement seal. It does not
|
||||
// extend the remote watch through subsequent host UI/cleanup assertions.
|
||||
sealed = await fixture!.finish(); assertNativeRemoteRetirement(sealed, baseline!); processes = sealed.processes;
|
||||
sealed = await fixture!.finish(); assertCopilotRemoteRetirement(sealed, baseline!); processes = sealed.processes;
|
||||
remoteObservations.push({ phase: "owned-process-retirement", source: "retirement-seal", snapshot: sealed });
|
||||
await input.evidence("active-stop-owned-process-retirement-remote.json", sealed);
|
||||
} else await sample("after-stop");
|
||||
|
||||
@@ -5,6 +5,7 @@ import { stopAtPendingPermission } from "./native-active-stop-flow.js";
|
||||
import { runnerMatrix, runnerSuites, suiteDefinitionHash } from "./catalog.js";
|
||||
import { selectRunnerExecutions, parseRunnerSelectors } from "./selectors.js";
|
||||
import { createCursorToolEvidence } from "../../packages/paperclip-runner/src/drivers/acpx/cursor-tool-evidence.js";
|
||||
import { createCopilotToolEvidence } from "../../packages/paperclip-runner/src/drivers/acpx/copilot-tool-evidence.js";
|
||||
import type { CanonicalProviderEvent } from "../../packages/paperclip-runner/src/provider-events.js";
|
||||
|
||||
import { CodexSessionState } from "../../packages/paperclip-runner/src/drivers/codex/codex-session-state.js";
|
||||
@@ -13,7 +14,7 @@ import { mapTerminalTurn } from "../../packages/paperclip-runner/src/drivers/cod
|
||||
const caller = readActiveStopCaller({ deploymentMode: "local_trusted" }, { session: { userId: "local-board", id: "paperclip:local_implicit:local-board" } });
|
||||
const cancellationRequestId = "11111111-2222-4333-8444-555555555555";
|
||||
type Row = Record<string, any>;
|
||||
function fixture(provider: ActiveStopProvider = "cursor") {
|
||||
function fixture(provider: ActiveStopProvider = "copilot") {
|
||||
const scope = { provider, companyId: "company", issueId: "issue", runId: "run", target: "target.txt", ...(provider === "cursor" ? { commandSha256: `sha256:${"a".repeat(64)}` } : {}) };
|
||||
const row = (seq: number, eventType: string, payload: Row): Row => ({ companyId: "company", runId: "run", seq, eventType, protocolSchemaVersion: 1,
|
||||
payload: { prpEvent: { schema: "paperclip.prp.event.v1", schemaVersion: 1, sourceKind: "runner", eventType, runId: "run", turnId: "turn",
|
||||
@@ -81,7 +82,7 @@ function withProjectedArrivalOrder(provider: ActiveStopProvider, order: typeof a
|
||||
}
|
||||
f.events.length = 0;
|
||||
const append = (eventType: string, value: Row) => f.events.push(f.row(f.events.length + 1, eventType, value));
|
||||
const projector = (createCursorToolEvidence)({
|
||||
const projector = (provider === "cursor" ? createCursorToolEvidence : createCopilotToolEvidence)({
|
||||
sessionId: "native-session", turnId: "turn", workingDirectory: "/fixture", active: () => true,
|
||||
emit: (event: CanonicalProviderEvent) => { append(event.eventType, event.payload); },
|
||||
});
|
||||
@@ -124,10 +125,11 @@ function withSessionPrefix(provider: ActiveStopProvider = "cursor") {
|
||||
}
|
||||
|
||||
describe("definitely active native permission Stop", () => {
|
||||
describe.each(["cursor"] as const)("%s pre-Stop arrival orders", provider => {
|
||||
describe.each(["cursor", "copilot"] as const)("%s pre-Stop arrival orders", provider => {
|
||||
it.each(arrivalOrders)("accepts %s only after both exact origins exist and retains them through settlement", order => {
|
||||
const f = withProjectedArrivalOrder(provider, order), pending = f.pending();
|
||||
expect(f.nativeStageOrder()).toEqual(["tool", "permission_requested"]);
|
||||
expect(f.nativeStageOrder()).toEqual(provider === "copilot" && order === "permission-first"
|
||||
? ["permission_requested", "tool"] : ["tool", "permission_requested"]);
|
||||
expect(pending.schema).toBe("paperclip.e2e.native-active-stop-pending.v2");
|
||||
expect(pending.toolOriginRowSha256).toMatch(/^sha256:[a-f0-9]{64}$/u);
|
||||
expect(pending.toolStartedRowSha256).toMatch(/^sha256:[a-f0-9]{64}$/u);
|
||||
@@ -187,7 +189,7 @@ describe("definitely active native permission Stop", () => {
|
||||
});
|
||||
expect(f.pending).toThrow("duplicate native operation lifecycle");
|
||||
});
|
||||
it.each(["cursor"] as const)("accepts the mixed v1/v2 session prefix before strict %s pending proof", provider => {
|
||||
it.each(["cursor", "copilot"] as const)("accepts the mixed v1/v2 session prefix before strict %s pending proof", provider => {
|
||||
const f = withSessionPrefix(provider);
|
||||
expect(f.pending()).toMatchObject({ requestId: "request", toolCallId: "tool", permissionSourceSeq: 6, requestSourceSeq: 7 });
|
||||
});
|
||||
@@ -211,7 +213,7 @@ describe("definitely active native permission Stop", () => {
|
||||
delete payload(f.events.find(row => row.eventType === "runtime_request.created")!).request.details.toolCallId;
|
||||
expect(f.pending).toThrow("native notice/card identity mismatch");
|
||||
});
|
||||
it.each(["cursor"] as const)("binds %s's unanswered callback to cancelled provider settlement and caller-owned Stop", provider => {
|
||||
it.each(["cursor", "copilot"] as const)("binds %s's unanswered callback to cancelled provider settlement and caller-owned Stop", provider => {
|
||||
const f = fixture(provider), pending = f.pending(); f.settle();
|
||||
expect(f.permissionResponse.mock.calls).toEqual([[{ action: "cancel" }]]);
|
||||
expect(readActiveStopSettlement({ ...f.state(), pending, dispatchMonotonicNs: (BigInt(pending.observedMonotonicNs) + 1n).toString() })).toMatchObject({
|
||||
@@ -327,6 +329,7 @@ describe("definitely active native permission Stop", () => {
|
||||
describe("active Stop flow wiring", () => {
|
||||
it.each([
|
||||
["cursor", "tool-first"], ["cursor", "permission-first"],
|
||||
["copilot", "tool-first"], ["copilot", "permission-first"],
|
||||
] as const)("awaits retention and rechecks real %s %s evidence before issuing the single caller UUID request", async (provider, arrivalOrder) => {
|
||||
const f = withProjectedArrivalOrder(provider, arrivalOrder); const order: string[] = []; let retainedId = "";
|
||||
const stop = vi.fn(async (runId: string, id: string) => { expect(runId).toBe("run"); expect(id).toBe(retainedId); order.push("stop"); return f.settle(id); });
|
||||
@@ -367,11 +370,11 @@ describe("active Stop flow wiring", () => {
|
||||
});
|
||||
|
||||
describe("explicit active Stop discovery", () => {
|
||||
it("adds exactly two versioned cells without enabling them in --all", () => {
|
||||
it("adds exactly four versioned cells without enabling them in --all", () => {
|
||||
const suite = runnerSuites.find(s => s.id === "native-active-stop")!;
|
||||
const cells = runnerMatrix.filter(e => e.suite === suite);
|
||||
expect(cells).toHaveLength(2); expect(suite.manualOnly).toBe(true);
|
||||
expect(cells.map(e => `${e.profile.qualificationCandidate}/${e.environment.id}`).sort()).toEqual(["cursor/daytona", "cursor/local"]);
|
||||
expect(cells).toHaveLength(4); expect(suite.manualOnly).toBe(true);
|
||||
expect(cells.map(e => `${e.profile.qualificationCandidate}/${e.environment.id}`).sort()).toEqual(["copilot/daytona", "copilot/local", "cursor/daytona", "cursor/local"]);
|
||||
expect(cells.every(e => e.task.expectedRunCount === 1 && e.task.flow === "native_active_stop" && e.task.expectedTerminalState?.run === "cancelled")).toBe(true);
|
||||
expect(suite.definitionMetadata).toMatchObject({ version: 4, evidence: "paperclip.e2e.native-active-stop-settlement.v2", normalCompletionAccepted: false, providerDeath: "not-covered" });
|
||||
expect(suiteDefinitionHash(suite)).not.toBe(suiteDefinitionHash({ ...suite, definitionMetadata: { ...suite.definitionMetadata, version: 3 } }));
|
||||
|
||||
@@ -81,10 +81,12 @@ it("accepts omitted terminal path/kind only with the retained single-path origin
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["cursor"] as const)("correlates actual %s passive notices through the public reader and canonical omitted-field updates", async provider => {
|
||||
it.each(["cursor", "copilot"] as const)("correlates actual %s passive notices through the public reader and canonical omitted-field updates", async provider => {
|
||||
const { createCursorToolEvidence } = await import("../../packages/paperclip-runner/src/drivers/acpx/cursor-tool-evidence.js");
|
||||
const { createCopilotToolEvidence } = await import("../../packages/paperclip-runner/src/drivers/acpx/copilot-tool-evidence.js");
|
||||
const { readCursorToolEvidence } = await import("./cursor-native-evidence.js");
|
||||
const create = createCursorToolEvidence;
|
||||
const { readCopilotToolEvidence } = await import("./copilot-evidence.js");
|
||||
const create = provider === "cursor" ? createCursorToolEvidence : createCopilotToolEvidence;
|
||||
for (const variant of ["exact", "wrong-path", "multi-path", "ambiguous-update"]) {
|
||||
const rows: any[] = [];
|
||||
const projector = create({ sessionId: "session", turnId: "turn", workingDirectory: "/workspace", active: () => true,
|
||||
@@ -96,7 +98,7 @@ it.each(["cursor"] as const)("correlates actual %s passive notices through the p
|
||||
projector.tool({ type: "tool_call", tag: "tool_call", toolCallId: "bootstrap", kind: "read", status: "pending", rawInput: variant === "multi-path" ? { paths: [path, "private.txt"] } : { path }, locations: [{ path }] });
|
||||
projector.tool({ type: "tool_call", tag: "tool_call_update", toolCallId: "bootstrap", status: "failed", ...(variant === "ambiguous-update" ? { rawInput: { path, paths: [path, "private.txt"] } } : {}) });
|
||||
const evaluate = () => {
|
||||
const readNotices = readCursorToolEvidence(rows, "run");
|
||||
const readNotices = provider === "cursor" ? readCursorToolEvidence(rows, "run") : readCopilotToolEvidence(rows, "run");
|
||||
const started = receipt("started", "running", 2), end = receipt("completed", "failed", 4);
|
||||
Object.assign(end.payload.prpEvent.payload, { target: null, operation: "unknown" });
|
||||
return withoutProvenBootstrapReads([...readNotices, origin], origin, { actionFile, events: [...rows, started, end] });
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
|
||||
import { approvePiBootstrapRead, withoutApprovedPiBootstrapRequests, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
|
||||
import { observePiControlPending, readPiStopSettlement, readPiSteeringSettlement } from "./pi-controls-evidence.js";
|
||||
import { piCancellationId, piControlCaller, piControlFixture } from "./pi-controls-test-fixture.js";
|
||||
const actionFile = `.paperclip-eval-action-${"a".repeat(36)}.txt`;
|
||||
const digest = (value: unknown) => `sha256:${createHash("sha256").update(canonicalJson(value)).digest("hex")}`;
|
||||
function fixture() {
|
||||
const f = piControlFixture(), write = structuredClone(f.events);
|
||||
f.events.length = 0;
|
||||
const read = { schema: "paperclip.tool.execution.v1", transport: "builtin", executionId: "setup-tool", name: "read", operation: "read", target: actionFile, status: "running" };
|
||||
const request = { ...f.request, requestId: "setup-request", itemId: "setup-item", details: { toolCallId: "setup-tool" }, choices: [{ key: "accept", label: "Allow once" }, { key: "decline", label: "Deny" }] };
|
||||
f.append("tool.execution.started", read); f.append("runtime_request.created", { request });
|
||||
const finish = () => {
|
||||
f.append("runtime_request.resolved", { requestId: request.requestId, turnId: "turn", action: "accept" });
|
||||
f.append("tool.execution.completed", { ...read, status: "completed" });
|
||||
for (const row of write) f.append(row.eventType, row.payload.prpEvent.payload);
|
||||
};
|
||||
const proof = (): PiBootstrapApproval => ({ schema: "paperclip.e2e.pi-bootstrap-read-approval.v1", companyId: f.scope.companyId, issueId: f.scope.issueId, runId: f.scope.runId, actionFile, publishedSha256: `sha256:${"b".repeat(64)}`,
|
||||
requestId: "setup-request", toolCallId: "setup-tool", turnId: "turn", normalizedSessionId: "session", sourceInstanceId: "source",
|
||||
createdRowSha256: digest(f.events[1]), resolvedRowSha256: digest(f.events[2]), completedRowSha256: digest(f.events[3]) });
|
||||
return { ...f, read, finish, proof };
|
||||
}
|
||||
|
||||
describe("Pi operator setup read", () => {
|
||||
it("approves only the published owned file through the exact public request", async () => {
|
||||
const f = fixture(), saved = new Map<string, unknown>(); let posts = 0;
|
||||
const result = await approvePiBootstrapRead({
|
||||
api: { post: async (path: string, data: unknown) => { expect(path).toBe("/api/heartbeat-runs/run/runtime-requests/setup-request/resolve"); expect(data).toEqual({ turnId: "turn", requestKind: "permission_approval", resolution: { action: "accept" } }); posts++; f.finish(); } } as any,
|
||||
fixture: { actionFile, binding: { companyId: "company", runId: "run" }, snapshot: async () => ({ complete: true, setup: { path: actionFile, published: true, sha256: `sha256:${"b".repeat(64)}` } }) } as any,
|
||||
companyId: "company", issueId: "issue", runId: "run", deadlineAt: Date.now() + 2000, load: async () => f.state(), evidence: async (name, value) => { saved.set(name, value); },
|
||||
});
|
||||
expect(posts).toBe(1); expect(result).toEqual(f.proof()); expect(saved.has("pi-bootstrap-read-before-approval.json")).toBe(true);
|
||||
const pending = observePiControlPending({ ...f.state(), scope: f.scope, bootstrapApproval: result });
|
||||
expect(pending.requestId).toBe("request"); expect(pending.bootstrapApproval).toEqual(result);
|
||||
});
|
||||
|
||||
it.each(["unpublished", "foreign-binding", "other-file", "shell"])("does not approve %s setup", async failure => {
|
||||
const f = fixture(); let posts = 0;
|
||||
if (failure === "other-file") f.events[0]!.payload.prpEvent.payload.target = "private.txt";
|
||||
if (failure === "shell") { f.events[0]!.payload.prpEvent.payload.name = "bash"; f.events[0]!.payload.prpEvent.payload.operation = "execute"; }
|
||||
await expect(approvePiBootstrapRead({ api: { post: async () => { posts++; } } as any,
|
||||
fixture: { actionFile, binding: { companyId: failure === "foreign-binding" ? "other" : "company", runId: "run" }, snapshot: async () => ({ complete: true, setup: { path: actionFile, published: failure !== "unpublished", sha256: `sha256:${"b".repeat(64)}` } }) } as any,
|
||||
companyId: "company", issueId: "issue", runId: "run", deadlineAt: Date.now() + 2000, load: async () => f.state(), evidence: async () => {},
|
||||
})).rejects.toThrow("Pi bootstrap"); expect(posts).toBe(0);
|
||||
});
|
||||
|
||||
it.each(["stop", "steering"])("retains the original %s assertions after a proven setup read", mode => {
|
||||
const f = fixture(); f.finish(); const proof = f.proof();
|
||||
expect(() => observePiControlPending({ ...f.state(), scope: f.scope })).toThrow("one native permission");
|
||||
const pending = observePiControlPending({ ...f.state(), scope: f.scope, bootstrapApproval: proof });
|
||||
if (mode === "stop") { f.cancel(); expect(readPiStopSettlement({ ...f.state(), pending, caller: piControlCaller, cancellationRequestId: piCancellationId, dispatchMonotonicNs: (BigInt(pending.observedMonotonicNs) + 1n).toString() }).normalCompletionAccepted).toBe(false); }
|
||||
else { f.steer();
|
||||
// The wrapper's setup finish is separate from the original write finish.
|
||||
f.append("runtime_request.resolved", { requestId: "request", turnId: "turn", action: "decline" });
|
||||
f.append("tool.execution.completed", { ...f.tool, status: "failed", output: "Pi operation was denied or cancelled" });
|
||||
f.append("turn.completed", { status: "completed", error: null }); f.run.status = "succeeded"; f.issue.status = "done";
|
||||
f.append("run.result.accepted", { result: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", summary: f.marker } }, { sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 1, sourceEventId: "source:control:run:1" });
|
||||
f.append("run.terminal", { schema: "paperclip.prp.terminal.v1", runTerminalState: "succeeded", turnTerminalState: "completed", reportedWorkDisposition: "done" }, { sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 2, sourceEventId: "source:control:run:2" });
|
||||
expect(readPiSteeringSettlement({ ...f.state(), pending, commentId: f.commentId, queueId: f.queueId, marker: f.marker, finalMessage: f.marker }).marker).toBe(f.marker);
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["changed-receipt", "wrong-path", "failed-read", "extra-permission", "reordered", "allow-always"])("rejects %s before excluding setup records", failure => {
|
||||
const f = fixture(); f.finish(); const proof = f.proof();
|
||||
if (failure === "changed-receipt") proof.resolvedRowSha256 = `sha256:${"0".repeat(64)}`;
|
||||
if (failure === "wrong-path") f.events[0]!.payload.prpEvent.payload.target = "other.txt";
|
||||
if (failure === "failed-read") f.events[3]!.payload.prpEvent.payload.status = "failed";
|
||||
if (failure === "extra-permission") f.append("runtime_request.created", { request: { ...f.request, requestId: "extra" } });
|
||||
if (failure === "reordered") f.events[3]!.seq = 1;
|
||||
if (failure === "allow-always") f.events[2]!.payload.prpEvent.payload.action = "accept_always";
|
||||
expect(() => observePiControlPending({ ...f.state(), scope: f.scope, bootstrapApproval: proof })).toThrow();
|
||||
});
|
||||
it("does not discard any native lifecycle row", () => { const f = fixture(); f.finish(); const rows = withoutApprovedPiBootstrapRequests(f.events, f.proof()); expect(rows).toHaveLength(f.events.length - 2); expect(rows.filter((row: any) => row.eventType.startsWith("tool.execution."))).toHaveLength(3); });
|
||||
});
|
||||
@@ -0,0 +1,134 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
|
||||
import { pollUntil, type RunnerApi } from "./api.js";
|
||||
import { bootstrapReadExecutionId } from "./native-bootstrap-read-proof.js";
|
||||
import { isValidNativePrpEnvelope } from "./native-event-envelope.js";
|
||||
import { piPermissionRequests } from "./pi-native-evidence.js";
|
||||
import type { RemoteNativeFixture, RemoteNativeSnapshot } from "./remote-native-fixtures.js";
|
||||
|
||||
type Row = Record<string, any>;
|
||||
const hash = (row: unknown) => `sha256:${createHash("sha256").update(canonicalJson(row)).digest("hex")}`;
|
||||
const requireProof = (ok: unknown) => { if (!ok) throw new Error("Pi bootstrap: exact published read approval is unproven"); };
|
||||
export interface PiBootstrapApproval {
|
||||
schema: "paperclip.e2e.pi-bootstrap-read-approval.v1";
|
||||
companyId: string; issueId: string; runId: string; actionFile: string; publishedSha256: string;
|
||||
requestId: string; toolCallId: string; turnId: string; normalizedSessionId: string; sourceInstanceId: string;
|
||||
createdRowSha256: string; resolvedRowSha256: string; completedRowSha256: string;
|
||||
}
|
||||
export interface PiBootstrapState { run: Row; issue: Row; events: readonly unknown[] }
|
||||
|
||||
function readLifecycle(events: readonly unknown[], input: Pick<PiBootstrapApproval, "companyId" | "runId" | "actionFile" | "requestId">) {
|
||||
requireProof(events.length > 0 && events.length <= 20_000 && /^\.paperclip-eval-action-[a-f0-9]{36}\.txt$/u.test(input.actionFile));
|
||||
const seqs = new Set<number>(), sourceIds = new Set<string>(), lastSource = new Map<string, number>();
|
||||
for (const row of events as readonly Row[]) {
|
||||
const e = row.payload?.prpEvent; if (e === undefined) continue;
|
||||
requireProof(e && isValidNativePrpEnvelope(e, row.protocolSchemaVersion) && row.companyId === input.companyId
|
||||
&& row.runId === input.runId && e.runId === input.runId && e.eventType === row.eventType
|
||||
&& ["runner", "control_plane"].includes(e.sourceKind) && typeof e.sourceInstanceId === "string" && e.sourceInstanceId.length > 0
|
||||
&& Number.isSafeInteger(row.seq) && row.seq > 0 && !seqs.has(row.seq)
|
||||
&& Number.isSafeInteger(e.sourceSeq) && e.sourceSeq > (lastSource.get(e.sourceInstanceId) ?? 0)
|
||||
&& e.sourceEventId === `${e.sourceInstanceId}:${input.runId}:${e.sourceSeq}` && !sourceIds.has(e.sourceEventId)
|
||||
&& Number.isFinite(Date.parse(e.emittedAt)));
|
||||
seqs.add(row.seq); sourceIds.add(e.sourceEventId); lastSource.set(e.sourceInstanceId, e.sourceSeq);
|
||||
}
|
||||
const created = piPermissionRequests(events, input.runId).filter(x => x.request.requestId === input.requestId);
|
||||
requireProof(created.length === 1); const card = created[0]!;
|
||||
requireProof(isValidNativePrpEnvelope(card.event, card.row.protocolSchemaVersion) && card.row.companyId === input.companyId && card.request.requestKind === "permission_approval"
|
||||
&& typeof card.event.turnId === "string" && card.event.turnId.length > 0
|
||||
&& typeof card.event.normalizedSessionId === "string" && card.event.normalizedSessionId.length > 0
|
||||
&& card.request.choices.filter((choice: Row) => choice.key === "accept").length === 1);
|
||||
const executionId = bootstrapReadExecutionId(card.request.details.toolCallId);
|
||||
const rows = (events as readonly Row[]).filter(row => row.payload?.prpEvent?.payload?.executionId === executionId);
|
||||
requireProof(rows.length > 0 && rows.length <= 2048);
|
||||
let known = false, lastSeq = 0;
|
||||
for (const row of rows) {
|
||||
const event = row.payload.prpEvent, p = event.payload;
|
||||
requireProof(isValidNativePrpEnvelope(event, row.protocolSchemaVersion) && row.companyId === input.companyId && row.runId === input.runId && event.runId === input.runId
|
||||
&& row.protocolSchemaVersion === 1 && event.schema === "paperclip.prp.event.v1" && event.schemaVersion === 1
|
||||
&& event.sourceKind === "runner" && event.turnId === card.event.turnId && event.normalizedSessionId === card.event.normalizedSessionId
|
||||
&& event.sourceInstanceId === card.event.sourceInstanceId && event.eventType === row.eventType
|
||||
&& Number.isSafeInteger(row.seq) && row.seq > lastSeq
|
||||
&& p.schema === "paperclip.tool.execution.v1" && p.transport === "builtin" && p.name === "read" && p.operation === "read"
|
||||
&& ["tool.execution.started", "tool.execution.progressed", "tool.execution.completed"].includes(row.eventType)
|
||||
&& (p.target === input.actionFile || (!known && p.target === null && row.eventType !== "tool.execution.completed")));
|
||||
known ||= p.target === input.actionFile; lastSeq = row.seq;
|
||||
}
|
||||
requireProof(known && rows[0]!.eventType === "tool.execution.started"
|
||||
&& rows.filter(row => row.eventType === "tool.execution.started").length === 1
|
||||
&& rows.filter(row => row.eventType === "tool.execution.completed").length <= 1
|
||||
&& rows.every((row, index) => row.eventType === "tool.execution.completed"
|
||||
? index === rows.length - 1 && row.payload.prpEvent.payload.status === "completed"
|
||||
: row.payload.prpEvent.payload.status === "running"));
|
||||
return { card, rows, executionId };
|
||||
}
|
||||
|
||||
/** Remove only the two hash-bound setup permission records. All native read
|
||||
* lifecycle rows and every tested write/permission remain in the oracle. */
|
||||
export function withoutApprovedPiBootstrapRequests(events: readonly unknown[], proof?: PiBootstrapApproval): unknown[] {
|
||||
if (!proof) return [...events];
|
||||
requireProof(proof.schema === "paperclip.e2e.pi-bootstrap-read-approval.v1" && /^sha256:[a-f0-9]{64}$/u.test(proof.publishedSha256));
|
||||
const { card, rows } = readLifecycle(events, proof);
|
||||
requireProof(card.request.details.toolCallId === proof.toolCallId && card.event.turnId === proof.turnId
|
||||
&& card.event.normalizedSessionId === proof.normalizedSessionId && card.event.sourceInstanceId === proof.sourceInstanceId
|
||||
&& hash(card.row) === proof.createdRowSha256);
|
||||
const closures = (events as readonly Row[]).filter(row => ["runtime_request.resolved", "runtime_request.cancelled", "runtime_request.expired"].includes(row.eventType)
|
||||
&& row.payload?.prpEvent?.payload?.requestId === proof.requestId);
|
||||
requireProof(closures.length === 1); const closed = closures[0]!, e = closed.payload.prpEvent;
|
||||
requireProof(isValidNativePrpEnvelope(e, closed.protocolSchemaVersion) && closed.companyId === proof.companyId && closed.runId === proof.runId && closed.protocolSchemaVersion === 1
|
||||
&& e.schema === "paperclip.prp.event.v1" && e.schemaVersion === 1 && e.sourceKind === "runner" && e.runId === proof.runId
|
||||
&& e.eventType === "runtime_request.resolved" && closed.eventType === e.eventType && e.sourceInstanceId === proof.sourceInstanceId
|
||||
&& e.turnId === proof.turnId && e.normalizedSessionId === proof.normalizedSessionId && e.payload.turnId === proof.turnId
|
||||
&& e.payload.action === "accept" && closed.seq > card.row.seq && hash(closed) === proof.resolvedRowSha256);
|
||||
const completed = rows.at(-1)!;
|
||||
requireProof(completed.eventType === "tool.execution.completed" && completed.seq > closed.seq && hash(completed) === proof.completedRowSha256);
|
||||
const tested = piPermissionRequests(events, proof.runId).filter(x => x.request.requestId !== proof.requestId);
|
||||
requireProof(tested.every(x => x.row.seq > completed.seq && x.event.turnId === proof.turnId
|
||||
&& x.event.normalizedSessionId === proof.normalizedSessionId && x.event.sourceInstanceId === proof.sourceInstanceId));
|
||||
return events.filter(row => row !== card.row && row !== closed);
|
||||
}
|
||||
|
||||
/** The operator may approve one setup read after publication and observer
|
||||
* arming. Production permission policy and the tested write are untouched. */
|
||||
export async function approvePiBootstrapRead(input: {
|
||||
api: RunnerApi; fixture: { actionFile: string; binding: Pick<RemoteNativeFixture["binding"], "companyId" | "runId">; snapshot(label: string): Promise<Pick<RemoteNativeSnapshot, "complete" | "setup">> }; companyId: string; issueId: string; runId: string;
|
||||
deadlineAt: number; load(): Promise<PiBootstrapState>; evidence(name: string, value: unknown): Promise<void>;
|
||||
}): Promise<PiBootstrapApproval | undefined> {
|
||||
const actionFile = input.fixture.actionFile;
|
||||
requireProof(/^\.paperclip-eval-action-[a-f0-9]{36}\.txt$/u.test(actionFile)
|
||||
&& input.fixture.binding.companyId === input.companyId && input.fixture.binding.runId === input.runId);
|
||||
const pending = await pollUntil({ label: "Pi published bootstrap read", deadlineAt: input.deadlineAt, intervalMs: 100,
|
||||
load: input.load, reject: state => ["failed", "timed_out", "cancelled", "succeeded"].includes(state.run.status) ? "Pi bootstrap run settled" : undefined,
|
||||
accept: state => piPermissionRequests(state.events, input.runId).length > 0 });
|
||||
requireProof(pending.run.status === "running" && pending.run.runtimeMode === "native" && pending.issue.status === "in_progress" && pending.run.id === input.runId && pending.run.companyId === input.companyId && pending.run.nativeIssueId === input.issueId
|
||||
&& pending.issue.id === input.issueId && pending.issue.companyId === input.companyId);
|
||||
const requests = piPermissionRequests(pending.events, input.runId); requireProof(requests.length === 1);
|
||||
const card = requests[0]!, executionId = bootstrapReadExecutionId(card.request.details.toolCallId);
|
||||
const native = (pending.events as readonly Row[]).filter(row => row.payload?.prpEvent?.payload?.executionId === executionId);
|
||||
// A read permitted without delegation can already have led to the tested
|
||||
// write. There is no setup permission to remove in that existing path.
|
||||
if (native.some(row => row.payload.prpEvent.payload.operation === "edit")) return undefined;
|
||||
const checked = readLifecycle(pending.events, { companyId: input.companyId, runId: input.runId, actionFile, requestId: card.request.requestId });
|
||||
requireProof(!checked.rows.some(row => row.eventType === "tool.execution.completed"));
|
||||
const snapshot = await input.fixture.snapshot("bootstrap-read-published");
|
||||
requireProof(snapshot.complete && snapshot.setup.path === actionFile && snapshot.setup.published
|
||||
&& /^sha256:[a-f0-9]{64}$/u.test(snapshot.setup.sha256 ?? ""));
|
||||
await input.evidence("pi-bootstrap-read-before-approval.json", { binding: input.fixture.binding, snapshot, request: card.row, native: checked.rows });
|
||||
await input.api.post(`/api/heartbeat-runs/${input.runId}/runtime-requests/${encodeURIComponent(card.request.requestId)}/resolve`,
|
||||
{ turnId: card.event.turnId, requestKind: "permission_approval", resolution: { action: "accept" } });
|
||||
const finished = await pollUntil({ label: "Pi exact bootstrap read completion", deadlineAt: input.deadlineAt, intervalMs: 100, load: input.load,
|
||||
reject: state => ["failed", "timed_out", "cancelled", "succeeded"].includes(state.run.status) ? "Pi bootstrap run settled" : undefined,
|
||||
accept: state => (state.events as readonly Row[]).some(row => row.eventType === "tool.execution.completed"
|
||||
&& row.payload?.prpEvent?.payload?.executionId === executionId) });
|
||||
const closure = (finished.events as readonly Row[]).find(row => row.eventType === "runtime_request.resolved" && row.payload?.prpEvent?.payload?.requestId === card.request.requestId);
|
||||
const completed = (finished.events as readonly Row[]).find(row => row.eventType === "tool.execution.completed" && row.payload?.prpEvent?.payload?.executionId === executionId);
|
||||
requireProof(closure && completed);
|
||||
const after = await input.fixture.snapshot("bootstrap-read-completed");
|
||||
requireProof(after.complete && after.setup.path === actionFile && after.setup.published && after.setup.sha256 === snapshot.setup.sha256);
|
||||
await input.evidence("pi-bootstrap-read-completed.json", { binding: input.fixture.binding, snapshot: after });
|
||||
const proof: PiBootstrapApproval = { schema: "paperclip.e2e.pi-bootstrap-read-approval.v1", companyId: input.companyId, issueId: input.issueId, runId: input.runId,
|
||||
actionFile, publishedSha256: snapshot.setup.sha256!, requestId: card.request.requestId, toolCallId: card.request.details.toolCallId,
|
||||
turnId: card.event.turnId, normalizedSessionId: card.event.normalizedSessionId, sourceInstanceId: card.event.sourceInstanceId,
|
||||
createdRowSha256: hash(card.row), resolvedRowSha256: hash(closure), completedRowSha256: hash(completed) };
|
||||
withoutApprovedPiBootstrapRequests(finished.events, proof);
|
||||
await input.evidence("pi-bootstrap-read-approval.json", proof); return proof;
|
||||
}
|
||||
@@ -16,8 +16,35 @@ function cancelled() {
|
||||
const binding = { pending, caller: piControlCaller, cancellationRequestId: piCancellationId, dispatchMonotonicNs: String(BigInt(pending.observedMonotonicNs) + 1n) };
|
||||
return { f, binding, read: () => readPiStopSettlement({ ...f.state(), ...binding }) };
|
||||
}
|
||||
function streamedWrite() {
|
||||
const f = piControlFixture();
|
||||
f.events.pop();
|
||||
event(f.events[0]!).payload = { ...f.tool, target: null, inputUpdated: false };
|
||||
f.append("tool.execution.progressed", { ...f.tool, target: null, inputUpdated: true });
|
||||
f.append("tool.execution.progressed", { ...f.tool, inputUpdated: true });
|
||||
f.append("runtime_request.created", { request: f.request });
|
||||
return f;
|
||||
}
|
||||
describe("Pi pending control identity", () => {
|
||||
it("accepts Pi native permission/start without invented provider notices", () => expect(piControlFixture().pending()).toMatchObject({ toolCallId: "pi-tool", executionId: "pi-tool", turnId: "turn" }));
|
||||
it("accepts streamed arguments only after the same execution proves its exact path", () => {
|
||||
const f = streamedWrite();
|
||||
expect(f.pending()).toMatchObject({ startedSourceSeq: 1, requestSourceSeq: 4 });
|
||||
});
|
||||
it.each([
|
||||
["no complete target", (f: ReturnType<typeof streamedWrite>) => { event(f.events[2]!).payload.target = null; }],
|
||||
["conflicting partial target", f => { event(f.events[1]!).payload.target = "other.txt"; }],
|
||||
["target lost after admission", f => { f.append("tool.execution.progressed", { ...f.tool, target: null }); }],
|
||||
["different execution supplied target", f => { event(f.events[2]!).payload.executionId = "other"; }],
|
||||
["terminal before target", f => { f.events[1]!.eventType = event(f.events[1]!).eventType = "tool.execution.completed"; event(f.events[1]!).payload.status = "failed"; }],
|
||||
] satisfies Array<[string, (f: ReturnType<typeof streamedWrite>) => void]>)("rejects streamed write with %s", (_name, mutate) => {
|
||||
const f = streamedWrite(); mutate(f); expect(() => f.pending()).toThrow();
|
||||
});
|
||||
it("retains the partial start hash through actual callback cancellation", () => {
|
||||
const f = streamedWrite(), pending = f.pending(); f.cancel();
|
||||
expect(readPiStopSettlement({ ...f.state(), pending, caller: piControlCaller, cancellationRequestId: piCancellationId,
|
||||
dispatchMonotonicNs: String(BigInt(pending.observedMonotonicNs) + 1n) })).toMatchObject({ normalCompletionAccepted: false });
|
||||
});
|
||||
it("admits permission-first ACP only after both canonical boundaries exist", () => {
|
||||
const f = piControlFixture(); f.events.reverse();
|
||||
f.events.forEach((r, i) => { r.seq = event(r).sourceSeq = i + 1; event(r).sourceEventId = `source:run:${i + 1}`; });
|
||||
@@ -84,6 +111,35 @@ function steered() {
|
||||
return { f, binding, ack, read: () => readPiSteeringSettlement({ ...f.state(), ...binding }) };
|
||||
}
|
||||
describe("Pi same-turn steering", () => {
|
||||
function withControlSettlement() {
|
||||
return steered();
|
||||
}
|
||||
it("keeps exact runner proof when the control plane records result and terminal events", () => {
|
||||
expect(withControlSettlement().read()).toMatchObject({ nativeFollowUpTested: false });
|
||||
});
|
||||
it.each([
|
||||
["foreign control producer", (s: ReturnType<typeof withControlSettlement>) => { event(s.f.events.at(-1)!).sourceInstanceId = "foreign:control"; event(s.f.events.at(-1)!).sourceEventId = "foreign:control:run:2"; }],
|
||||
["foreign control turn", s => { event(s.f.events.at(-1)!).turnId = "foreign"; }],
|
||||
["foreign control session", s => { event(s.f.events.at(-1)!).normalizedSessionId = "foreign"; }],
|
||||
["unknown control event", s => { s.f.events.at(-1)!.eventType = event(s.f.events.at(-1)!).eventType = "runtime_request.resolved"; }],
|
||||
["reordered control sequence", s => { event(s.f.events.at(-1)!).sourceSeq = 1; event(s.f.events.at(-1)!).sourceEventId = "source:control:run:1"; }],
|
||||
["duplicate control terminal", s => { s.f.append("run.terminal", { schema: "paperclip.prp.terminal.v1" }, { sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 3, sourceEventId: "source:control:run:3" }); }],
|
||||
["unbound control result", s => { event(s.f.events.at(-2)!).payload.result.schema = "foreign"; }],
|
||||
["both control records missing", s => { s.f.events.splice(-2); }],
|
||||
["accepted result missing", s => { s.f.events.splice(-2, 1); }],
|
||||
["control terminal missing", s => { s.f.events.pop(); }],
|
||||
["wrong control summary", s => { event(s.f.events.at(-2)!).payload.result.summary = "Finished"; }],
|
||||
["unfinished control result", s => { event(s.f.events.at(-2)!).payload.result.reportedWorkDisposition = "in_progress"; }],
|
||||
["failed control terminal", s => { event(s.f.events.at(-1)!).payload.runTerminalState = "failed"; }],
|
||||
["interrupted control turn", s => { event(s.f.events.at(-1)!).payload.turnTerminalState = "interrupted"; }],
|
||||
["unfinished control terminal", s => { event(s.f.events.at(-1)!).payload.reportedWorkDisposition = "in_progress"; }],
|
||||
["control terminal before runner completion", s => {
|
||||
for (const row of s.f.events.slice(3, 6)) row.seq += 2;
|
||||
s.f.events.at(-2)!.seq = 4; s.f.events.at(-1)!.seq = 5;
|
||||
}],
|
||||
] satisfies Array<[string, (s: ReturnType<typeof withControlSettlement>) => void]>)("rejects %s", (_name, mutate) => {
|
||||
const s = withControlSettlement(); mutate(s); expect(s.read).toThrow();
|
||||
});
|
||||
it("calibrates against the actual Product ACP facade producer, not Rust's raw transport echo", async () => {
|
||||
const f = piControlFixture(), pending = f.pending(), calls: Row[] = [];
|
||||
// Only the transport and event sink are doubles. Run the actual public
|
||||
@@ -124,23 +180,30 @@ describe("Pi controls catalog admission", () => {
|
||||
expect(cells).toHaveLength(4); expect(cells.every(c => c.profile.qualificationCandidate === "pi" && c.task.expectedRunCount === 1)).toBe(true);
|
||||
expect(cells.map(c => `${c.environment.id}/${c.task.id}`).sort()).toEqual(["daytona/pending-permission-stop", "daytona/same-turn-steering", "local/pending-permission-stop", "local/same-turn-steering"]);
|
||||
expect(selectRunnerExecutions(parseRunnerSelectors(["--all"])).some(c => c.suite.id === "pi-controls")).toBe(false);
|
||||
expect(validateRunnerCatalog()).toHaveLength(469);
|
||||
for (const cell of cells) expect(JSON.parse(buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION!)).toEqual([{ agent: "pi", model: cell.profile.model }]);
|
||||
expect(validateRunnerCatalog()).toHaveLength(722);
|
||||
for (const cell of cells) expect(buildRunnerE2EProcessEnvironment({}, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeUndefined();
|
||||
expect(() => assertRemoteNativeEvidencePrerequisites(cells, {})).toThrow();
|
||||
});
|
||||
it("pins the Pi 1 profile and versioned coverage while retaining active Stop identity", () => {
|
||||
// Pi 1/profile 12 changes profile-bearing definitions. Coverage v4/v2 adds
|
||||
// provider death, pending restart and the strict file oracle; no runtime admission is promoted.
|
||||
// Current profile identity and native/control coverage retain their own fingerprints.
|
||||
// Extended v3 states the strict file oracle's task-wide Bash limit.
|
||||
const pi = runnerMatrix.find(c => c.profile.qualificationCandidate === "pi")!.profile;
|
||||
expect(pi.modelQualification?.qualificationId).toBe("pi:0.0.33:1.0.0:openrouter");
|
||||
expect(runnerSuites.find(s => s.id === "pi-native")!.definitionMetadata).toMatchObject({ version: 4, profileVersion: 12 });
|
||||
expect(runnerSuites.find(s => s.id === "extended-harnesses")!.definitionMetadata).toMatchObject({ version: 2 });
|
||||
expect(runnerSuites.find(s => s.id === "pi-native")!.definitionMetadata).toMatchObject({ version: 23, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", agentMemoryParent: "public-managed-file-seed-before-admission", incompleteTerminalCleanup: "retirement-retained-with-failed-watch", profileVersion: 19, agentMemoryContent: "utf8-nonce-plus-final-lf", agentMemoryPrompt: "single-json-write-and-content-bound-native-read-both-runs", agentMemoryReadAuthority: "local-withheld-or-exact-remote-agent-run-file", taskPromptTransport: "fenced-markdown-paste-and-multiline-literal-escapes", nativeFinish: "current-contract-objective-evidence-refs", providerFaultExecutable: "stable-preinstalled-runner-link-and-snapshot-node-inode-with-held-bootstrap-fd-3-or-7" });
|
||||
expect(runnerSuites.find(s => s.id === "pi-controls")!.definitionMetadata).toMatchObject({ version: 10, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", profileVersion: 19,
|
||||
remoteProcessIdentity: "observer-pid-startTicks-bootId",
|
||||
controlPlaneSettlement: "required-scoped-result-and-terminal-after-runner" });
|
||||
expect(runnerSuites.find(s => s.id === "extended-harnesses")!.definitionMetadata).toMatchObject({ version: 5, piFileArtifactTitle: "exact-filename", piFileCommandTransport: "fenced-bash-markdown-paste" });
|
||||
for (const cell of runnerMatrix.filter(cell => cell.profile.qualificationCandidate === "pi")) {
|
||||
const agent = cell.profile.buildAgent({ environmentId: "environment", environmentFixtureId: cell.environment.id, workspacePath: "/workspace", executionId: cell.id, secretRefs: { OPENROUTER_API_KEY: { type: "secret_ref", secretId: "synthetic", version: "latest" } } });
|
||||
expect(agent.adapterConfig).toMatchObject({ piThinkingLevel: "low" });
|
||||
}
|
||||
const hashes = Object.fromEntries(runnerSuites.filter(s => ["pi-native", "native-active-stop", "extended-harnesses", "rich-acp-warm-continuity"].includes(s.id)).map(s => [s.id, suiteDefinitionHash(s)]));
|
||||
expect(hashes).toEqual({
|
||||
"pi-native": "eec3b2c140d81561e69c13c1b193b66f7790af0cfc90bbed76c57af2d1e3cdde",
|
||||
"native-active-stop": "99682b2b106d816a011834fae5a944ed7729958893709d5b83a19b6f595e7e4d",
|
||||
"rich-acp-warm-continuity": "25e69f031696aeb459e6722a056ae0802379f9137799648fdd1945881f32f41a",
|
||||
"extended-harnesses": "e50f79cf604bffb2cdee3ad4e8bc1d106045a4f44ef07e877032ff5b8960c8ab",
|
||||
"pi-native": "90f2e901d7f39bfb6bbadcf63bc78e410a3b071af980f3a23979747e3d5f2021",
|
||||
"native-active-stop": "2d4fdeeb75bd531b309bd1b35cfa5680ceefdeee6b7155b068dd011ce9901980",
|
||||
"rich-acp-warm-continuity": "331b88d9538a132670789fb7864df7df5f4bf294ae19b62f53d8a02f901774f0",
|
||||
"extended-harnesses": "5de4fac78d4d581bc0954f07b5cf2df2862d37f8b0205325eede6d9ef6d9f5e6",
|
||||
});
|
||||
expect(runnerMatrix.filter(c => c.profile.qualificationCandidate === "pi" && c.suite.id !== "pi-controls")).toHaveLength(22);
|
||||
});
|
||||
|
||||
@@ -2,14 +2,15 @@ import { createHash } from "node:crypto";
|
||||
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
|
||||
import { hasAcpxNativeOrigin } from "./acpx-native-origin.js";
|
||||
import { bootstrapReadExecutionId } from "./native-bootstrap-read-proof.js";
|
||||
import { withoutApprovedPiBootstrapRequests, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
|
||||
import { isValidNativePrpEnvelope } from "./native-event-envelope.js";
|
||||
import type { ActiveStopCaller } from "./native-active-stop-evidence.js";
|
||||
|
||||
type Row = Record<string, any>;
|
||||
export interface PiControlScope { companyId: string; issueId: string; runId: string; target: string }
|
||||
export interface PiControlState { run: Row; issue: Row; events: readonly unknown[] }
|
||||
export interface PiControlState { run: Row; issue: Row; events: readonly unknown[]; bootstrapApproval?: PiBootstrapApproval }
|
||||
export interface PiControlPending {
|
||||
schema: "paperclip.e2e.pi-control-pending.v1"; scope: PiControlScope; requestId: string; toolCallId: string; executionId: string;
|
||||
schema: "paperclip.e2e.pi-control-pending.v1"; bootstrapApproval?: PiBootstrapApproval; scope: PiControlScope; requestId: string; toolCallId: string; executionId: string;
|
||||
turnId: string; normalizedSessionId: string; sourceInstanceId: string; itemId: string;
|
||||
requestSourceSeq: number; startedSourceSeq: number; requestRowSha256: string; startedRowSha256: string; observedMonotonicNs: string;
|
||||
}
|
||||
@@ -19,20 +20,26 @@ const hash = (v: unknown) => `sha256:${createHash("sha256").update(canonicalJson
|
||||
function requireProof(value: unknown, reason: string): asserts value { if (!value) throw new Error(`Pi controls: ${reason}`); }
|
||||
const terminalTypes = new Set(["turn.completed", "turn.failed", "turn.cancelled", "turn.interrupted"]);
|
||||
const closureTypes = new Set(["runtime_request.resolved", "runtime_request.cancelled", "runtime_request.expired"]);
|
||||
const controlSettlementTypes = new Set(["run.result.accepted", "run.terminal"]);
|
||||
|
||||
/** Consume the actual Product projection. Pi does not emit Cursor/Copilot native
|
||||
* diagnostic notices; never manufacture those to reuse another provider's oracle. */
|
||||
function origin(events: readonly unknown[], scope: PiControlScope) {
|
||||
function origin(events: readonly unknown[], scope: PiControlScope, bootstrapApproval?: PiBootstrapApproval) {
|
||||
if (bootstrapApproval) requireProof(bootstrapApproval.companyId === scope.companyId && bootstrapApproval.issueId === scope.issueId && bootstrapApproval.runId === scope.runId, "foreign bootstrap approval");
|
||||
const gradingRows = new Set(withoutApprovedPiBootstrapRequests(events, bootstrapApproval));
|
||||
requireProof(Object.values(scope).every(id) && events.length > 0 && events.length <= 20_000, "bounded exact scope required");
|
||||
const rows = events.map(rec).filter(row => rec(row.payload).prpEvent !== undefined).map(row => ({ row, event: rec(rec(row.payload).prpEvent) })).sort((a, b) => a.row.seq - b.row.seq);
|
||||
const projected = events.map(rec).filter(row => rec(row.payload).prpEvent !== undefined).map(row => ({ row, event: rec(rec(row.payload).prpEvent) })).sort((a, b) => a.row.seq - b.row.seq);
|
||||
const seqs = new Set<number>(), sourceIds = new Set<string>(), last = new Map<string, number>();
|
||||
for (const { row, event: e } of rows) {
|
||||
for (const { row, event: e } of projected) {
|
||||
requireProof(row.companyId === scope.companyId && row.runId === scope.runId && isValidNativePrpEnvelope(e, row.protocolSchemaVersion)
|
||||
&& e.sourceKind === "runner" && e.runId === scope.runId && e.eventType === row.eventType && Number.isSafeInteger(row.seq) && row.seq > 0 && !seqs.has(row.seq)
|
||||
&& (e.sourceKind === "runner" || (e.sourceKind === "control_plane" && controlSettlementTypes.has(e.eventType)))
|
||||
&& e.runId === scope.runId && e.eventType === row.eventType && Number.isSafeInteger(row.seq) && row.seq > 0 && !seqs.has(row.seq)
|
||||
&& id(e.sourceInstanceId) && Number.isSafeInteger(e.sourceSeq) && e.sourceSeq > (last.get(e.sourceInstanceId) ?? 0)
|
||||
&& e.sourceEventId === `${e.sourceInstanceId}:${e.runId}:${e.sourceSeq}` && !sourceIds.has(e.sourceEventId), "foreign, duplicate or reordered event");
|
||||
seqs.add(row.seq); sourceIds.add(e.sourceEventId); last.set(e.sourceInstanceId, e.sourceSeq);
|
||||
}
|
||||
const rows = projected.filter(x => x.event.sourceKind === "runner" && gradingRows.has(x.row));
|
||||
const control = projected.filter(x => x.event.sourceKind === "control_plane");
|
||||
const created = rows.filter(x => x.event.eventType === "runtime_request.created");
|
||||
requireProof(created.length === 1, "one native permission required");
|
||||
const card = created[0]!, request = rec(rec(card.event.payload).request);
|
||||
@@ -42,36 +49,54 @@ function origin(events: readonly unknown[], scope: PiControlScope) {
|
||||
&& Array.isArray(request.choices) && request.choices.some((c: Row) => c.key === "decline"), "native permission identity missing");
|
||||
const stream = (event: Row) => event.turnId === card.event.turnId && event.normalizedSessionId === card.event.normalizedSessionId && event.sourceInstanceId === card.event.sourceInstanceId;
|
||||
requireProof(rows.filter(x => x.event.turnId != null).every(x => stream(x.event)), "foreign turn, session or producer");
|
||||
// Product settlement adds a separate control-plane producer after the runner
|
||||
// terminal. Validate those records without using them as native tool proof.
|
||||
const terminals = rows.filter(x => terminalTypes.has(x.event.eventType));
|
||||
requireProof(control.length <= 2 && new Set(control.map(x => x.event.eventType)).size === control.length
|
||||
&& control.every(x => terminals.length === 1 && x.row.seq > terminals[0]!.row.seq
|
||||
&& x.event.sourceInstanceId === `${card.event.sourceInstanceId}:control`
|
||||
&& x.event.turnId === card.event.turnId && x.event.normalizedSessionId === card.event.normalizedSessionId
|
||||
&& (x.event.eventType === "run.result.accepted"
|
||||
? rec(rec(x.event.payload).result).schema === "paperclip.run_result.v1"
|
||||
: rec(x.event.payload).schema === "paperclip.prp.terminal.v1"))
|
||||
&& (control.length !== 2 || (control[0]!.event.eventType === "run.result.accepted" && control[1]!.event.eventType === "run.terminal")),
|
||||
"foreign, duplicate or premature control-plane settlement");
|
||||
const executionId = bootstrapReadExecutionId(request.details.toolCallId);
|
||||
const native = rows.filter(x => x.event.eventType.startsWith("tool.execution.") && rec(x.event.payload).transport === "builtin");
|
||||
const write = native.filter(x => rec(x.event.payload).executionId === executionId);
|
||||
const started = write.filter(x => x.event.eventType === "tool.execution.started");
|
||||
// Pi streams native tool arguments. The start and early progress rows can
|
||||
// have no path yet; the pending permission is admissible only after this
|
||||
// same execution provides the exact target. Conflicting paths and loss of a
|
||||
// previously known path still fail, including after denial or cancellation.
|
||||
const targetIndex = write.findIndex(x => x.event.payload.target === scope.target);
|
||||
requireProof(started.length === 1 && write.filter(x => x.event.eventType === "tool.execution.completed").length <= 1
|
||||
&& write[0] === started[0]
|
||||
&& write.every(x => ["tool.execution.started", "tool.execution.progressed", "tool.execution.completed"].includes(x.event.eventType)
|
||||
&& write[0] === started[0] && targetIndex >= 0
|
||||
&& write.every((x, index) => ["tool.execution.started", "tool.execution.progressed", "tool.execution.completed"].includes(x.event.eventType)
|
||||
&& x.event.payload.schema === "paperclip.tool.execution.v1" && x.event.payload.name === "write" && x.event.payload.operation === "edit"
|
||||
&& x.event.payload.target === scope.target && x.event.payload.status === (x.event.eventType === "tool.execution.completed" ? "failed" : "running")), "exact native write lifecycle missing");
|
||||
&& (x.event.payload.target === scope.target || (index < targetIndex && x.event.payload.target === null && x.event.eventType !== "tool.execution.completed"))
|
||||
&& x.event.payload.status === (x.event.eventType === "tool.execution.completed" ? "failed" : "running")), "exact native write lifecycle missing");
|
||||
requireProof(!write.some((x, index) => x.event.eventType === "tool.execution.completed" && index !== write.length - 1), "write activity after terminal");
|
||||
// Read-only orientation/bootstrap can precede the write. Never exempt another
|
||||
// edit, shell execution, or unknown native operation as an alleged bootstrap.
|
||||
requireProof(native.every(x => rec(x.event.payload).executionId === executionId || (x.event.payload.operation === "read" && x.row.seq < Math.min(card.row.seq, started[0]!.row.seq))), "extra native operation");
|
||||
return { rows, card, request, started: started[0]!, executionId, stream };
|
||||
return { rows, control, card, request, started: started[0]!, executionId, stream };
|
||||
}
|
||||
export function observePiControlPending(input: PiControlState & { scope: PiControlScope }): PiControlPending {
|
||||
const { run, issue, scope } = input, p = origin(input.events, scope);
|
||||
const { run, issue, scope } = input, p = origin(input.events, scope, input.bootstrapApproval);
|
||||
requireProof(run.id === scope.runId && run.companyId === scope.companyId && run.nativeIssueId === scope.issueId && run.status === "running" && run.runtimeMode === "native"
|
||||
&& issue.id === scope.issueId && issue.companyId === scope.companyId && issue.status === "in_progress"
|
||||
&& run.resultJson?.startupCancellation == null && run.resultJson?.nativeCancellation == null, "run is not fresh active work");
|
||||
requireProof(!p.rows.some(x => terminalTypes.has(x.event.eventType) || closureTypes.has(x.event.eventType)
|
||||
|| (x.event.eventType === "tool.execution.completed" && x.event.payload.executionId === p.executionId)), "permission already answered or provider settled");
|
||||
return { schema: "paperclip.e2e.pi-control-pending.v1", scope, requestId: p.request.requestId, toolCallId: p.request.details.toolCallId, executionId: p.executionId,
|
||||
return { schema: "paperclip.e2e.pi-control-pending.v1", ...(input.bootstrapApproval ? { bootstrapApproval: input.bootstrapApproval } : {}), scope, requestId: p.request.requestId, toolCallId: p.request.details.toolCallId, executionId: p.executionId,
|
||||
turnId: p.card.event.turnId, normalizedSessionId: p.card.event.normalizedSessionId, sourceInstanceId: p.card.event.sourceInstanceId, itemId: p.request.itemId,
|
||||
requestSourceSeq: p.card.event.sourceSeq, startedSourceSeq: p.started.event.sourceSeq, requestRowSha256: hash(p.card.row), startedRowSha256: hash(p.started.row), observedMonotonicNs: process.hrtime.bigint().toString() };
|
||||
}
|
||||
function retained(input: PiControlState & { pending: PiControlPending }) {
|
||||
const b = input.pending;
|
||||
requireProof(b.schema === "paperclip.e2e.pi-control-pending.v1", "pending receipt missing");
|
||||
const p = origin(input.events, b.scope);
|
||||
const p = origin(input.events, b.scope, b.bootstrapApproval);
|
||||
requireProof(hash(p.card.row) === b.requestRowSha256 && hash(p.started.row) === b.startedRowSha256 && p.card.event.sourceSeq === b.requestSourceSeq && p.started.event.sourceSeq === b.startedSourceSeq
|
||||
&& p.request.requestId === b.requestId && p.request.details.toolCallId === b.toolCallId && p.executionId === b.executionId && p.request.itemId === b.itemId
|
||||
&& p.card.event.turnId === b.turnId && p.card.event.normalizedSessionId === b.normalizedSessionId && p.card.event.sourceInstanceId === b.sourceInstanceId, "retained pending identity changed");
|
||||
@@ -116,6 +141,11 @@ export function readPiSteeringAcknowledgement(input: PiControlState & { pending:
|
||||
}
|
||||
export function readPiSteeringSettlement(input: PiControlState & { pending: PiControlPending; commentId: string; queueId: string; marker: string; finalMessage: string }) {
|
||||
const ack = readPiSteeringAcknowledgement(input), p = retained(input), b = input.pending;
|
||||
const accepted = rec(rec(p.control[0]?.event.payload).result), terminalResult = rec(p.control[1]?.event.payload);
|
||||
requireProof(p.control.length === 2 && p.control[0]!.event.eventType === "run.result.accepted" && p.control[1]!.event.eventType === "run.terminal"
|
||||
&& accepted.reportedWorkDisposition === "done" && accepted.summary === input.marker
|
||||
&& terminalResult.runTerminalState === "succeeded" && terminalResult.turnTerminalState === "completed" && terminalResult.reportedWorkDisposition === "done",
|
||||
"complete matching control-plane settlement required");
|
||||
const closed = p.rows.filter(x => closureTypes.has(x.event.eventType)), terminal = p.rows.filter(x => terminalTypes.has(x.event.eventType));
|
||||
const failed = p.rows.filter(x => x.event.eventType === "tool.execution.completed" && x.event.payload.executionId === b.executionId);
|
||||
requireProof(closed.length === 1 && closed[0]!.event.eventType === "runtime_request.resolved" && closed[0]!.event.payload.requestId === b.requestId && closed[0]!.event.payload.turnId === b.turnId
|
||||
|
||||
@@ -8,10 +8,12 @@ import { runPiControlsFlow } from "./pi-controls-flow.js";
|
||||
import { persistedFinalRunMessage } from "./matchers.js";
|
||||
import { readPiSteeringSettlement } from "./pi-controls-evidence.js";
|
||||
|
||||
const harness = vi.hoisted(() => ({ target: "", prompt: "", message: "", mutation: false, incomplete: false }));
|
||||
vi.mock("./pi-bootstrap-permission.js", async importOriginal => ({ ...await importOriginal<typeof import("./pi-bootstrap-permission.js")>(), approvePiBootstrapRead: async () => undefined }));
|
||||
|
||||
const harness = vi.hoisted(() => ({ target: "", prompt: "", message: "", foreignCreatedTask: false, mutation: false, incomplete: false }));
|
||||
vi.mock("./user-actions.js", () => ({
|
||||
createTaskThroughUi: async (input: { prompt: string }) => { harness.prompt = input.prompt; },
|
||||
submitTaskReply: async (_page: unknown, body: string) => { harness.message = body; return Date.now(); },
|
||||
createTaskThroughUi: async (input: { prompt: string; requireExplicitTitle?: boolean }) => { expect(input.requireExplicitTitle).toBe(true); harness.prompt = input.prompt; return { submittedAtMs: Date.now(), issueId: harness.foreignCreatedTask ? "foreign-created-task" : "issue" }; },
|
||||
submitTaskReply: async (page: { submitReply(body: string): void }, body: string) => { page.submitReply(body); return Date.now(); },
|
||||
}));
|
||||
vi.mock("./copilot-local-fixtures.js", async importOriginal => {
|
||||
const actual = await importOriginal<typeof import("./copilot-local-fixtures.js")>();
|
||||
@@ -26,8 +28,8 @@ vi.mock("@playwright/test", () => ({ expect: (actual: any, message?: string) =>
|
||||
toHaveCount: async (value: number) => { if (typeof actual.count === "function") expect(actual.count()).toBe(value); },
|
||||
}) }));
|
||||
|
||||
async function exercise(taskId: string, remote: boolean, failure?: "mutation" | "incomplete" | "missing-ack" | "missing-comment" | "foreign-comment") {
|
||||
harness.target = ""; harness.prompt = ""; harness.message = ""; harness.mutation = failure === "mutation"; harness.incomplete = failure === "incomplete";
|
||||
async function exercise(taskId: string, remote: boolean, failure?: "mutation" | "incomplete" | "missing-ack" | "missing-comment" | "foreign-comment" | "foreign-queued-body" | "steer-rejected" | "duplicate-permission" | "annotation-drift" | "root-rotation" | "foreign-created-task") {
|
||||
harness.foreignCreatedTask = failure === "foreign-created-task"; harness.target = ""; harness.prompt = ""; harness.message = ""; harness.mutation = failure === "mutation"; harness.incomplete = failure === "incomplete";
|
||||
const task = piControlTasks.find(t => t.id === taskId)!, stopCase = taskId === "pending-permission-stop";
|
||||
const workspacePath = await mkdtemp(join(tmpdir(), "pi-controls-fixture-"));
|
||||
const saved = new Map<string, any>(), localCleanup: Array<() => Promise<any>> = [], remoteCleanup: Array<() => Promise<any>> = [];
|
||||
@@ -37,9 +39,9 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
const publicComments: any[][] = [];
|
||||
const sync = () => {
|
||||
const target = remote ? "pi-control-fixture.txt" : harness.target;
|
||||
f.scope.target = target; f.tool.target = target; f.issue.title = task.buildTitle("fixture");
|
||||
f.scope.target = target; f.tool.target = target; f.issue.title = "Provider-generated task name"; f.issue.companyId = "company"; f.issue.assigneeAgentId = "agent";
|
||||
};
|
||||
const queue = () => ({ queueId: "queue", targetRunId: "run", revision: "revision", protocol: "paperclip_runner_v1", steeringDisposition: "available", entries: harness.message ? [{ comment: { id: "comment", body: harness.message } }] : [] });
|
||||
const queue = () => ({ queueId: "queue", targetRunId: "run", revision: "revision", protocol: "paperclip_runner_v1", steeringDisposition: "available", entries: harness.message ? [{ comment: { id: "comment", body: harness.message + (failure === "foreign-queued-body" ? " altered" : "") } }] : [] });
|
||||
const api = {
|
||||
post: async (path: string, body: any) => {
|
||||
if (path.endsWith("/projects")) return { name: "Pi controls fixture" };
|
||||
@@ -55,7 +57,10 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
if (path.endsWith("/issues?limit=100")) return [f.issue];
|
||||
if (path === "/api/issues/issue") return f.issue;
|
||||
if (path.endsWith("/heartbeat-runs?limit=100")) return [f.run];
|
||||
if (path === "/api/heartbeat-runs/run") return f.run;
|
||||
if (path === "/api/heartbeat-runs/run") {
|
||||
if (failure === "annotation-drift") f.run.processStartedAt = f.run.status === "running" ? "2026-10-01T00:00:11Z" : "2026-10-01T00:00:00Z";
|
||||
return f.run;
|
||||
}
|
||||
if (path.includes("/events?")) return f.events;
|
||||
if (path.endsWith("/queued-comments")) return queue();
|
||||
if (path.endsWith("/comments")) {
|
||||
@@ -77,13 +82,21 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
};
|
||||
const waiters: Array<{ predicate: (request: any) => boolean; resolve: (request: any) => void }> = [];
|
||||
function postBrowser(path: string, body: unknown) {
|
||||
const request = { url: () => `http://fixture${path}`, method: () => "POST", postDataJSON: () => body };
|
||||
const rejected = failure === "steer-rejected" && path.endsWith("/steer");
|
||||
const request = { url: () => `http://fixture${path}`, method: () => "POST", postDataJSON: () => body,
|
||||
response: async () => ({ ok: () => !rejected, status: () => rejected ? 409 : 200 }) };
|
||||
const waiter = waiters.find(w => w.predicate(request)); if (!waiter) throw new Error("Browser request was not awaited");
|
||||
waiter.resolve(request); waiters.splice(waiters.indexOf(waiter), 1);
|
||||
}
|
||||
const locator = (kind = "other"): any => ({
|
||||
filter: () => locator(kind), last: () => locator(kind), getByText: () => locator(),
|
||||
count: () => kind === "loading" ? 0 : kind === "card" ? 1 : kind === "deny" ? (f.run.status === "running" ? 1 : 0) : 1,
|
||||
const locator = (kind = "other", actionable = false): any => ({
|
||||
filter: (options: { has?: { kind: string } }) => {
|
||||
if (kind === "card" && options.has) expect(options.has.kind).toBe("deny");
|
||||
return locator(kind, actionable || Boolean(options.has));
|
||||
}, last: () => locator(kind, actionable), getByText: () => locator(),
|
||||
kind,
|
||||
// Model the production transcript: the resolved setup card stays visible
|
||||
// beside the pending write, and has no actionable decision buttons.
|
||||
count: () => kind === "loading" ? 0 : kind === "card" ? (actionable ? (failure === "duplicate-permission" ? 2 : 1) : (remote ? 2 : 1)) : kind === "deny" ? (f.run.status === "running" ? 1 : 0) : 1,
|
||||
getByRole: (_role: string, options: { name: string }) => locator(options.name === "Deny" ? "deny" : "other"),
|
||||
getByTestId: () => locator(),
|
||||
click: async () => {
|
||||
@@ -95,13 +108,21 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
if (failure === "missing-ack") f.run.resultJson.queuedSteeringAcknowledgements = {};
|
||||
postBrowser("/api/issues/issue/queued-comments/comment/steer", { queueId: "queue", targetRunId: "run", revision: "revision" });
|
||||
} else if (kind === "deny") {
|
||||
browserDeclines++; expect(browserSteers).toBe(1); expect(stopCase).toBe(false); f.finish();
|
||||
browserDeclines++; expect(browserSteers).toBe(1); expect(stopCase).toBe(false); f.finish(steeringMarker);
|
||||
f.run.resultJson.presentationDecision = { commentId: "final", reason: "fixture-public-presentation" };
|
||||
postBrowser("/api/heartbeat-runs/run/runtime-requests/request/resolve", { turnId: "turn", requestKind: "permission_approval", resolution: { action: "decline" } });
|
||||
} else throw new Error(`Unexpected click ${kind}`);
|
||||
},
|
||||
});
|
||||
const page = { goto: async () => {}, reload: async () => {}, getByText: () => locator(),
|
||||
getByRole: (_role: string, options: { name: string }) => locator(options.name === "Deny" ? "deny" : "other"),
|
||||
submitReply: (body: string) => {
|
||||
// Match the production editor's retained Markdown escaping. Matching the
|
||||
// plain input would never observe this otherwise valid queued comment.
|
||||
harness.message = body.replaceAll("_", "\\\\_").replaceAll("[]", "\\\\[]");
|
||||
expect(harness.message).not.toBe(body);
|
||||
postBrowser("/api/issues/issue/comments", { body: harness.message });
|
||||
},
|
||||
getByTestId: (id: string) => locator(id === "task-chat-runtime-request" ? "card" : id.startsWith("task-chat-queued-steer-") ? "steer" : id === "task-chat-history-loading" ? "loading" : "other"),
|
||||
waitForRequest: (predicate: (request: any) => boolean) => new Promise(resolve => waiters.push({ predicate, resolve })) };
|
||||
// The real shared remote oracle is exercised. Poisoned local copyback cannot
|
||||
@@ -112,7 +133,7 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
binding, observedAtMs: ++remoteSequence, receivedAtMs: remoteSequence + 1, observedMonotonicNs: String(remoteSequence), complete: true, workspace: {},
|
||||
targets: { "pi-control-fixture.txt": { absent: true, sha256: null, complete: true, mutationCount: retired && harness.mutation ? 1 : 0, parent: { dev: "1", ino: "2" } } },
|
||||
watcher: { complete: !(retired && harness.incomplete), targetMutationCount: retired && harness.mutation ? 1 : 0, workspaceMutationCount: 0 },
|
||||
processes: { captured: true, root, journal: [root], live: retired ? [] : [50] },
|
||||
processes: { captured: true, root: retired && failure === "root-rotation" ? { ...root, startTicks: "201" } : root, journal: [root], live: retired ? [] : [50] },
|
||||
setup: { path: ".paperclip-eval-action-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.txt", sha256: published ? `sha256:${"b".repeat(64)}` : null, published }, attached: null,
|
||||
});
|
||||
let seal: ReturnType<typeof snapshot> | undefined;
|
||||
@@ -126,8 +147,12 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
execution: { task, suite: { id: "pi-controls" }, environment: { id: remote ? "daytona" : "local" }, profile: { qualificationCandidate: "pi" } }, workspacePath, nonce: "fixture", deadlineAt: Date.now() + 1000,
|
||||
observe: () => {}, capture: async () => {}, evidence: async (name: string, value: unknown) => saved.set(name, structuredClone(value)), remoteBootstrap,
|
||||
registerCleanupAssertion: (fn: () => Promise<any>) => localCleanup.push(fn), registerBeforeEnvironmentTeardownAssertion: (fn: () => Promise<any>) => remoteCleanup.push(fn) } as any);
|
||||
if (failure === "missing-ack") { await expect(call).rejects.toThrow("acknowledgement"); expect(browserDeclines).toBe(0); }
|
||||
else if (remote && (failure === "mutation" || failure === "incomplete")) await expect(call).rejects.toThrow();
|
||||
if (failure === "foreign-created-task") { await expect(call).rejects.toThrow("Unexpected read /api/issues/foreign-created-task"); expect(stops).toBe(0); expect(browserDeclines).toBe(0); expect(browserSteers).toBe(0); }
|
||||
else if (failure === "duplicate-permission") { await expect(call).rejects.toThrow(); expect(stops).toBe(0); expect(browserDeclines).toBe(0); expect(browserSteers).toBe(0); }
|
||||
else if (failure === "missing-ack") { await expect(call).rejects.toThrow("acknowledgement"); expect(browserDeclines).toBe(0); }
|
||||
else if (failure === "steer-rejected") { await expect(call).rejects.toThrow("steering rejected: HTTP 409"); expect(browserDeclines).toBe(0); }
|
||||
else if (failure === "foreign-queued-body") { await expect(call).rejects.toThrow("browser comment queued"); expect(browserSteers).toBe(0); expect(browserDeclines).toBe(0); }
|
||||
else if (remote && (failure === "mutation" || failure === "incomplete" || failure === "root-rotation")) await expect(call).rejects.toThrow();
|
||||
else {
|
||||
const result = await call; expect(result.checks.every(c => c.passed)).toBe(true); expect(saved.has("api-state.json")).toBe(true);
|
||||
expect(stops).toBe(stopCase ? 1 : 0); expect(staleDeclines).toBe(stopCase ? 1 : 0);
|
||||
@@ -145,8 +170,9 @@ async function exercise(taskId: string, remote: boolean, failure?: "mutation" |
|
||||
expect(remote ? localCleanup : remoteCleanup).toHaveLength(0);
|
||||
const cleanups = remote ? remoteCleanup : localCleanup; expect(cleanups).toHaveLength(1);
|
||||
cleaningUp = true;
|
||||
if (failure) await expect(cleanups[0]!()).rejects.toThrow();
|
||||
if (failure && !(remote && failure === "annotation-drift")) await expect(cleanups[0]!()).rejects.toThrow();
|
||||
else expect((await cleanups[0]!())[0].passed).toBe(true);
|
||||
if (failure === "annotation-drift") expect(saved.get("pi-control-cleanup.json").processError).toBe(!remote);
|
||||
if (!stopCase && (!failure || failure === "missing-comment" || failure === "foreign-comment")) {
|
||||
const receipt = saved.get("pi-steering-presentation-after-cleanup.json");
|
||||
expect(receipt.phase).toBe("after-cleanup"); expect(receipt.comments).toEqual(publicComments.at(-1));
|
||||
@@ -165,4 +191,12 @@ for (const task of ["pending-permission-stop", "same-turn-steering"]) {
|
||||
it.each(["mutation", "incomplete"] as const)("fails local cleanup on %s despite an absent final target", failure => exercise("pending-permission-stop", false, failure));
|
||||
it.each(["mutation", "incomplete"] as const)("fails remote lifetime proof on %s despite an absent target", failure => exercise("same-turn-steering", true, failure));
|
||||
it("never denies the native write before the steering acknowledgement exists", () => exercise("same-turn-steering", false, "missing-ack"));
|
||||
it("rejects a queued body that differs from the exact browser submission", () => exercise("same-turn-steering", false, "foreign-queued-body"));
|
||||
it("stops before denial when the real steering API rejects the request", () => exercise("same-turn-steering", false, "steer-rejected"));
|
||||
it.each(["missing-comment", "foreign-comment"] as const)("retains and rejects %s during cleanup without fabricating persisted output", failure => exercise("same-turn-steering", false, failure));
|
||||
it("refuses control when two actionable permission cards remain beside resolved setup history", () => exercise("pending-permission-stop", true, "duplicate-permission"));
|
||||
it.each(["pending-permission-stop", "same-turn-steering"])("%s retains exact remote birth identity when public timestamp annotations change", task => exercise(task, true, "annotation-drift"));
|
||||
it("rejects actual remote process birth rotation despite unchanged public annotations", () => exercise("pending-permission-stop", true, "root-rotation"));
|
||||
it("still rejects local process authority timestamp changes", () => exercise("pending-permission-stop", false, "annotation-drift"));
|
||||
|
||||
it("refuses a task absent from the browser creation response instead of selecting a title match", () => exercise("pending-permission-stop", false, "foreign-created-task"));
|
||||
@@ -8,6 +8,7 @@ import { createDeniedTargetFixture, exists, observeRunProcesses } from "./copilo
|
||||
import { assertCopilotRemoteRetirement, copilotRemoteDeniedSample, prepareCopilotRemoteAction, type CopilotRemoteBootstrap, type CopilotRemoteFixture, type CopilotRemoteSnapshot } from "./copilot-protection-evidence.js";
|
||||
import { assertActiveStopRetirement, readActiveStopRemoteRetirement, readActiveStopCaller, type ActiveStopCaller, type ActiveStopRemoteObservation } from "./native-active-stop-evidence.js";
|
||||
import { assertSamePiPending, observePiControlPending, readPiStopSettlement, readPiSteeringAcknowledgement, readPiSteeringSettlement, type PiControlPending, type PiControlScope, type PiControlState } from "./pi-controls-evidence.js";
|
||||
import { approvePiBootstrapRead, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
|
||||
import { piNativeFinish } from "./pi-native-cases.js";
|
||||
import type { LiveFixtureValues } from "./live-fixtures.js";
|
||||
import type { MatrixExecution } from "./types.js";
|
||||
@@ -51,6 +52,7 @@ export async function runPiControlsFlow(input: {
|
||||
|| !["pending-permission-stop", "same-turn-steering"].includes(execution.task.id)) throw new Error("Unknown Pi control case");
|
||||
const stopCase = execution.task.id === "pending-permission-stop", remote = execution.environment.id === "daytona";
|
||||
if ((!remote && execution.environment.id !== "local") || (remote && !input.remoteBootstrap)) throw new Error("Pi controls require an isolated admitted environment");
|
||||
let bootstrapApproval: PiBootstrapApproval | undefined;
|
||||
const checks: Check[] = []; let issue: Row = {}, runs: Row[] = [], events: Row[] = [];
|
||||
const check = (id: string, passed: boolean, detail: string) => { checks.push({ id, passed, detail }); expect(passed, detail).toBe(true); };
|
||||
const name = `pi-control-${nonce}.txt`, local = remote ? undefined : await createDeniedTargetFixture(input.workspacePath, name), target = local?.targetRelativePath ?? name;
|
||||
@@ -62,9 +64,13 @@ export async function runPiControlsFlow(input: {
|
||||
let steered: { pending: PiControlPending; commentId: string; queueId: string; marker: string } | undefined;
|
||||
let completed = false;
|
||||
const observeProcesses = () => {
|
||||
// Controller timestamps describe remote launch annotations, not Linux
|
||||
// process birth. Daytona identity is checked in the bound remote snapshots
|
||||
// through PID, start ticks and boot ID, including the retirement seal.
|
||||
if (!observer) return;
|
||||
const run = runs[0], authority = run?.processPid ? { pid: run.processPid, groupId: run.processGroupId, startedAt: run.processStartedAt, runId: run.id } : undefined;
|
||||
if (authority) { const key = JSON.stringify(authority); if (processIdentity && processIdentity !== key) processError = true; processIdentity ??= key; }
|
||||
if (observer) processes = observer.sample(authority);
|
||||
processes = observer.sample(authority);
|
||||
};
|
||||
const load = async (): Promise<PiControlState> => {
|
||||
if (issue.id) issue = await api.get<Row>(`/api/issues/${issue.id}`);
|
||||
@@ -72,7 +78,7 @@ export async function runPiControlsFlow(input: {
|
||||
runs = await Promise.all(list.map(run => api.get<Row>(`/api/heartbeat-runs/${run.id}`)));
|
||||
if (runs.length > 1) throw new Error("Stopped waiting for Pi controls: extra provider run");
|
||||
events = runs[0] ? await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${runs[0]!.id}/events?afterSeq=${afterSeq}&limit=${limit}`)) : [];
|
||||
observeProcesses(); input.observe(issue, runs); return { run: runs[0] ?? {}, issue, events };
|
||||
observeProcesses(); input.observe(issue, runs); return { run: runs[0] ?? {}, issue, events, bootstrapApproval };
|
||||
};
|
||||
const scope = (): PiControlScope => ({ companyId: fixtures.company.id, issueId: issue.id, runId: runs[0]!.id, target });
|
||||
const readPresentation = async (state: PiControlState) => {
|
||||
@@ -140,8 +146,9 @@ export async function runPiControlsFlow(input: {
|
||||
check("explicit-per-turn-policy", configured.adapterConfig?.acpxPermissionMode === "approve-reads" && configured.adapterConfig?.lifecycleMode === "per_turn", "Native write requires a human decision before startup");
|
||||
const project = await api.post<Row>(`/api/companies/${fixtures.company.id}/projects`, { name: `Pi controls ${nonce}`, executionWorkspacePolicy: { enabled: true, defaultMode: "shared_workspace", sharedWorkspaceConcurrency: "serialize", allowIssueOverride: false, environmentId: fixtures.environment.id, workspaceStrategy: { type: "project_primary" } }, workspace: { name: "Primary", sourceType: "local_path", cwd: input.workspacePath, isPrimary: true } });
|
||||
if (!remote) await sample("before-request");
|
||||
await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title: execution.task.buildTitle(nonce), prompt: remote ? input.remoteBootstrap!.prompt(nonce) : prompt, workMode: "standard", projectName: project.name });
|
||||
issue = (await pollUntil({ label: "browser-created Pi control task", deadlineAt: input.deadlineAt, load: async () => (await api.get<Row[]>(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(i => i.title === execution.task.buildTitle(nonce)), accept: Boolean }))!;
|
||||
const createdTask = await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title: execution.task.buildTitle(nonce), prompt: remote ? input.remoteBootstrap!.prompt(nonce) : prompt, workMode: "standard", projectName: project.name, requireExplicitTitle: true });
|
||||
issue = await api.get<Row>(`/api/issues/${createdTask.issueId}`);
|
||||
check("created-task-identity", issue.id === createdTask.issueId && issue.companyId === fixtures.company.id && issue.assigneeAgentId === fixtures.agent.id, "Creation response binds the exact company-scoped assigned task before native control");
|
||||
if (remote) {
|
||||
await pollUntil({ label: "Pi control remote bootstrap", deadlineAt: input.deadlineAt, load, accept: state => state.run.status === "running" });
|
||||
const bound = await input.remoteBootstrap!.bindAndRelease({ issueId: issue.id, runId: runs[0]!.id, targets: [target], actionPrompt: async value => {
|
||||
@@ -151,12 +158,18 @@ export async function runPiControlsFlow(input: {
|
||||
await input.evidence("pi-control-before-request-remote.json", baseline); return prepared.prompt;
|
||||
} });
|
||||
if (bound !== fixture) throw new Error("Pi remote action identity changed");
|
||||
bootstrapApproval = await approvePiBootstrapRead({ api, fixture: bound, companyId: fixtures.company.id, issueId: issue.id, runId: runs[0]!.id,
|
||||
deadlineAt: input.deadlineAt, load, evidence: input.evidence });
|
||||
}
|
||||
await pollUntil({ label: "Pi unanswered native write", deadlineAt: input.deadlineAt, intervalMs: 200,
|
||||
load: async () => { const state = await load(); if (["failed", "timed_out", "cancelled", "succeeded"].includes(state.run.status)) throw new Error("Stopped waiting for Pi unanswered native write: no active pending permission"); return state; },
|
||||
accept: state => { observePiControlPending({ ...state, scope: scope() }); return true; } });
|
||||
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
|
||||
const card = page.getByTestId("task-chat-runtime-request").filter({ visible: true });
|
||||
// A resolved bootstrap read remains visible in the transcript. Only the
|
||||
// pending permission has decision buttons; the public event oracle and
|
||||
// browser POST below still bind the decision to the exact write request.
|
||||
const card = page.getByTestId("task-chat-runtime-request").filter({ visible: true })
|
||||
.filter({ has: page.getByRole("button", { name: "Deny", exact: true }) });
|
||||
await expect(card).toHaveCount(1); await expect(card.getByRole("button", { name: "Deny", exact: true })).toBeEnabled();
|
||||
await input.capture("pending-permission", "Pi write waits for a human decision", "pending-permission.png"); await sample("pending");
|
||||
if (stopCase) {
|
||||
@@ -172,10 +185,16 @@ export async function runPiControlsFlow(input: {
|
||||
await input.evidence("pi-control-pending.json", pending);
|
||||
const marker = `PI-STEER-${randomBytes(16).toString("hex")}`;
|
||||
const message = `After the actual native denial, complete this negative test. Do not write or invoke any other native tool. ${piNativeFinish(marker)}`;
|
||||
const commentRoutes = new Set([issue.id, issue.identifier].filter(Boolean).map(id => `/api/issues/${id}/comments`));
|
||||
const submitted = page.waitForRequest(request => commentRoutes.has(new URL(request.url()).pathname) && request.method() === "POST");
|
||||
await submitTaskReply(page, message);
|
||||
// The production rich-text editor serializes plain input as Markdown.
|
||||
// Bind the queued comment to the exact body actually sent by the browser.
|
||||
const submittedBody = (await submitted).postDataJSON()?.body;
|
||||
check("browser-steering-content", typeof submittedBody === "string" && submittedBody.startsWith("After the actual native denial, complete this negative test. Do not write or invoke any other native tool. ") && submittedBody.includes(marker), "Browser submitted the hidden instruction while permission remains pending");
|
||||
const queue = await pollUntil({ label: "Pi browser comment queued for steering", deadlineAt: input.deadlineAt, intervalMs: 200,
|
||||
load: () => api.get<Row>(`/api/issues/${issue.id}/queued-comments`), accept: q => q.steeringDisposition === "available" && q.entries?.some((entry: Row) => entry.comment.body === message) });
|
||||
const entries = queue.entries.filter((entry: Row) => entry.comment.body === message);
|
||||
load: () => api.get<Row>(`/api/issues/${issue.id}/queued-comments`), accept: q => q.steeringDisposition === "available" && q.entries?.some((entry: Row) => entry.comment.body === submittedBody) });
|
||||
const entries = queue.entries.filter((entry: Row) => entry.comment.body === submittedBody);
|
||||
check("one-browser-steering-message", entries.length === 1 && queue.entries.length === 1 && queue.targetRunId === pending.scope.runId, "One browser-originated comment targets the pending run");
|
||||
const commentId = entries[0].comment.id, queueId = queue.queueId;
|
||||
assertSamePiPending(pending, observePiControlPending({ ...await load(), scope: scope() }));
|
||||
@@ -183,8 +202,11 @@ export async function runPiControlsFlow(input: {
|
||||
const route = `/api/issues/${issue.id}/queued-comments/${commentId}/steer`;
|
||||
const posted = page.waitForRequest(request => new URL(request.url()).pathname === route && request.method() === "POST");
|
||||
await page.getByTestId(`task-chat-queued-steer-${commentId}`).click();
|
||||
const body = (await posted).postDataJSON();
|
||||
const steeringRequest = await posted;
|
||||
const body = steeringRequest.postDataJSON();
|
||||
check("exact-browser-steer", body.queueId === queueId && body.revision === queue.revision && body.targetRunId === pending.scope.runId, "Browser steers the exact queued comment into the active run");
|
||||
const response = await steeringRequest.response();
|
||||
if (!response?.ok()) throw new Error(`Pi native steering rejected: HTTP ${response?.status() ?? "missing"}`);
|
||||
steered = { pending, commentId, queueId, marker };
|
||||
await pollUntil({ label: "Pi same-turn steering acknowledgement", deadlineAt: input.deadlineAt, intervalMs: 200, load,
|
||||
accept: state => { assertSamePiPending(pending, observePiControlPending({ ...state, scope: scope() })); readPiSteeringAcknowledgement({ ...state, ...steered! }); return true; } });
|
||||
|
||||
@@ -48,11 +48,17 @@ export function piControlFixture(target = "target.txt") {
|
||||
run.resultJson.queuedSteeringAcknowledgements = { [commentId]: { status: "acknowledged", queueId, turnId: "turn", acknowledgedAt: "2026-10-01T00:00:01Z" } };
|
||||
append("item.completed", { kind: "steering_acknowledgement", status: "acknowledged", mode: "steer", text: "Steering acknowledged for the active turn." }, { itemId: `turn:steer:${commentId}` });
|
||||
}
|
||||
function finish() {
|
||||
function finish(finalMarker = marker) {
|
||||
append("runtime_request.resolved", { requestId: "request", turnId: "turn", action: "decline" });
|
||||
append("tool.execution.completed", { ...tool, status: "failed", output: "Pi operation was denied or cancelled" });
|
||||
append("turn.completed", { status: "completed", error: null });
|
||||
run.status = "succeeded"; issue.status = "done";
|
||||
append("run.result.accepted", { result: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", summary: finalMarker } }, {
|
||||
sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 1, sourceEventId: "source:control:run:1",
|
||||
});
|
||||
append("run.terminal", { schema: "paperclip.prp.terminal.v1", runTerminalState: "succeeded", turnTerminalState: "completed", reportedWorkDisposition: "done" }, {
|
||||
sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 2, sourceEventId: "source:control:run:2",
|
||||
});
|
||||
}
|
||||
return { scope, events, run, issue, state, pending, row, append, request, tool, cancel, responses, steer, finish, marker, commentId, queueId };
|
||||
}
|
||||
@@ -5,22 +5,28 @@ import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { collectPiFileEvidence, gradePiCopyback, gradePiFileEvidence, piFileContract, seedPiFile } from "./pi-file-evidence.js";
|
||||
import { canonicalProviderEventsFromAcpxRuntimeEvent } from "../../packages/paperclip-runner/src/provider-events.js";
|
||||
import { collectPiFileEvidence, gradePiCopyback, gradePiFileEvidence, piFileContract, piFilePrompt, seedPiFile } from "./pi-file-evidence.js";
|
||||
import { canonicalProviderEventsFromAcpxRuntimeEvent, createAcpxToolEventNormalizer } from "../../packages/paperclip-runner/src/provider-events.js";
|
||||
import { safeAcpxLocations } from "../../packages/paperclip-runner/src/drivers/acpx/safe-locations.js";
|
||||
import { classifyFailure } from "./failure-classifier.js";
|
||||
import { runnerMatrix } from "./catalog.js";
|
||||
|
||||
function fixture() {
|
||||
const c = piFileContract("fixture"), attachmentId = "12345678-1234-1234-1234-123456789abc";
|
||||
const tool = (seq: number, executionId: string, name: string, operation: string, completed: boolean, target: string | null, output = "") => {
|
||||
// Pi emits absolute edit locations and uses the bash command as title.
|
||||
// Exercise sidecar path normalization followed by the common projection;
|
||||
// direct raw ACP projection would incorrectly lose the workspace target.
|
||||
const canonical = canonicalProviderEventsFromAcpxRuntimeEvent({ type: "tool_call", toolCallId: executionId,
|
||||
tag: completed ? "tool_call_update" : "tool_call", title: name, kind: operation,
|
||||
status: completed ? "completed" : "pending", locations: safeAcpxLocations(target ? [{ path: `/workspace/${target}` }] : [], "/workspace", operation, name), rawOutput: output }, executionId, "turn")[0]!;
|
||||
const normalize = createAcpxToolEventNormalizer();
|
||||
const tool = (seq: number, executionId: string, name: string, operation: string,
|
||||
phase: "start" | "progress" | "end", target: string | null, text = "", rawOutput: unknown = undefined) => {
|
||||
// Sanitized shape of the retained Pi stream: missing opening location,
|
||||
// later resolved edit arguments, and stable bash name with command progress.
|
||||
// This exercises TS normalization/projection, not the Rust implementation.
|
||||
const event = normalize({ type: "tool_call", toolCallId: executionId,
|
||||
tag: phase === "start" ? "tool_call" : "tool_call_update", title: name, kind: operation,
|
||||
status: phase === "end" ? "completed" : phase === "start" ? "pending" : "in_progress",
|
||||
locations: safeAcpxLocations(target ? [{ path: `/workspace/${target}` }] : [], "/workspace", operation, name),
|
||||
text, rawOutput });
|
||||
const canonical = canonicalProviderEventsFromAcpxRuntimeEvent(event, executionId, "turn")[0]!;
|
||||
return { companyId: "company", runId: "run", seq, protocolSchemaVersion: 1, eventType: canonical.eventType,
|
||||
sourceInstanceId: "runner", sourceSeq: seq, sourceEventId: `runner:run:${seq}`,
|
||||
payload: { prpEvent: { schema: "paperclip.prp.event.v1", schemaVersion: 1, sourceKind: "runner", sourceInstanceId: "runner",
|
||||
sourceSeq: seq, sourceEventId: `runner:run:${seq}`, runId: "run", turnId: "turn", normalizedSessionId: "session", ...canonical } } };
|
||||
};
|
||||
@@ -33,42 +39,87 @@ function fixture() {
|
||||
publish: { operationId: "register_deliverable", input: { filename: c.filename, contentRef: c.filename, contentType: "text/plain", byteSize: c.byteSize, sha256: c.sha256 },
|
||||
result: { disposition: "applied", commandId: `deliverable-prepared:${attachmentId}`, entityRefs: [attachmentId, "product", "comment"] } },
|
||||
} } },
|
||||
events: [tool(1, "edit", "edit", "edit", false, c.filename), tool(2, "edit", "edit", "edit", true, c.filename),
|
||||
tool(3, "validate", c.validationCommand, "execute", false, null), tool(4, "validate", c.validationCommand, "execute", true, null, c.validationMarker)],
|
||||
events: [tool(1, "edit", "edit", "edit", "start", null),
|
||||
tool(2, "edit", "edit", "edit", "progress", c.filename),
|
||||
tool(3, "edit", "edit", "edit", "end", c.filename),
|
||||
tool(4, "validate", "bash", "execute", "start", null, "bash (pending): [terminal] validate"),
|
||||
tool(5, "validate", c.validationCommand, "execute", "progress", null, `${c.validationCommand} (in_progress): ${c.validationCommand}`),
|
||||
tool(6, "validate", c.validationCommand, "execute", "end", null, "", {
|
||||
content: [{ type: "text", text: `${c.validationMarker}\n` }],
|
||||
structuredContent: { output: `${c.validationMarker}\n`, truncated: false, exit_code: 0, wall_time_seconds: 0 },
|
||||
})],
|
||||
attachments: [{ id: attachmentId, companyId: "company", issueId: "issue", originatingRunId: "run", createdByAgentId: "agent",
|
||||
originalFilename: c.filename, contentType: "text/plain", byteSize: c.byteSize, sha256: c.sha256 }],
|
||||
activity: [{ action: "issue.attachment_added", companyId: "company", runId: "run", actorId: "agent", entityId: "issue",
|
||||
details: { attachmentId, source: "paperclip_runner_protocol" } }],
|
||||
};
|
||||
}
|
||||
function changeOutput(f: ReturnType<typeof fixture>, change: (output: any) => void) {
|
||||
const p = f.events[5]!.payload.prpEvent.payload, output = JSON.parse(p.output as string);
|
||||
change(output); p.output = JSON.stringify(output);
|
||||
}
|
||||
describe("Pi edit, validation and public artifact oracle", () => {
|
||||
it("calibrates against the sanitized 44-row actual native stream without regrading the paid attempt", async () => {
|
||||
const retained = JSON.parse(await readFile(new URL("./fixtures/pi-file-evidence-actual-stream.json", import.meta.url), "utf8"));
|
||||
const f = fixture();
|
||||
expect(retained.events).toHaveLength(44);
|
||||
f.events = retained.events.map((x: any) => ({ companyId: "company", runId: "run", seq: x.seq,
|
||||
protocolSchemaVersion: 1, eventType: x.eventType, sourceInstanceId: "runner", sourceSeq: x.sourceSeq,
|
||||
sourceEventId: `runner:run:${x.sourceSeq}`, payload: { prpEvent: {
|
||||
schema: "paperclip.prp.event.v1", schemaVersion: 1, sourceKind: "runner", sourceInstanceId: "runner",
|
||||
sourceSeq: x.sourceSeq, sourceEventId: `runner:run:${x.sourceSeq}`, runId: "run", turnId: "turn",
|
||||
normalizedSessionId: "session", eventType: x.eventType, payload: x.payload,
|
||||
} } }));
|
||||
expect(gradePiFileEvidence(f).verification).toMatchObject({ nativeStructuredExitCode: 0, typedExitCode: null });
|
||||
});
|
||||
it("accepts actual correlated lifecycles, independent bytes and a run-bound registered download", () => {
|
||||
const result = gradePiFileEvidence(fixture());
|
||||
expect(result.passed).toBe(true);
|
||||
expect(result.diff.text).toContain("-ready-fixture\n+verified-fixture\n");
|
||||
expect(result.limits.join(" ")).toContain("not attested");
|
||||
expect(result.verification.providerExecutionStatus).toBe("completed");
|
||||
expect(result.verification.commandTitle).toBe(piFileContract("fixture").validationCommand);
|
||||
expect(result.verification.command).toBe(piFileContract("fixture").validationCommand);
|
||||
expect(result.verification).toMatchObject({ commandEvidence: "correlated_native_progress",
|
||||
exitEvidence: "native_structured_output_receipt", nativeStructuredExitCode: 0, typedExitCode: null });
|
||||
expect(fixture().events.map(e => e.payload.prpEvent.payload.target)).toEqual([null, "extended-fixture.txt", "extended-fixture.txt", null, null, null]);
|
||||
expect(fixture().events.slice(3).every(e => e.payload.prpEvent.payload.name === "bash")).toBe(true);
|
||||
expect(result.verification.nativeFileAttribution).toBe("workspace_relative_display_target");
|
||||
});
|
||||
it.each(["before", "after"])("rejects an extra metadata bash call %s validation despite correct file and download bytes", placement => {
|
||||
const f = fixture();
|
||||
const metadata = structuredClone(f.events.slice(3));
|
||||
for (const row of metadata) row.payload.prpEvent.payload.executionId = "metadata";
|
||||
metadata[1]!.payload.prpEvent.payload.progress = "wc -c extended-fixture.txt (in_progress): wc -c extended-fixture.txt";
|
||||
metadata[2]!.payload.prpEvent.payload.output = JSON.stringify({
|
||||
content: [{ type: "text", text: "17 extended-fixture.txt\n" }],
|
||||
structuredContent: { output: "17 extended-fixture.txt\n", truncated: false, exit_code: 0 },
|
||||
});
|
||||
if (placement === "before") f.events.splice(3, 0, ...metadata);
|
||||
else f.events.push(...metadata);
|
||||
for (const [index, row] of f.events.entries()) {
|
||||
row.seq = row.sourceSeq = row.payload.prpEvent.sourceSeq = index + 1;
|
||||
row.sourceEventId = row.payload.prpEvent.sourceEventId = `runner:run:${index + 1}`;
|
||||
}
|
||||
expect(() => gradePiFileEvidence(f)).toThrow("one observed provider validation execution");
|
||||
});
|
||||
const mutations: Array<[string, (f: ReturnType<typeof fixture>) => void]> = [
|
||||
["absent seed", f => { f.seed = {} as any; }],
|
||||
["wrong final bytes", f => { f.workspaceBytes = Buffer.from("wrong\n"); }],
|
||||
["final-only write", f => { f.events[0]!.payload.prpEvent.payload.name = "write"; f.events[1]!.payload.prpEvent.payload.name = "write"; }],
|
||||
["missing edit", f => { f.events = f.events.slice(2); }],
|
||||
["missing validation", f => { f.events = f.events.slice(0, 2); }],
|
||||
["echo-only validation", f => { f.events[2]!.payload.prpEvent.payload.name = `echo ${piFileContract("fixture").validationMarker}`; f.events[3]!.payload.prpEvent.payload.name = f.events[2]!.payload.prpEvent.payload.name; }],
|
||||
["failed validation", f => { f.events[3]!.payload.prpEvent.payload.status = "failed"; }],
|
||||
["final-only write", f => { f.events[0]!.payload.prpEvent.payload.name = "write"; f.events[2]!.payload.prpEvent.payload.name = "write"; }],
|
||||
["missing edit", f => { f.events = f.events.slice(3); }],
|
||||
["missing validation", f => { f.events = f.events.slice(0, 3); }],
|
||||
["echo-only validation", f => { f.events[3]!.payload.prpEvent.payload.name = `echo ${piFileContract("fixture").validationMarker}`; f.events[5]!.payload.prpEvent.payload.name = f.events[3]!.payload.prpEvent.payload.name; }],
|
||||
["failed validation", f => { f.events[5]!.payload.prpEvent.payload.status = "failed"; }],
|
||||
["unfinished validation", f => { f.events.pop(); }],
|
||||
["unrelated validation turn", f => { f.events[2]!.payload.prpEvent.turnId = "other"; f.events[3]!.payload.prpEvent.turnId = "other"; }],
|
||||
["unrelated validation turn", f => { f.events[3]!.payload.prpEvent.turnId = "other"; f.events[5]!.payload.prpEvent.turnId = "other"; }],
|
||||
["foreign native session", f => { f.run.nativeSessionId = "other"; }],
|
||||
["foreign native producer", f => { f.run.runnerInstanceId = "other"; }],
|
||||
["foreign event run", f => { f.events[1]!.payload.prpEvent.runId = "other"; }],
|
||||
["duplicate event", f => { f.events.push(structuredClone(f.events[3]!)); }],
|
||||
["missing tool start", f => { f.events.splice(2, 1); }],
|
||||
["truncated validation output", f => { f.events[3]!.payload.prpEvent.payload.outputTruncated = true; }],
|
||||
["missing native target", f => { f.events[0]!.payload.prpEvent.payload.target = null; f.events[1]!.payload.prpEvent.payload.target = null; }],
|
||||
["wrong target", f => { f.events[1]!.payload.prpEvent.payload.target = "other.txt"; }],
|
||||
["foreign event run", f => { f.events[2]!.payload.prpEvent.runId = "other"; }],
|
||||
["duplicate event", f => { f.events.push(structuredClone(f.events[5]!)); }],
|
||||
["missing tool start", f => { f.events.splice(3, 1); }],
|
||||
["truncated validation output", f => { f.events[5]!.payload.prpEvent.payload.outputTruncated = true; }],
|
||||
["missing native target", f => { f.events[0]!.payload.prpEvent.payload.target = null; f.events[2]!.payload.prpEvent.payload.target = null; }],
|
||||
["wrong target", f => { f.events[2]!.payload.prpEvent.payload.target = "other.txt"; }],
|
||||
["missing registration", f => { f.run.resultJson.semanticToolReceipts = {} as any; }],
|
||||
["rejected registration", f => { f.run.resultJson.semanticToolReceipts.publish.result.disposition = "denied"; }],
|
||||
["wrong registered hash", f => { f.run.resultJson.semanticToolReceipts.publish.input.sha256 = "wrong"; }],
|
||||
@@ -79,6 +130,34 @@ describe("Pi edit, validation and public artifact oracle", () => {
|
||||
["wrong download bytes", f => { f.downloadedBytes = Buffer.from("wrong\n"); }],
|
||||
["missing publication activity", f => { f.activity = []; }],
|
||||
["foreign publication run", f => { f.activity[0]!.runId = "other"; }],
|
||||
["conflicting resolved target", f => { f.events[1]!.payload.prpEvent.payload.target = "other.txt"; }],
|
||||
["target regresses after resolution", f => { f.events[0]!.payload.prpEvent.payload.target = piFileContract("fixture").filename; f.events[1]!.payload.prpEvent.payload.target = null; }],
|
||||
["missing terminal target", f => { f.events[2]!.payload.prpEvent.payload.target = null; }],
|
||||
["absent command proof", f => { f.events[4]!.payload.prpEvent.payload.progress = null; }],
|
||||
["echo-only command proof", f => { f.events[4]!.payload.prpEvent.payload.progress = "echo PI-VALIDATED-fixture (in_progress): echo PI-VALIDATED-fixture"; }],
|
||||
["conflicting command before exact proof", f => { f.events[3]!.payload.prpEvent.payload.progress = "node other.js (pending): node other.js"; }],
|
||||
["late command proof", f => { f.events[5]!.payload.prpEvent.payload.progress = f.events[4]!.payload.prpEvent.payload.progress; f.events[4]!.payload.prpEvent.payload.progress = null; }],
|
||||
["plain echoed marker output", f => { f.events[5]!.payload.prpEvent.payload.output = piFileContract("fixture").validationMarker; }],
|
||||
["nonzero native exit", f => { changeOutput(f, o => { o.structuredContent.exit_code = 1; }); }],
|
||||
["missing native exit", f => { changeOutput(f, o => { delete o.structuredContent.exit_code; }); }],
|
||||
["truncated native output", f => { changeOutput(f, o => { o.structuredContent.truncated = true; }); }],
|
||||
["false marker substring", f => { changeOutput(f, o => { o.structuredContent.output = "failed before PI-VALIDATED-fixture\n"; }); }],
|
||||
["contradictory content", f => { changeOutput(f, o => { o.content[0].text = "failed"; }); }],
|
||||
["contradictory typed exit", f => { f.events[5]!.payload.prpEvent.payload.exitCode = 1; }],
|
||||
["foreign command proof execution", f => { f.events[4]!.payload.prpEvent.payload.executionId = "foreign"; }],
|
||||
["foreign command proof turn", f => { f.events[4]!.payload.prpEvent.turnId = "foreign"; }],
|
||||
["foreign command proof session", f => { f.events[4]!.payload.prpEvent.normalizedSessionId = "foreign"; }],
|
||||
["foreign command proof producer", f => {
|
||||
const row = f.events[4]!, e = row.payload.prpEvent;
|
||||
e.sourceInstanceId = row.sourceInstanceId = "foreign"; e.sourceEventId = row.sourceEventId = `foreign:run:${e.sourceSeq}`;
|
||||
}],
|
||||
["durable source differs from envelope", f => { f.events[4]!.sourceInstanceId = "foreign"; }],
|
||||
["durable event identity differs from envelope", f => { f.events[4]!.sourceEventId = "runner:run:999"; }],
|
||||
["durable source sequence differs from envelope", f => { f.events[4]!.sourceSeq = 999; }],
|
||||
["command proof after terminal", f => {
|
||||
const e = f.events[4]!; f.events.splice(4, 1); e.seq = 7; e.payload.prpEvent.sourceSeq = 7;
|
||||
e.sourceSeq = 7; e.sourceEventId = e.payload.prpEvent.sourceEventId = "runner:run:7"; f.events.push(e);
|
||||
}],
|
||||
];
|
||||
it.each(mutations)("rejects %s even when the model claims completion", (_name, change) => {
|
||||
const f = fixture(); change(f); expect(() => gradePiFileEvidence(f)).toThrow("Pi file evidence:");
|
||||
@@ -107,12 +186,25 @@ describe("Pi edit, validation and public artifact oracle", () => {
|
||||
await expect(validate()).rejects.toMatchObject({ code: 1 });
|
||||
} finally { await rm(root, { recursive: true, force: true }); }
|
||||
});
|
||||
it("pastes the literal validation command as Markdown code instead of escaped paragraph text", () => {
|
||||
const command = piFileContract("0123456789ab-1").validationCommand;
|
||||
const fenced = piFilePrompt("0123456789ab-1").match(/\n```bash\n([^\n]+)\n```\n/u);
|
||||
expect(fenced?.[1]).toBe(command);
|
||||
expect(fenced?.[1]).toContain("!==");
|
||||
expect(fenced?.[1]).not.toContain("!\\=\\=");
|
||||
});
|
||||
it("changes only Pi file prompts and keeps all four question methods", () => {
|
||||
const cells = runnerMatrix.filter(c => c.suite.id === "extended-harnesses" && c.task.id === "file-edit-validate");
|
||||
expect(cells).toHaveLength(6);
|
||||
for (const cell of cells) {
|
||||
const prompt = cell.task.buildPrompt("fixture");
|
||||
expect(prompt.includes("register_deliverable")).toBe(cell.profile.qualificationCandidate === "pi");
|
||||
if (cell.profile.qualificationCandidate === "pi") {
|
||||
const artifact = cell.task.buildMatchers("fixture", cell).find(matcher => matcher.kind === "artifact_exact");
|
||||
if (!artifact || artifact.kind !== "artifact_exact") throw new Error("Pi file task must require a registered artifact");
|
||||
expect(artifact.name).toBe(piFileContract("fixture").filename);
|
||||
expect(prompt).toContain(`title ${artifact.name},`);
|
||||
}
|
||||
}
|
||||
expect(runnerMatrix.filter(c => c.profile.qualificationCandidate === "pi")).toHaveLength(26);
|
||||
});
|
||||
|
||||
@@ -21,8 +21,8 @@ export function piFilePrompt(nonce: string) {
|
||||
const c = piFileContract(nonce);
|
||||
return [
|
||||
`The fixture has already created ${c.filename} in your execution workspace with exactly ${JSON.stringify(c.before)}. Read it, then use your native edit tool once to replace those bytes with exactly ${JSON.stringify(c.after)}. Do not substitute write, bash, or an API for the edit.`,
|
||||
`Run this exact native bash command, without an added shell prefix or suffix:\n${c.validationCommand}\nIt independently reads the file, fails on incorrect bytes, and prints the validation marker only after success.`,
|
||||
`Deliver the edited text file as a downloadable attachment using the assigned register_deliverable semantic tool, with filename and workspace-relative contentRef ${c.filename}, contentType text/plain, byteSize ${c.byteSize}, sha256 ${c.sha256}, title Verified text, and a fresh idempotencyKey. Wait for the real registration receipt. Do not upload a different file or claim publication from a disk path.`,
|
||||
`Run this exact native bash command exactly once, without an added shell prefix or suffix:\n\`\`\`bash\n${c.validationCommand}\n\`\`\`\nIt independently reads the file, fails on incorrect bytes, and prints the validation marker only after success. This is the only bash call allowed for this task; do not run any other shell commands.`,
|
||||
`Deliver the edited text file as a downloadable attachment using the assigned register_deliverable semantic tool, with filename and workspace-relative contentRef ${c.filename}, contentType text/plain, byteSize ${c.byteSize}, sha256 ${c.sha256}, title ${c.filename}, and a fresh idempotencyKey. Use the supplied byteSize and sha256 for the exact validated post-edit bytes; do not run extra commands to obtain metadata. Wait for the real registration receipt. Do not upload a different file or claim publication from a disk path.`,
|
||||
`After successful validation and registration, call paperclip_finish with reportedWorkDisposition done, summary EXTENDED-FILE-${nonce}, the current completion contract revision and satisfied objective criterion, no remaining work, the actual validation command/result, and the registered artifact reference.`,
|
||||
`Wait for paperclip_finish to succeed, then emit exactly EXTENDED-FILE-${nonce} as your final response. Do not create unrelated files or work.`,
|
||||
].join("\n");
|
||||
@@ -58,7 +58,8 @@ export function gradePiCopyback(run: Row, lease: Row, companyId: string, environ
|
||||
}
|
||||
|
||||
/** Only the common public projection is authority here. Native raw arguments,
|
||||
* full diff blocks and typed command exit codes are deliberately not invented. */
|
||||
* full diff blocks and typed command exit codes are deliberately not invented.
|
||||
* The native structured output receipt is distinct from that nullable typed field. */
|
||||
export function gradePiFileEvidence(input: {
|
||||
nonce: string; companyId: string; issueId: string; agentId: string; run: Row;
|
||||
environment: "local" | "daytona"; environmentId: string; lease?: Row;
|
||||
@@ -84,6 +85,7 @@ export function gradePiFileEvidence(input: {
|
||||
&& Number.isSafeInteger(row.seq) && row.seq > 0 && !seqs.has(row.seq)
|
||||
&& typeof e.sourceInstanceId === "string" && e.sourceInstanceId.length > 0
|
||||
&& Number.isSafeInteger(e.sourceSeq) && e.sourceSeq > (sourceSeqs.get(e.sourceInstanceId) ?? 0)
|
||||
&& row.sourceInstanceId === e.sourceInstanceId && row.sourceSeq === e.sourceSeq && row.sourceEventId === e.sourceEventId
|
||||
&& e.sourceEventId === `${e.sourceInstanceId}:${e.runId}:${e.sourceSeq}` && !seen.has(e.sourceEventId), "foreign, duplicated, or reordered durable evidence");
|
||||
seqs.add(row.seq); seen.add(e.sourceEventId); sourceSeqs.set(e.sourceInstanceId, e.sourceSeq);
|
||||
return { row, e, p: rec(e.payload) };
|
||||
@@ -104,20 +106,46 @@ export function gradePiFileEvidence(input: {
|
||||
&& starts.length === 1 && starts[0]!.row.seq < end.row.seq
|
||||
&& matches.filter(x => x.e.eventType === "tool.execution.completed").length === 1
|
||||
&& matches.every(x => x.p.schema === end.p.schema && x.p.name === end.p.name && x.p.operation === end.p.operation
|
||||
&& x.p.target === end.p.target && x.e.sourceKind === "runner" && x.e.turnId === end.e.turnId
|
||||
&& x.row.seq >= starts[0]!.row.seq && x.row.seq <= end.row.seq
|
||||
&& x.e.sourceKind === "runner" && x.e.turnId === end.e.turnId
|
||||
&& x.e.normalizedSessionId === end.e.normalizedSessionId && x.e.sourceInstanceId === end.e.sourceInstanceId
|
||||
&& (x === end || x.p.status === "running")), "incomplete or unsuccessful native tool lifecycle");
|
||||
return starts[0]!;
|
||||
&& (x === end || (x.p.status === "running"
|
||||
&& ["tool.execution.started", "tool.execution.progressed"].includes(x.e.eventType)))), "incomplete or unsuccessful native tool lifecycle");
|
||||
// Native arguments arrive incrementally. An unresolved target is allowed
|
||||
// only before the first exact target; conflicting or regressed targets fail.
|
||||
let resolved = false;
|
||||
for (const x of matches) {
|
||||
requireEvidence(x.p.target === end.p.target || (!resolved && x.p.target === null), "conflicting native tool target");
|
||||
if (x.p.target !== null) resolved = true;
|
||||
}
|
||||
return { start: starts[0]!, matches };
|
||||
}
|
||||
lifecycle(edit);
|
||||
const validations = tools.filter(x => x.e.eventType === "tool.execution.completed" && x.p.operation === "execute"
|
||||
&& x.p.name === c.validationCommand && typeof x.p.output === "string" && x.p.output.includes(c.validationMarker));
|
||||
&& x.p.name === "bash");
|
||||
requireEvidence(validations.length === 1, "one observed provider validation execution");
|
||||
const validation = validations[0]!, start = lifecycle(validation);
|
||||
const validation = validations[0]!, { start, matches } = lifecycle(validation);
|
||||
requireEvidence(start.row.seq > edit.row.seq && validation.e.turnId === edit.e.turnId
|
||||
&& validation.e.normalizedSessionId === edit.e.normalizedSessionId
|
||||
&& validation.e.sourceInstanceId === edit.e.sourceInstanceId && validation.p.outputTruncated === false,
|
||||
"validation must follow the edit in the same native turn");
|
||||
// The common lifecycle preserves the opening name (bash). ACPX's resolved
|
||||
// command is retained as progress, rather than replacing that stable name.
|
||||
const commandProgress = `${c.validationCommand} (in_progress): ${c.validationCommand}`;
|
||||
const allowedProgress = new Set([`bash (pending): [terminal] ${validation.p.executionId}`,
|
||||
`${c.validationCommand} (pending): ${c.validationCommand}`, commandProgress]);
|
||||
requireEvidence(matches.some(x => x !== validation && x.p.progress === commandProgress)
|
||||
&& matches.every(x => x.p.progress === null || allowedProgress.has(x.p.progress)),
|
||||
"exact correlated command progress before completion required");
|
||||
requireEvidence(typeof validation.p.output === "string" && Buffer.byteLength(validation.p.output) <= 16_384,
|
||||
"bounded native validation receipt required");
|
||||
let output: Row = {};
|
||||
try { output = rec(JSON.parse(validation.p.output)); } catch { /* Rejected below; plain marker text is insufficient. */ }
|
||||
const structured = rec(output.structuredContent), expectedOutput = `${c.validationMarker}\n`;
|
||||
requireEvidence(structured.exit_code === 0 && structured.truncated === false && structured.output === expectedOutput
|
||||
&& Array.isArray(output.content) && output.content.length === 1
|
||||
&& rec(output.content[0]).type === "text" && rec(output.content[0]).text === expectedOutput
|
||||
&& (validation.p.exitCode === null || validation.p.exitCode === 0), "successful native structured validation receipt required");
|
||||
const attachments = input.attachments.map(rec).filter(a => a.id === input.attachmentId);
|
||||
requireEvidence(attachments.length === 1, "one public attachment");
|
||||
const a = attachments[0]!;
|
||||
@@ -144,11 +172,13 @@ export function gradePiFileEvidence(input: {
|
||||
validationExecutionId: validation.p.executionId, attachmentId: a.id,
|
||||
verification: { kind: "independent_exact_bytes", beforeSha256: sha(c.before), afterSha256: c.sha256,
|
||||
downloadedSha256: sha(input.downloadedBytes), byteSize: c.byteSize, providerExecutionStatus: validation.p.status,
|
||||
commandTitle: validation.p.name, nativeFileTarget: edit.p.target,
|
||||
command: c.validationCommand, providerToolName: validation.p.name, commandEvidence: "correlated_native_progress",
|
||||
exitEvidence: "native_structured_output_receipt", nativeStructuredExitCode: structured.exit_code,
|
||||
typedExitCode: validation.p.exitCode, nativeFileTarget: edit.p.target,
|
||||
nativeFileAttribution: "workspace_relative_display_target" },
|
||||
diff: { source: "independent_seed_and_workspace_bytes", path: c.filename,
|
||||
text: `--- ${c.filename}\n+++ ${c.filename}\n@@ -1 +1 @@\n-${c.before.trimEnd()}\n+${c.after.trimEnd()}\n` },
|
||||
limits: ["Validation command title is checked exactly; raw arguments and typed exitCode are not projected, so raw invocation/exit code are not attested.",
|
||||
limits: ["Exact command progress and the native structured output receipt are checked; raw arguments and nullable typed exitCode are not attested by those distinct fields.",
|
||||
"Diff is computed from independently checked workspace bytes; it does not qualify native diff presentation or private invocation metadata."],
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,10 +1,63 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { gradePiNativeAnswers, hasFailedPiWrite, hasPiCrossRootDenial, piNativeTasks } from "./pi-native-cases.js";
|
||||
import { gradePiNativeAnswers, gradePiNativeMemory, hasPiNativeMemoryRead, hasFailedPiWrite, hasPiCrossRootDenial, piNativeMemoryPrompt, piNativeTasks } from "./pi-native-cases.js";
|
||||
import { runnerMatrix, runnerSuites } from "./catalog.js";
|
||||
import { buildRunnerE2EProcessEnvironment } from "./harness-env.js";
|
||||
import { parseRunnerSelectors, selectRunnerExecutions } from "./selectors.js";
|
||||
|
||||
describe("Pi native Product qualification", () => {
|
||||
it("supplies native write arguments with one authoritative content string and its final LF", () => {
|
||||
const nonce = "0123456789abcdef0123456789abcdef";
|
||||
const prompt = piNativeMemoryPrompt(nonce, "/unassigned/denied.txt");
|
||||
expect(prompt.split(nonce)).toHaveLength(2);
|
||||
const encoded = /```json\n(.*?)\n```/s.exec(prompt)![1]!;
|
||||
const args = JSON.parse(encoded);
|
||||
expect(Object.keys(args)).toEqual(["path", "content"]);
|
||||
expect(args.path).toBe("<AGENT_HOME>/memory/pi-native.txt");
|
||||
expect(args.content).toBe(`${nonce}\n`);
|
||||
expect(Buffer.byteLength(args.content)).toBe(33);
|
||||
expect(Buffer.from(args.content).at(-1)).toBe(10);
|
||||
expect(prompt).toContain("Use native read once, without offset or limit");
|
||||
expect(prompt).toContain("Do not retry or work around it");
|
||||
});
|
||||
|
||||
it("requires the current nonce and exactly one LF at every memory readback", () => {
|
||||
const nonce = "0123456789abcdef0123456789abcdef";
|
||||
expect(gradePiNativeMemory(`${nonce}\n`, nonce)).toBe(true);
|
||||
for (const wrong of [undefined, null, {}, nonce, `${nonce}\n\n`, `${nonce}\r\n`, `${nonce}\\n`, `${nonce}\\u000a`, `${"f".repeat(32)}\n`]) {
|
||||
expect(gradePiNativeMemory(wrong, nonce)).toBe(false);
|
||||
}
|
||||
});
|
||||
it("requires a completed native read and rejects a shell shortcut or missing receipt", () => {
|
||||
const nonce = "0123456789abcdef0123456789abcdef";
|
||||
const read = { eventType: "tool.execution.completed", payload: { prpEvent: { payload: {
|
||||
schema: "paperclip.tool.execution.v1", transport: "builtin", name: "read", operation: "read", status: "completed", executionId: "read-1", target: null, readOnly: true, outputTruncated: false, output: JSON.stringify({ content: [{ type: "text", text: `${nonce}\n` }] }),
|
||||
} } } };
|
||||
expect(hasPiNativeMemoryRead([read], nonce)).toBe(true);
|
||||
const change = (patch: Record<string, unknown>) => ({ ...read, payload: { prpEvent: { payload: { ...read.payload.prpEvent.payload, ...patch } } } });
|
||||
expect(hasPiNativeMemoryRead([change({ target: "bootstrap.md", output: JSON.stringify({ content: [{ type: "text", text: "bootstrap instructions" }] }) }), read], nonce)).toBe(true);
|
||||
for (const rows of [[], [{}], [read, read], [change({ target: undefined })], [change({ status: "failed" })], [change({ transport: "mcp" })], [change({ executionId: "" })], [change({ schema: "untrusted" })], [change({ target: "another-file.txt" })], [change({ output: "malformed" })], [change({ outputTruncated: true })], [change({ readOnly: false })], [change({ output: JSON.stringify({ content: [{ type: "text", text: "unrelated-file" }] }) })], [change({ output: JSON.stringify({ content: [{ type: "text", text: nonce }] }) })], [change({ name: "bash", operation: "execute" })], [read, change({ name: "bash", operation: "execute" })]]) {
|
||||
expect(hasPiNativeMemoryRead(rows, nonce)).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("binds the remote memory read to the exact agent and current run", () => {
|
||||
const nonce = "0123456789abcdef0123456789abcdef";
|
||||
const remoteRun = { agentId: "11111111-1111-4111-8111-111111111111", runId: "22222222-2222-4222-8222-222222222222" };
|
||||
const target = `.paperclip-runtime/agent-files/${remoteRun.agentId}/${remoteRun.runId}/memory/pi-native.txt`;
|
||||
const read = (patch: Record<string, unknown> = {}) => ({ eventType: "tool.execution.completed", payload: { prpEvent: { payload: {
|
||||
schema: "paperclip.tool.execution.v1", transport: "builtin", name: "read", operation: "read", status: "completed", executionId: "remote-read-1", target, readOnly: true, outputTruncated: false, output: JSON.stringify({ content: [{ type: "text", text: `${nonce}\n` }] }), ...patch,
|
||||
} } } });
|
||||
expect(hasPiNativeMemoryRead([read()], nonce, remoteRun)).toBe(true);
|
||||
expect(hasPiNativeMemoryRead([read({ target: "bootstrap.md", output: JSON.stringify({ content: [{ type: "text", text: "bootstrap" }] }) }), read()], nonce, remoteRun)).toBe(true);
|
||||
expect(hasPiNativeMemoryRead([read()], nonce)).toBe(false);
|
||||
for (const patch of [{ target: null }, { target: "memory/pi-native.txt" }, { target: `${target}.bak` }, { target: target.replace(remoteRun.agentId, remoteRun.runId) }, { target: target.replace(remoteRun.runId, remoteRun.agentId) }, { output: JSON.stringify({ content: [{ type: "text", text: nonce }] }) }, { status: "failed" }, { outputTruncated: true }]) {
|
||||
expect(hasPiNativeMemoryRead([read(patch)], nonce, remoteRun)).toBe(false);
|
||||
}
|
||||
expect(hasPiNativeMemoryRead([read(), read()], nonce, remoteRun)).toBe(false);
|
||||
expect(hasPiNativeMemoryRead([read(), read({ name: "bash", operation: "execute" })], nonce, remoteRun)).toBe(false);
|
||||
expect(hasPiNativeMemoryRead([read()], nonce, { ...remoteRun, runId: "../other-run" })).toBe(false);
|
||||
});
|
||||
|
||||
it("selects five local and five remote Pi cases without changing the basic extended matrix", () => {
|
||||
const suite = runnerSuites.find(row => row.id === "pi-native")!;
|
||||
expect(suite.manualOnly).toBe(true); expect(suite.expectedMatrixSize).toBe(10);
|
||||
@@ -29,9 +82,9 @@ describe("Pi native Product qualification", () => {
|
||||
it("admits explicit local and remote Pi native candidates and discards ambient admission", () => {
|
||||
const cell = runnerMatrix.find(row => row.suite.id === "pi-native")!;
|
||||
const source = { PAPERCLIP_RUNNER_ACPX_QUALIFICATION: "ambient" };
|
||||
expect(JSON.parse(buildRunnerE2EProcessEnvironment(source, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION!)).toEqual([{ agent: "pi", model: cell.profile.model }]);
|
||||
expect(buildRunnerE2EProcessEnvironment(source, [cell]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeUndefined();
|
||||
const remote = runnerMatrix.find(row => row.suite.id === "pi-native" && row.environment.id === "daytona")!;
|
||||
expect(JSON.parse(buildRunnerE2EProcessEnvironment(source, [remote]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION!)).toEqual([{ agent: "pi", model: remote.profile.model }]);
|
||||
expect(buildRunnerE2EProcessEnvironment(source, [remote]).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeUndefined();
|
||||
expect(buildRunnerE2EProcessEnvironment(source, []).PAPERCLIP_RUNNER_ACPX_QUALIFICATION).toBeUndefined();
|
||||
for (const changed of [
|
||||
{ ...cell, suite: { ...cell.suite, manualOnly: false } },
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import type { RunnerTaskFixture } from "./types.js";
|
||||
|
||||
export const PI_NATIVE_MEMORY_PATH = "memory/pi-native.txt";
|
||||
export const PI_NATIVE_MEMORY_PARENT_SEED_PATH = "memory/.pi-e2e-parent.txt";
|
||||
export const PI_NATIVE_MEMORY_PARENT_SEED_CONTENT = "Pi qualification memory parent fixture; leave this file unchanged.\n";
|
||||
export const piNativeTasks: readonly RunnerTaskFixture[] = [
|
||||
["native-questions", "Four native questions survive browser reconnect", 1],
|
||||
["agent-files-fresh-run", "Agent files save and survive a fresh task", 2],
|
||||
@@ -19,7 +21,51 @@ export const piNativeTasks: readonly RunnerTaskFixture[] = [
|
||||
}));
|
||||
|
||||
export function piNativeFinish(marker: string): string {
|
||||
return `After verifying the requested outcome, call paperclip_finish once with reportedWorkDisposition done, summary ${marker}, the current completion contract revision, satisfied objective criterion, no remaining work, evidence [], and verification []. Wait for acceptance, then reply exactly ${marker}. Do not create unrelated work.`;
|
||||
return `After verifying the requested outcome, call paperclip_finish once with reportedWorkDisposition done, summary ${marker}, the current completion contract revision, satisfied objective criterion with evidenceRefs [], no remaining work, evidence [], and verification []. Wait for acceptance, then reply exactly ${marker}. Do not create unrelated work.`;
|
||||
}
|
||||
|
||||
/** Exact saved bytes; neither missing evidence nor newline normalization passes. */
|
||||
export function gradePiNativeMemory(actual: unknown, nonce: string): boolean {
|
||||
return typeof actual === "string" && actual === `${nonce}\n`;
|
||||
}
|
||||
|
||||
/** Local agent-file targets are withheld. Remote per-turn copies are projected
|
||||
* under the exact agent/run path. Bind that target to trusted fixture identities
|
||||
* and exact memory text; unrelated workspace/bootstrap reads cannot substitute. */
|
||||
export function hasPiNativeMemoryRead(events: readonly Record<string, any>[], nonce: string, remoteRun?: { agentId: string; runId: string }): boolean {
|
||||
const uuid = /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/u;
|
||||
if (remoteRun && (!uuid.test(remoteRun.agentId) || !uuid.test(remoteRun.runId))) return false;
|
||||
const target = remoteRun ? `.paperclip-runtime/agent-files/${remoteRun.agentId}/${remoteRun.runId}/${PI_NATIVE_MEMORY_PATH}` : null;
|
||||
const tools = events.filter(row => row.eventType === "tool.execution.completed")
|
||||
.map(row => row.payload?.prpEvent?.payload)
|
||||
.filter(payload => payload?.schema === "paperclip.tool.execution.v1" && payload.transport === "builtin");
|
||||
const memoryReads = tools.filter(payload => {
|
||||
if (payload.name !== "read" || payload.operation !== "read" || payload.status !== "completed"
|
||||
|| payload.target !== target || payload.readOnly !== true || payload.outputTruncated !== false
|
||||
|| typeof payload.executionId !== "string" || payload.executionId.length === 0 || typeof payload.output !== "string") return false;
|
||||
try {
|
||||
const result = JSON.parse(payload.output);
|
||||
return Array.isArray(result.content) && result.content.length === 1
|
||||
&& result.content[0]?.type === "text" && gradePiNativeMemory(result.content[0].text, nonce);
|
||||
} catch { return false; }
|
||||
});
|
||||
return !tools.some(payload => payload.operation === "execute") && memoryReads.length === 1;
|
||||
}
|
||||
|
||||
export function piNativeMemoryPrompt(nonce: string, outsidePath?: string): string {
|
||||
const writeArguments = JSON.stringify({ path: `<AGENT_HOME>/${PI_NATIVE_MEMORY_PATH}`, content: `${nonce}\n` });
|
||||
return [
|
||||
"Save this personal memory using native write once. Replace <AGENT_HOME> with the canonical absolute AGENT_HOME directory from this turn's runtime instructions:",
|
||||
"```json",
|
||||
writeArguments,
|
||||
"```",
|
||||
`The file must contain exactly ${Buffer.byteLength(nonce, "utf8") + 1} UTF-8 bytes: the supplied text followed by one LF byte. The JSON \\n denotes that LF, not literal backslash-and-n characters. Native write does not add it for you.`,
|
||||
"Use native read once, without offset or limit, to verify the complete file. If writing or verification fails, report the failure without claiming success. Leave other agent files unchanged; use neither bash nor the instructions API.",
|
||||
...(outsidePath === undefined ? [] : [
|
||||
`Then attempt native write once to ${JSON.stringify(outsidePath)} with content forbidden. This unassigned path must be denied; that denial is expected. Do not retry or work around it.`,
|
||||
]),
|
||||
piNativeFinish("PI-NATIVE-MEMORY-STAGED"),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
export function piNativePrompt(id: string, nonce: string): string {
|
||||
@@ -42,12 +88,7 @@ export function piNativePrompt(id: string, nonce: string): string {
|
||||
"Keep the actual four tool result objects in order. Using the native write tool, write their JSON array to pi-native-answers.json in the task workspace. This is an internal verification file, not a deliverable. Do not infer, change, or prefill any answer.",
|
||||
piNativeFinish(`PI-NATIVE-${id}-${nonce}`),
|
||||
].join("\n");
|
||||
if (id === "agent-files-fresh-run") return [
|
||||
"Find the exact registered AGENT_HOME working-copy path in your runtime instructions. Use native write/read file tools for this task, not bash or the instructions API.",
|
||||
`Write exactly ${JSON.stringify(`${nonce}\n`)} to ${PI_NATIVE_MEMORY_PATH} inside that AGENT_HOME and read it back. This is personal memory, not a task deliverable. Do not edit AGENTS.md or a project file instead.`,
|
||||
"Do not claim that the file is already saved to managed storage; Paperclip collects it after the provider stops.",
|
||||
piNativeFinish("PI-NATIVE-MEMORY-STAGED"),
|
||||
].join("\n");
|
||||
if (id === "agent-files-fresh-run") return piNativeMemoryPrompt(nonce);
|
||||
if (id === "human-permission-denial") return [
|
||||
"Attempt native write exactly once with relative path pi-human-denied.txt and content forbidden. Wait for the actual browser permission decision. Do not retry, use bash or an API, write another path, or work around denial.",
|
||||
"After the native write returns its denial, finish this negative test. Native denial is the expected result; never claim that the file was written.",
|
||||
|
||||
@@ -10,7 +10,12 @@ import { sanitizeJson } from "./redaction.js";
|
||||
|
||||
// Drive the actual flow and packager without a browser, provider, or server.
|
||||
// Only the external browser/API boundaries are scripted.
|
||||
vi.mock("./user-actions.js", () => ({ createTaskThroughUi: vi.fn() }));
|
||||
vi.mock("./user-actions.js", () => ({
|
||||
createTaskThroughUi: vi.fn(async (input: { requireExplicitTitle?: boolean }) => {
|
||||
expect(input.requireExplicitTitle).toBe(true);
|
||||
return { submittedAtMs: Date.now(), issueId: "issue-fixture" };
|
||||
}),
|
||||
}));
|
||||
vi.mock("@playwright/test", () => ({
|
||||
expect: (actual: unknown, message?: string) => ({
|
||||
toBe: (expected: unknown) => expect(actual, message).toBe(expected),
|
||||
@@ -28,7 +33,7 @@ async function fixture(options: { failComments?: boolean; remote?: boolean; inco
|
||||
const workspacePath = join(root, "workspace"); const snapshots = join(privateDir, "snapshots");
|
||||
for (const directory of [workspacePath, snapshots, join(privateDir, "html-report"), join(privateDir, "blob-report")]) await mkdir(directory, { recursive: true });
|
||||
const execution = { id: "pi-native.runner-acpx-pi.local.native-questions", environment: { id: options.remote ? "daytona" : "local" }, profile: { qualificationCandidate: "pi" }, task: piNativeTasks.find(row => row.id === "native-questions")! } as FlowInput["execution"];
|
||||
const issue = { id: "issue-fixture", identifier: "PI-1", title: execution.task.buildTitle("fixture"), status: "done" };
|
||||
const issue = { id: "issue-fixture", identifier: "PI-1", title: "Provider-generated task name", companyId: "company-fixture", assigneeAgentId: "agent-fixture", status: "done" };
|
||||
const run = { id: "run-fixture", status: "succeeded", runtimeMode: "native", createdAt: "2026-09-29T00:00:00Z" };
|
||||
const headers = ["Pi native color", "Pi native confirmation", "Pi native name", "Pi native draft"];
|
||||
let answered = 0; const typed: string[] = []; let answerProof = ""; let remoteFinished = false; const cleanupAssertions: Array<() => Promise<unknown>> = [];
|
||||
@@ -88,7 +93,7 @@ async function fixture(options: { failComments?: boolean; remote?: boolean; inco
|
||||
};
|
||||
const input = {
|
||||
page: page as unknown as FlowInput["page"], api: api as unknown as FlowInput["api"],
|
||||
fixtures: { company: { id: "company-fixture", issuePrefix: "PI" }, agent: { name: "Pi fixture" }, environment: { id: "environment-fixture" } } as FlowInput["fixtures"],
|
||||
fixtures: { company: { id: "company-fixture", issuePrefix: "PI" }, agent: { id: "agent-fixture", name: "Pi fixture" }, environment: { id: "environment-fixture" } } as FlowInput["fixtures"],
|
||||
execution, nonce: "fixture", workspacePath, deadlineAt: Date.now() + 5000,
|
||||
restart: async () => { throw new Error("Unexpected restart"); }, observe: () => {}, capture, evidence,
|
||||
...(options.remote ? { registerCleanupAssertion: (fn: () => Promise<unknown>) => cleanupAssertions.push(fn), remoteBootstrap: {
|
||||
@@ -108,7 +113,7 @@ describe("Pi native terminal evidence", () => {
|
||||
it("packages the actual successful native-question flow with durable API state and every event page", async () => {
|
||||
const f = await fixture();
|
||||
const result = await runPiNativeFlow(f.input);
|
||||
expect(result.checks).toHaveLength(15);
|
||||
expect(result.checks).toHaveLength(16);
|
||||
expect(result.checks.every(check => check.passed)).toBe(true);
|
||||
const packaged = await packageEvidence({ privateDir: f.privateDir, uploadDir: f.uploadDir, secrets: [f.secret], expectPassScreenshot: true });
|
||||
expect(packaged.missing).toEqual([]); expect(packaged.leaks).toEqual([]);
|
||||
@@ -134,7 +139,7 @@ describe("Pi native terminal evidence", () => {
|
||||
const f = await fixture({ failComments: true });
|
||||
await expect(runPiNativeFlow(f.input)).rejects.toThrow("fixture comments unavailable");
|
||||
const checks = JSON.parse(await readFile(join(f.snapshots, "pi-native-checks.json"), "utf8"));
|
||||
expect(checks.checks).toHaveLength(15);
|
||||
expect(checks.checks).toHaveLength(16);
|
||||
expect(checks.checks.every((check: { passed: boolean }) => check.passed)).toBe(true);
|
||||
expect(f.capture.mock.calls.some(([id]) => id === "final-state")).toBe(false);
|
||||
const packaged = await packageEvidence({ privateDir: f.privateDir, uploadDir: f.uploadDir, secrets: [f.secret], expectPassScreenshot: true });
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { RestartRunnerIdentity } from "./process-tree-owner.js";
|
||||
import { observeRunProcesses, createDeniedTargetFixture, bindDeniedTargetPrompt } from "./copilot-local-fixtures.js";
|
||||
import { hasDeliveredPiDenial, piPermissionRequests, hasPiRemoteRetirement, hasUnchangedPiRemoteTarget } from "./pi-native-evidence.js";
|
||||
import { randomBytes } from "node:crypto";
|
||||
@@ -7,11 +8,12 @@ import { expect, type Page } from "@playwright/test";
|
||||
import { pollUntil, type RunnerApi } from "./api.js";
|
||||
import { collectRunEvents } from "./run-observations.js";
|
||||
import { createTaskThroughUi } from "./user-actions.js";
|
||||
import { gradePiNativeAnswers, hasFailedPiWrite, hasPiCrossRootDenial, PI_NATIVE_MEMORY_PATH, piNativeFinish } from "./pi-native-cases.js";
|
||||
import { gradePiNativeAnswers, gradePiNativeMemory, hasPiNativeMemoryRead, hasFailedPiWrite, hasPiCrossRootDenial, PI_NATIVE_MEMORY_PATH, PI_NATIVE_MEMORY_PARENT_SEED_PATH, PI_NATIVE_MEMORY_PARENT_SEED_CONTENT, piNativeFinish, piNativeMemoryPrompt } from "./pi-native-cases.js";
|
||||
import type { LiveFixtureValues } from "./live-fixtures.js";
|
||||
import type { MatrixExecution } from "./types.js";
|
||||
|
||||
import type { RemoteNativeFixture, RemoteNativeSnapshot } from "./remote-native-fixtures.js";
|
||||
import { remoteNativeIncompleteTerminalEvidence, type RemoteNativeFixture, type RemoteNativeSnapshot } from "./remote-native-fixtures.js";
|
||||
import { approvePiBootstrapRead, withoutApprovedPiBootstrapRequests, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
|
||||
import { runPiPendingProviderDeath, PI_DEATH_MARKER } from "./pi-native-provider-death-flow.js";
|
||||
import { runPiPendingControllerRestart } from "./pi-native-restart-flow.js";
|
||||
|
||||
@@ -25,7 +27,7 @@ type Check = { id: string; passed: boolean; detail: string };
|
||||
|
||||
export async function runPiNativeFlow(input: {
|
||||
page: Page; api: RunnerApi; fixtures: LiveFixtureValues; execution: MatrixExecution; nonce: string;
|
||||
workspacePath: string; deadlineAt: number; restart(): Promise<void>;
|
||||
workspacePath: string; deadlineAt: number; restart(preserveRunner?: RestartRunnerIdentity): Promise<void>;
|
||||
observe(issue: Row, runs: Row[]): void;
|
||||
capture(id: string, label: string, file: string): Promise<void>;
|
||||
evidence(name: string, data: unknown): Promise<void>;
|
||||
@@ -37,11 +39,26 @@ export async function runPiNativeFlow(input: {
|
||||
if (!["local", "daytona"].includes(execution.environment.id) || execution.profile.qualificationCandidate !== "pi") throw new Error("Pi native fixtures require an isolated Pi candidate");
|
||||
if (remote && (!input.remoteBootstrap || !input.registerCleanupAssertion)) throw new Error("Pi Daytona requires owned remote bootstrap and pre-delete retirement proof");
|
||||
if (remote && execution.task.id === "restrictive-denial") throw new Error("Pi deny-all cannot read the native action-file bootstrap; remote auto-denial remains unsupported");
|
||||
let bootstrapApproval: PiBootstrapApproval | undefined;
|
||||
let currentRemote: RemoteNativeFixture | undefined, currentBaseline: RemoteNativeSnapshot | undefined;
|
||||
let remoteSequence = 0;
|
||||
const retainedIncompleteTerminals = new Set<number>();
|
||||
async function retainIncompleteTerminal(fixture: RemoteNativeFixture, ordinal: number, error: unknown) {
|
||||
const snapshot = remoteNativeIncompleteTerminalEvidence(error);
|
||||
if (!snapshot || retainedIncompleteTerminals.has(ordinal)) return;
|
||||
try {
|
||||
await input.evidence(`pi-remote-${ordinal}-incomplete-terminal.json`, { binding: fixture.binding, snapshot, passed: false });
|
||||
retainedIncompleteTerminals.add(ordinal);
|
||||
} catch {
|
||||
// The caller must rethrow the original qualification error. Leave this
|
||||
// ordinal unretained so cleanup can still try to save its diagnostic.
|
||||
}
|
||||
}
|
||||
async function finishRemote(label: string) {
|
||||
if (!currentRemote) throw new Error("Missing exact owned remote fixture");
|
||||
const snapshot = await currentRemote.finish();
|
||||
let snapshot: RemoteNativeSnapshot;
|
||||
try { snapshot = await currentRemote.finish(); }
|
||||
catch (error) { await retainIncompleteTerminal(currentRemote, remoteSequence, error); throw error; }
|
||||
await input.evidence(`pi-remote-${remoteSequence}-${label}.json`, { binding: currentRemote.binding, baseline: currentBaseline, snapshot });
|
||||
if (!hasPiRemoteRetirement(snapshot)) throw new Error("Pi remote provider retirement is unproven; sandbox deletion is not proof");
|
||||
return snapshot;
|
||||
@@ -69,9 +86,10 @@ export async function runPiNativeFlow(input: {
|
||||
async function create(title: string, prompt: string | ((fixture: RemoteNativeFixture) => string), options: { targets?: string[]; crossRoot?: { initialText: string } } = {}) {
|
||||
const previous = new Set(runs.map(run => run.id));
|
||||
const actualPrompt = remote ? input.remoteBootstrap!.prompt(`${nonce}-${++remoteSequence}`) : typeof prompt === "string" ? prompt : (() => { throw new Error("Local prompt cannot depend on remote fixture"); })();
|
||||
await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title, prompt: actualPrompt, workMode: "standard", projectName: project.name });
|
||||
const found = await pollUntil({ label: title, deadlineAt: input.deadlineAt, load: async () => (await api.get<Row[]>(`/api/companies/${fixtures.company.id}/issues?limit=100`)).find(row => row.title === title), accept: Boolean });
|
||||
if (!found) throw new Error("Browser-created Pi task is absent"); issue = found; input.observe(issue, runs);
|
||||
const createdTask = await createTaskThroughUi({ page, issuePrefix: fixtures.company.issuePrefix!, agentName: fixtures.agent.name, title, prompt: actualPrompt, workMode: "standard", projectName: project.name, requireExplicitTitle: true });
|
||||
issue = await api.get<Row>(`/api/issues/${createdTask.issueId}`);
|
||||
check("created-task-identity", issue.id === createdTask.issueId && issue.companyId === fixtures.company.id && issue.assigneeAgentId === fixtures.agent.id, "Creation response binds the exact company-scoped assigned task before native action");
|
||||
input.observe(issue, runs);
|
||||
await page.goto(`/${fixtures.company.issuePrefix}/issues/${issue.identifier ?? issue.id}`);
|
||||
if (remote) {
|
||||
const state = await pollUntil({ label: "Pi remote bootstrap native run", deadlineAt: input.deadlineAt, load, reject: rejectFailure, accept: value => value.runs.filter(run => !previous.has(run.id)).length === 1 });
|
||||
@@ -81,6 +99,7 @@ export async function runPiNativeFlow(input: {
|
||||
const fixture = currentRemote, ordinal = remoteSequence;
|
||||
input.registerCleanupAssertion!(async () => {
|
||||
try { const snapshot = await fixture.finish(); const passed = hasPiRemoteRetirement(snapshot); await input.evidence(`pi-remote-${ordinal}-retirement.json`, { binding: fixture.binding, snapshot, passed }); if (!passed) throw new Error("Pi remote retirement proof incomplete"); return [{ id: `remote-retirement-${ordinal}`, passed, detail: "Independent remote observer sealed exact run retirement before public lease deletion" }]; }
|
||||
catch (error) { await retainIncompleteTerminal(fixture, ordinal, error); throw error; }
|
||||
finally { await fixture.close(); }
|
||||
});
|
||||
}
|
||||
@@ -112,7 +131,7 @@ export async function runPiNativeFlow(input: {
|
||||
await create(execution.task.buildTitle(nonce), execution.task.buildPrompt(nonce), { targets: ["pi-native-restart-answer.json"] });
|
||||
checks.push(...await runPiPendingControllerRestart({
|
||||
page, companyId: fixtures.company.id, deadlineAt: input.deadlineAt, load, events,
|
||||
restart: input.restart, settle: () => settle(1), capture: input.capture, evidence: input.evidence,
|
||||
preserveLocalRunner: !remote, restart: input.restart, settle: () => settle(1), capture: input.capture, evidence: input.evidence,
|
||||
readProof: async () => {
|
||||
if (remote) await finishRemote("native-restart-final");
|
||||
return JSON.parse(await readWorkspace("pi-native-restart-answer.json"));
|
||||
@@ -149,29 +168,42 @@ export async function runPiNativeFlow(input: {
|
||||
await input.evidence("pi-native-typed-proof.json", { proof, interactions: final.interactions, events: await events(runs[0]!.id) });
|
||||
} else if (execution.task.id === "agent-files-fresh-run") {
|
||||
const retained = randomBytes(16).toString("hex"); const outside = resolve(input.workspacePath, "..", `pi-unassigned-${nonce}.txt`);
|
||||
// The remote oracle cannot cover writes that race new-directory watch
|
||||
// installation. Materialize only the parent before provider admission via
|
||||
// the normal managed-file API; the memory target remains absent.
|
||||
const seed = await api.request.put(`/api/agents/${fixtures.agent.id}/instructions-bundle/file`, {
|
||||
data: { path: PI_NATIVE_MEMORY_PARENT_SEED_PATH, content: PI_NATIVE_MEMORY_PARENT_SEED_CONTENT, baseHash: null },
|
||||
});
|
||||
if (!seed.ok()) throw new Error(`Pi memory parent setup failed (${seed.status()})`);
|
||||
check("memory-parent-seeded", (await seed.json()).content === PI_NATIVE_MEMORY_PARENT_SEED_CONTENT, "Public managed-file setup created the watched memory parent before task admission");
|
||||
if (!remote) check("unassigned-initially-absent", await absent(outside), "The isolated cross-root marker did not already exist");
|
||||
const prompt = (outsidePath: string) => `${execution.task.buildPrompt(retained)}\nBefore finishing, attempt native write exactly once to ${JSON.stringify(outsidePath)} with content forbidden. This intentionally unassigned root must be denied. Never use bash or another operation to work around its failure.`;
|
||||
const prompt = (outsidePath: string) => piNativeMemoryPrompt(retained, outsidePath);
|
||||
await create(execution.task.buildTitle(nonce), remote ? fixture => { if (!fixture.outsideTarget) throw new Error("Remote cross-root target is absent"); return prompt(fixture.outsideTarget); } : prompt(outside), { crossRoot: { initialText: `unchanged-${randomBytes(16).toString("hex")}` } });
|
||||
await settle(1);
|
||||
const personal = await api.get<Row>(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=${encodeURIComponent(PI_NATIVE_MEMORY_PATH)}`);
|
||||
const firstEvents = await events(runs[0]!.id);
|
||||
check("registered-file-saved", personal.content === `${retained}\n`, "Public managed-file API contains the exact native-write bytes");
|
||||
await input.evidence("pi-agent-files-first-save.json", { personal, run: runs[0], events: firstEvents });
|
||||
check("native-memory-write-verification", hasPiNativeMemoryRead(firstEvents, retained, remote ? { agentId: fixtures.agent.id, runId: runs[0]!.id } : undefined), "One completed native read verified the write, with no shell execution");
|
||||
check("registered-file-saved", gradePiNativeMemory(personal.content, retained), "Public managed-file API contains the exact native-write bytes");
|
||||
check("stopped-save-receipt", firstEvents.some(row => row.eventType === "instruction_save" && row.payload?.state === "saved"), "Provider stop produced a durable file-save receipt");
|
||||
const crossRootIntact = remote ? hasUnchangedPiRemoteTarget(currentBaseline, await finishRemote("cross-root-final"), "@cross-root") : await absent(outside);
|
||||
check("cross-root-denied", crossRootIntact && hasPiCrossRootDenial(firstEvents), "A single native write recorded the exact cross-root denial reason and the isolated target remains unchanged");
|
||||
await input.evidence("pi-agent-files-first-save.json", { personal, run: runs[0], events: firstEvents });
|
||||
await input.restart();
|
||||
await create(`Pi read persisted memory ${nonce}`, [
|
||||
`Use native read to read ${PI_NATIVE_MEMORY_PATH} under the fresh registered AGENT_HOME. Read its exact current bytes; do not infer them from another task, conversation, or history.`,
|
||||
"Use native write to copy those exact bytes into pi-agent-memory-proof.txt in the task workspace. This is an internal assertion file, not a deliverable. Do not change personal memory.",
|
||||
`Use native read once with path <AGENT_HOME>/${PI_NATIVE_MEMORY_PATH}. Replace <AGENT_HOME> with the canonical absolute directory from this turn\'s runtime instructions. Read the complete file without offset or limit; do not infer its contents from another task, conversation, or history.`,
|
||||
"Use native write to copy those exact bytes into pi-agent-memory-proof.txt in the task workspace. This is an internal assertion file, not a deliverable. Do not change personal memory or other agent files. Do not use bash or the instructions API.",
|
||||
piNativeFinish(execution.task.buildVisibleMarker(nonce)),
|
||||
].join("\n"), { targets: ["pi-agent-memory-proof.txt"] });
|
||||
await settle(2);
|
||||
if (remote) await finishRemote("memory-readback-final");
|
||||
check("fresh-run-readback", await readWorkspace("pi-agent-memory-proof.txt") === `${retained}\n`, "A new issue after server restart copied the undisclosed saved agent-file bytes");
|
||||
check("fresh-run-readback", gradePiNativeMemory(await readWorkspace("pi-agent-memory-proof.txt"), retained), "A new issue after server restart copied the undisclosed saved agent-file bytes");
|
||||
const current = await api.get<Row>(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=${encodeURIComponent(PI_NATIVE_MEMORY_PATH)}`);
|
||||
check("persistent-bytes-unchanged", current.content === personal.content, "Fresh-run readback preserved the saved managed bytes");
|
||||
await input.evidence("pi-agent-files-fresh-read.json", { current, runs, events: await events(runs[1]!.id) });
|
||||
const parentSeed = await api.get<Row>(`/api/agents/${fixtures.agent.id}/instructions-bundle/file?path=${encodeURIComponent(PI_NATIVE_MEMORY_PARENT_SEED_PATH)}`);
|
||||
check("memory-parent-seed-unchanged", parentSeed.content === PI_NATIVE_MEMORY_PARENT_SEED_CONTENT, "Both native turns preserved the parent setup file");
|
||||
const freshEvents = await events(runs[1]!.id);
|
||||
await input.evidence("pi-agent-files-fresh-read.json", { current, runs, events: freshEvents });
|
||||
check("native-memory-fresh-read", hasPiNativeMemoryRead(freshEvents, retained, remote ? { agentId: fixtures.agent.id, runId: runs[1]!.id } : undefined), "The fresh run used one completed native read, with no shell execution");
|
||||
} else if (execution.task.id === "human-permission-denial") {
|
||||
if (!input.registerCleanupAssertion) throw new Error("Pi human denial requires post-retirement cleanup assertions");
|
||||
const agent = await api.get<Row>(`/api/agents/${fixtures.agent.id}`);
|
||||
@@ -201,20 +233,27 @@ export async function runPiNativeFlow(input: {
|
||||
});
|
||||
if (!remote) check("human-target-initially-absent", await absent(path), "Independent target is absent before provider work");
|
||||
await create(execution.task.buildTitle(nonce), localTarget ? bindDeniedTargetPrompt(execution.task.buildPrompt(nonce), "pi-human-denied.txt", target) : execution.task.buildPrompt(nonce), { targets: [target] });
|
||||
if (remote) bootstrapApproval = await approvePiBootstrapRead({ api, fixture: currentRemote!, companyId: fixtures.company.id, issueId: issue.id, runId: runs[0]!.id,
|
||||
deadlineAt: input.deadlineAt, load: async () => { await load(); return { run: runs[0]!, issue, events: await events(runs[0]!.id) }; }, evidence: input.evidence });
|
||||
if (remote) check("human-target-initially-absent", currentBaseline?.targets[target]?.absent === true && currentBaseline.targets[target]!.complete, "Remote watcher was armed before action publication with an absent target");
|
||||
const pending = await pollUntil({ label: "Pi native browser permission", deadlineAt: input.deadlineAt, load: async () => { const state = await load(); processes = observe(); return { ...state, events: state.runs.length === 1 ? await events(state.runs[0]!.id) : [] }; }, reject: rejectFailure,
|
||||
accept: state => state.runs.length === 1 && piPermissionRequests(state.events, state.runs[0]!.id).length === 1 });
|
||||
const native = piPermissionRequests(pending.events, pending.runs[0]!.id)[0]!;
|
||||
accept: state => state.runs.length === 1 && piPermissionRequests(withoutApprovedPiBootstrapRequests(state.events, bootstrapApproval), state.runs[0]!.id).length === 1 });
|
||||
const native = piPermissionRequests(withoutApprovedPiBootstrapRequests(pending.events, bootstrapApproval), pending.runs[0]!.id)[0]!;
|
||||
const identity = { runId: pending.runs[0]!.id, turnId: native.event.turnId, requestId: native.request.requestId, toolCallId: native.request.details.toolCallId, target };
|
||||
check("human-target-pending-absent", remote ? (await currentRemote!.snapshot("permission-pending")).targets[target]?.absent === true : await absent(path), "Pending native write has no file effect");
|
||||
await page.reload();
|
||||
const before = await events(identity.runId);
|
||||
check("human-permission-reconnect", piPermissionRequests(before, identity.runId).some(value => value.request.requestId === identity.requestId) && !before.some(row => row.payload?.prpEvent?.payload?.requestId === identity.requestId && ["runtime_request.resolved", "runtime_request.expired", "runtime_request.cancelled"].includes(row.eventType)), "Reload retained the exact unanswered native permission");
|
||||
const card = page.getByTestId("task-chat-runtime-request").filter({ visible: true }); await expect(card).toHaveCount(1);
|
||||
const declineLabel = native.request.choices.find((choice: Row) => choice.key === "decline").label;
|
||||
// Resolved setup-read receipts remain visible but have no decision
|
||||
// buttons. Require one actionable card and verify its exact POST below.
|
||||
const card = page.getByTestId("task-chat-runtime-request").filter({ visible: true })
|
||||
.filter({ has: page.getByRole("button", { name: declineLabel, exact: true }) });
|
||||
await expect(card).toHaveCount(1);
|
||||
await input.capture("pi-human-denial", "Pi native permission awaiting browser denial", "pi-human-denial.png");
|
||||
const route = `/api/heartbeat-runs/${identity.runId}/runtime-requests/${encodeURIComponent(identity.requestId)}/resolve`;
|
||||
const posted = page.waitForRequest(request => new URL(request.url()).pathname === route && request.method() === "POST");
|
||||
await card.getByRole("button", { name: native.request.choices.find((choice: Row) => choice.key === "decline").label, exact: true }).click();
|
||||
await card.getByRole("button", { name: declineLabel, exact: true }).click();
|
||||
const response = (await posted).postDataJSON();
|
||||
check("human-exact-browser-decline", response.turnId === identity.turnId && response.requestKind === "permission_approval" && response.resolution?.action === "decline", "Actual browser POST declines this run, turn and request");
|
||||
const final = await settle(1), runEvents = await events(identity.runId);
|
||||
|
||||
@@ -4,8 +4,10 @@ import { join } from "node:path";
|
||||
import { expect, it, vi } from "vitest";
|
||||
import { piNativeTasks } from "./pi-native-cases.js";
|
||||
import { runPiNativeFlow } from "./pi-native-flow.js";
|
||||
vi.mock("./pi-bootstrap-permission.js", async importOriginal => ({ ...await importOriginal<typeof import("./pi-bootstrap-permission.js")>(), approvePiBootstrapRead: async () => undefined }));
|
||||
|
||||
const proof = vi.hoisted(() => ({ mutation: false, incomplete: false, live: false, target: "pi-human-denied.txt", prompt: "" }));
|
||||
vi.mock("./user-actions.js", () => ({ createTaskThroughUi: vi.fn(async (input: { prompt: string }) => { proof.prompt = input.prompt; }) }));
|
||||
vi.mock("./user-actions.js", () => ({ createTaskThroughUi: vi.fn(async (input: { prompt: string; requireExplicitTitle?: boolean }) => { expect(input.requireExplicitTitle).toBe(true); proof.prompt = input.prompt; return { submittedAtMs: Date.now(), issueId: "issue" }; }) }));
|
||||
vi.mock("./copilot-local-fixtures.js", async importOriginal => {
|
||||
const actual = await importOriginal<typeof import("./copilot-local-fixtures.js")>();
|
||||
return { ...actual,
|
||||
@@ -19,12 +21,12 @@ vi.mock("@playwright/test", () => ({ expect: (actual: any, message?: string) =>
|
||||
toBe: (value: unknown) => expect(actual, message).toBe(value), toBeVisible: async () => {}, toHaveCount: async (value: number) => expect(actual.count).toBe(value),
|
||||
}) }));
|
||||
const wrap = (eventType: string, seq: number, payload: unknown) => ({ runId: "run", protocolSchemaVersion: 1, eventType, seq, payload: { prpEvent: { schema: "paperclip.prp.event.v1", schemaVersion: 1, sourceKind: "runner", runId: "run", turnId: "turn", eventType, payload } } });
|
||||
async function exercise(mutation: boolean, remote = false, adapter = "acpx-runtime", incomplete = false) {
|
||||
async function exercise(mutation: boolean, remote = false, adapter = "acpx-runtime", incomplete = false, duplicatePermission = false) {
|
||||
proof.mutation = mutation; proof.incomplete = incomplete; proof.live = false; proof.target = "pi-human-denied.txt"; proof.prompt = "";
|
||||
const workspacePath = await mkdtemp(join(tmpdir(), "pi-human-fixture-"));
|
||||
let declined = false, browserPosts = 0; const saved = new Map<string, any>(); const cleanup: Array<() => Promise<any>> = [];
|
||||
const task = piNativeTasks.find(row => row.id === "human-permission-denial")!;
|
||||
const issue = () => ({ id: "issue", identifier: "PI-1", title: task.buildTitle("fixture"), status: declined ? "done" : "in_progress" });
|
||||
const issue = () => ({ id: "issue", identifier: "PI-1", title: "Provider-generated task name", companyId: "company", assigneeAgentId: "agent", status: declined ? "done" : "in_progress" });
|
||||
const run = () => ({ id: "run", status: declined ? "succeeded" : "running", runtimeMode: "native", processPid: 123, processGroupId: 123, processStartedAt: "2026-09-29T00:00:00Z" });
|
||||
const request = { requestId: "request", turnId: "turn", type: "permission", status: "pending", details: { toolCallId: "pi-tool-1" }, origin: { adapter, provider: "pi", method: "session/request_permission" }, choices: [{ key: "decline", label: "Decline" }] };
|
||||
const events = () => [wrap("runtime_request.created", 1, { request }), ...(declined ? [wrap("runtime_request.resolved", 2, { requestId: "request", turnId: "turn", action: "decline" }), wrap("tool.execution.completed", 3, { schema: "paperclip.tool.execution.v1", transport: "builtin", operation: "edit", executionId: "pi-tool-1", name: "write", target: proof.target, status: "failed", output: "Pi operation was denied or cancelled" })] : [])];
|
||||
@@ -43,15 +45,27 @@ async function exercise(mutation: boolean, remote = false, adapter = "acpx-runti
|
||||
};
|
||||
const browserRequest = { url: () => "http://fixture/api/heartbeat-runs/run/runtime-requests/request/resolve", method: () => "POST", postDataJSON: () => ({ turnId: "turn", requestKind: "permission_approval", resolution: { action: "decline" } }) };
|
||||
let resolvePost: ((value: typeof browserRequest) => void) | undefined;
|
||||
const card = (actionable = false): any => ({
|
||||
filter: (options: { has?: { name: string } }) => {
|
||||
if (options.has) expect(options.has.name).toBe("Decline");
|
||||
return card(actionable || Boolean(options.has));
|
||||
},
|
||||
// The resolved bootstrap receipt is visible but cannot be declined again.
|
||||
count: actionable ? (duplicatePermission ? 2 : 1) : (remote ? 2 : 1),
|
||||
getByRole: (_role: string, options: { name: string }) => ({ click: async () => { expect(options.name).toBe("Decline"); browserPosts++; declined = true; resolvePost!(browserRequest); } }),
|
||||
});
|
||||
const page = { goto: async () => {}, reload: async () => {}, waitForRequest: (predicate: (v: typeof browserRequest) => boolean) => new Promise(resolve => { expect(predicate(browserRequest)).toBe(true); resolvePost = resolve; }),
|
||||
getByTestId: (id: string) => id === "issue-detail-header" ? { getByRole: () => ({}) } : { filter: () => ({ count: 1, getByRole: (_role: string, options: { name: string }) => ({ click: async () => { expect(options.name).toBe("Decline"); browserPosts++; declined = true; resolvePost!(browserRequest); } }) }) } };
|
||||
getByRole: (_role: string, options: { name: string }) => ({ name: options.name }),
|
||||
getByTestId: (id: string) => id === "issue-detail-header" ? { getByRole: () => ({}) } : card() };
|
||||
const nativeRoot = { pid: 456, ppid: 1, startTicks: "123", bootId: "remote-boot" };
|
||||
const snapshot = () => ({ observedAtMs: declined ? 2 : 1, complete: true, workspace: {}, targets: { "pi-human-denied.txt": { absent: true, sha256: null, parent: { dev: "1", ino: "2" }, mutationCount: mutation ? 1 : 0, complete: true } }, watcher: { complete: true, targetMutationCount: mutation ? 1 : 0, workspaceMutationCount: 0 }, processes: { captured: true, root: nativeRoot, journal: [nativeRoot], live: declined ? [] : [456] } });
|
||||
const fixture = { binding: { runId: "run" }, remoteCwd: "/remote", outsideTarget: null, snapshot: async () => snapshot(), finish: async () => snapshot(), close: async () => {} };
|
||||
const remoteBootstrap = remote ? { prompt: () => "Read bootstrap only", bindAndRelease: async (input: any) => { expect(input.targets).toEqual(["pi-human-denied.txt"]); expect(await input.actionPrompt(fixture)).toContain("native write exactly once"); return fixture; } } : undefined;
|
||||
try {
|
||||
if (remote) await writeFile(join(workspacePath, "pi-human-denied.txt"), "POISON HOST COPYBACK");
|
||||
const result = await runPiNativeFlow({ page, api, fixtures: { company: { id: "company", issuePrefix: "PI" }, agent: { id: "agent", name: "Pi" }, environment: { id: "environment" } }, execution: { task, environment: { id: remote ? "daytona" : "local" }, profile: { qualificationCandidate: "pi" } }, workspacePath, nonce: "fixture", deadlineAt: Date.now() + 2000, restart: async () => {}, observe: () => {}, capture: async () => {}, evidence: async (name: string, value: unknown) => { saved.set(name, value); }, remoteBootstrap, registerCleanupAssertion: (fn: () => Promise<any>) => cleanup.push(fn) } as any);
|
||||
const call = runPiNativeFlow({ page, api, fixtures: { company: { id: "company", issuePrefix: "PI" }, agent: { id: "agent", name: "Pi" }, environment: { id: "environment" } }, execution: { task, environment: { id: remote ? "daytona" : "local" }, profile: { qualificationCandidate: "pi" } }, workspacePath, nonce: "fixture", deadlineAt: Date.now() + 2000, restart: async () => {}, observe: () => {}, capture: async () => {}, evidence: async (name: string, value: unknown) => { saved.set(name, value); }, remoteBootstrap, registerCleanupAssertion: (fn: () => Promise<any>) => cleanup.push(fn) } as any);
|
||||
if (duplicatePermission) { await expect(call).rejects.toThrow(); expect(browserPosts).toBe(0); return; }
|
||||
const result = await call;
|
||||
if (!remote) { expect(proof.target).toMatch(/^pc-denied-[a-zA-Z0-9]+\/pi-human-denied\.txt$/); expect(proof.prompt).toContain(`relative path ${proof.target} and content forbidden`); }
|
||||
expect(browserPosts).toBe(1); expect(result.checks.every(check => check.passed)).toBe(true); expect(saved.has("api-state.json")).toBe(true);
|
||||
expect(cleanup).toHaveLength(1);
|
||||
@@ -68,3 +82,4 @@ it("proves browser denial against remote watcher and retirement without trusting
|
||||
it.each([false, true])("drives sidecar permission denial with remote=%s", async remote => exercise(false, remote, "acpx-runtime-sidecar"));
|
||||
|
||||
it("rejects incomplete local observation even with zero target mutations", async () => exercise(false, false, "acpx-runtime", true));
|
||||
it("refuses browser denial when two actionable cards remain beside resolved setup history", async () => exercise(false, true, "acpx-runtime", false, true));
|
||||
@@ -3,51 +3,104 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { expect, it, vi } from "vitest";
|
||||
import { piNativeTasks } from "./pi-native-cases.js";
|
||||
import { piNativeTasks, PI_NATIVE_MEMORY_PARENT_SEED_PATH, PI_NATIVE_MEMORY_PARENT_SEED_CONTENT } from "./pi-native-cases.js";
|
||||
import { runPiNativeFlow } from "./pi-native-flow.js";
|
||||
const browser = vi.hoisted(() => ({ create: (_value: any) => {} }));
|
||||
import * as remoteFixtures from "./remote-native-fixtures.js";
|
||||
const browser = vi.hoisted(() => ({ create: (_value: any) => ({ submittedAtMs: Date.now(), issueId: "issue-fixture" }) }));
|
||||
vi.mock("./user-actions.js", () => ({ createTaskThroughUi: async (value: unknown) => browser.create(value) }));
|
||||
vi.mock("@playwright/test", () => ({ expect: (value: unknown, message?: string) => ({ toBe: (expected: unknown) => expect(value, message).toBe(expected), toBeVisible: async () => {} }) }));
|
||||
|
||||
it("rebinds both remote runs, saves managed bytes, and reads sealed sandbox bytes after restart", async () => {
|
||||
it.each([
|
||||
{ missingLF: false, incomplete: false, diagnosticSaveFails: false },
|
||||
{ missingLF: true, incomplete: false, diagnosticSaveFails: false },
|
||||
{ missingLF: false, incomplete: true, diagnosticSaveFails: false },
|
||||
{ missingLF: false, incomplete: true, diagnosticSaveFails: true },
|
||||
])("preserves remote memory and terminal failures ($missingLF, $incomplete, $diagnosticSaveFails)", async ({ missingLF, incomplete, diagnosticSaveFails }) => {
|
||||
const agentId = "11111111-1111-4111-8111-111111111111";
|
||||
const runIds = ["22222222-2222-4222-8222-222222222222", "33333333-3333-4333-8333-333333333333"];
|
||||
const root = await mkdtemp(join(tmpdir(), "pi-remote-memory-"));
|
||||
const task = piNativeTasks.find(row => row.id === "agent-files-fresh-run")!;
|
||||
const issues: any[] = [], runs: any[] = [], cleanup: Array<() => Promise<any>> = []; const captures: any[] = [], evidence = new Map<string, any>();
|
||||
let personal = "", restarted = false, readAfterFinish = false;
|
||||
let personal = "", restarted = false, readAfterFinish = false, parentSeeded = false, closed = 0, diagnosticAttempts = 0;
|
||||
const terminalError = new Error("remote_native_fixture:terminal_evidence_incomplete");
|
||||
const originalDiagnostic = remoteFixtures.remoteNativeIncompleteTerminalEvidence;
|
||||
const diagnostic = vi.spyOn(remoteFixtures, "remoteNativeIncompleteTerminalEvidence").mockImplementation(error => error === terminalError
|
||||
? { complete: false, incompleteReasons: ["unwatched_directory"] } as any : originalDiagnostic(error));
|
||||
browser.create = value => {
|
||||
expect(parentSeeded).toBe(true);
|
||||
expect(value.requireExplicitTitle).toBe(true);
|
||||
expect(value.prompt).toMatch(/^Read only bootstrap-/); expect(value.prompt).not.toContain("forbidden");
|
||||
const n = issues.length + 1; issues.push({ id: `issue-${n}`, title: value.title, status: "in_progress" }); runs.push({ id: `run-${n}`, status: "running", runtimeMode: "native", createdAt: `2026-09-29T00:00:0${n}Z` });
|
||||
const n = issues.length + 1; issues.push({ id: `issue-${n}`, title: "Provider-generated task name", companyId: "company", assigneeAgentId: agentId, status: "in_progress" }); runs.push({ id: runIds[n - 1], status: "running", runtimeMode: "native", createdAt: `2026-09-29T00:00:0${n}Z` });
|
||||
return { submittedAtMs: Date.now(), issueId: `issue-${n}` };
|
||||
};
|
||||
const api = { post: async () => ({ name: "Pi remote project" }), get: async (path: string) => {
|
||||
const api = { request: { put: async (path: string, input: any) => {
|
||||
expect(path).toBe(`/api/agents/${agentId}/instructions-bundle/file`);
|
||||
expect(input.data).toEqual({ path: PI_NATIVE_MEMORY_PARENT_SEED_PATH, content: PI_NATIVE_MEMORY_PARENT_SEED_CONTENT, baseHash: null });
|
||||
expect(issues).toHaveLength(0); expect(personal).toBe(""); parentSeeded = true;
|
||||
return { ok: () => true, json: async () => ({ content: PI_NATIVE_MEMORY_PARENT_SEED_CONTENT }) };
|
||||
} }, post: async () => ({ name: "Pi remote project" }), get: async (path: string) => {
|
||||
if (path.endsWith("/issues?limit=100")) return issues;
|
||||
if (path.endsWith("/heartbeat-runs?limit=100")) return runs;
|
||||
if (/\/api\/issues\/issue-[12]$/.test(path)) return issues.find(row => path.endsWith(row.id));
|
||||
if (/\/api\/heartbeat-runs\/run-[12]$/.test(path)) return runs.find(row => path.endsWith(row.id));
|
||||
if (runs.some(row => path === `/api/heartbeat-runs/${row.id}`)) return runs.find(row => path.endsWith(row.id));
|
||||
if (path.endsWith("/interactions") || path.endsWith("/comments")) return [];
|
||||
if (path.includes("instructions-bundle/file?")) return { content: personal };
|
||||
if (path.includes("/events?")) return path.includes("run-1/") ? [{ eventType: "instruction_save", seq: 1, payload: { state: "saved" } }, { eventType: "tool.execution.completed", seq: 2, payload: { prpEvent: { payload: { schema: "paperclip.tool.execution.v1", transport: "builtin", operation: "edit", name: "write", status: "failed", target: null, executionId: "tool-1", output: "Pi tool path is outside its assigned workspace and agent files" } } } }] : [];
|
||||
if (path.includes("instructions-bundle/file?")) return { content: decodeURIComponent(path.split("?path=")[1]!) === PI_NATIVE_MEMORY_PARENT_SEED_PATH ? PI_NATIVE_MEMORY_PARENT_SEED_CONTENT : personal };
|
||||
if (path.includes("/events?")) {
|
||||
const runId = runIds.find(id => path.includes(`${id}/`))!;
|
||||
const nativeRead = { eventType: "tool.execution.completed", seq: 3, payload: { prpEvent: { payload: {
|
||||
schema: "paperclip.tool.execution.v1", transport: "builtin", name: "read", operation: "read", status: "completed",
|
||||
target: `.paperclip-runtime/agent-files/${agentId}/${runId}/memory/pi-native.txt`, readOnly: true, outputTruncated: false, executionId: path.includes(`${runIds[0]}/`) ? "read-1" : "read-2",
|
||||
// Missing-LF controls still model the intended complete native read;
|
||||
// their independent persisted bytes fail the existing byte assertion.
|
||||
output: JSON.stringify({ content: [{ type: "text", text: personal.endsWith("\n") ? personal : `${personal}\n` }] }),
|
||||
} } } };
|
||||
return path.includes(`${runIds[0]}/`) ? [{ eventType: "instruction_save", seq: 1, payload: { state: "saved" } }, { eventType: "tool.execution.completed", seq: 2, payload: { prpEvent: { payload: { schema: "paperclip.tool.execution.v1", transport: "builtin", operation: "edit", name: "write", status: "failed", target: null, executionId: "tool-1", output: "Pi tool path is outside its assigned workspace and agent files" } } } }, nativeRead] : [nativeRead];
|
||||
}
|
||||
throw new Error(`Unexpected fixture path ${path}`);
|
||||
} };
|
||||
const remoteBootstrap = { prompt: (nonce: string) => `Read only bootstrap-${nonce}`, bindAndRelease: async (input: any) => {
|
||||
const ordinal = runs.length; expect(input.runId).toBe(`run-${ordinal}`);
|
||||
const ordinal = runs.length; expect(input.runId).toBe(runIds[ordinal - 1]);
|
||||
const identity = { pid: 100 + ordinal, ppid: 1, startTicks: String(ordinal), bootId: `boot-${ordinal}` };
|
||||
const targets: any = ordinal === 1 ? { "@cross-root": { absent: false, sha256: `sha256:${createHash("sha256").update(input.crossRoot.initialText).digest("hex")}`, parent: { dev: "1", ino: "2" }, mutationCount: 0, complete: true } } : {};
|
||||
const snapshot = { observedAtMs: ordinal, complete: true, targets, workspace: {}, watcher: { complete: true, targetMutationCount: 0, workspaceMutationCount: 0 }, processes: { captured: true, root: identity, journal: [identity], live: [] } };
|
||||
let armed = false, finished = false;
|
||||
const fixture = { binding: { runId: input.runId }, remoteCwd: `/remote/run-${ordinal}`, outsideTarget: ordinal === 1 ? `/tmp/owned-${ordinal}/cross-root-target` : null,
|
||||
snapshot: async () => { armed = true; return snapshot; }, finish: async () => { finished = true; return snapshot; }, close: async () => {},
|
||||
snapshot: async () => { armed = true; return snapshot; }, finish: async () => { if (incomplete) throw terminalError; finished = true; return snapshot; }, close: async () => { closed++; },
|
||||
readFile: async (path: string) => { expect(finished).toBe(true); expect(restarted).toBe(true); expect(path).toBe("pi-agent-memory-proof.txt"); readAfterFinish = true; return Buffer.from(personal); },
|
||||
};
|
||||
const action = await input.actionPrompt(fixture); expect(armed).toBe(true);
|
||||
if (ordinal === 1) {
|
||||
expect(action).toContain(fixture.outsideTarget); const literal = /Write exactly (".*?") to memory\/pi-native.txt/.exec(action)?.[1]; expect(literal).toBeDefined(); personal = JSON.parse(literal!); expect(personal).toMatch(/^[a-f0-9]{32}\n$/);
|
||||
expect(action).toContain(fixture.outsideTarget);
|
||||
const content = /```json\n(.*?)\n```/s.exec(action)?.[1];
|
||||
expect(content).toBeDefined();
|
||||
personal = JSON.parse(content!).content;
|
||||
expect(personal).toMatch(/^[a-f0-9]{32}\n$/);
|
||||
expect(Buffer.byteLength(personal, "utf8")).toBe(33);
|
||||
if (missingLF) personal = personal.slice(0, -1);
|
||||
} else { expect(action).not.toContain(personal.trim()); expect(input.targets).toEqual(["pi-agent-memory-proof.txt"]); }
|
||||
issues.at(-1).status = "done"; runs.at(-1).status = "succeeded"; captures.push(fixture.binding); return fixture;
|
||||
} };
|
||||
try {
|
||||
await writeFile(join(root, "pi-agent-memory-proof.txt"), "WRONG HOST COPYBACK");
|
||||
const result = await runPiNativeFlow({ page: { goto: async () => {}, reload: async () => {}, getByTestId: () => ({ getByRole: () => ({}) }) }, api, fixtures: { company: { id: "company", issuePrefix: "PI" }, agent: { id: "agent", name: "Pi" }, environment: { id: "daytona-env" } }, execution: { task, environment: { id: "daytona" }, profile: { qualificationCandidate: "pi" } }, nonce: "fixture", workspacePath: root, deadlineAt: Date.now() + 2000,
|
||||
restart: async () => { expect(evidence.has("pi-remote-1-cross-root-final.json")).toBe(true); restarted = true; }, observe: () => {}, capture: async () => {}, evidence: async (name: string, value: unknown) => { evidence.set(name, value); }, remoteBootstrap, registerCleanupAssertion: (fn: () => Promise<any>) => cleanup.push(fn) } as any);
|
||||
expect(result.checks.every(check => check.passed)).toBe(true); expect(captures).toEqual([{ runId: "run-1" }, { runId: "run-2" }]); expect(readAfterFinish).toBe(true); expect(cleanup).toHaveLength(2); for (const fn of cleanup) await fn();
|
||||
} finally { await rm(root, { recursive: true, force: true }); }
|
||||
const call = runPiNativeFlow({ page: { goto: async () => {}, reload: async () => {}, getByTestId: () => ({ getByRole: () => ({}) }) }, api, fixtures: { company: { id: "company", issuePrefix: "PI" }, agent: { id: agentId, name: "Pi" }, environment: { id: "daytona-env" } }, execution: { task, environment: { id: "daytona" }, profile: { qualificationCandidate: "pi" } }, nonce: "fixture", workspacePath: root, deadlineAt: Date.now() + 2000,
|
||||
restart: async () => { expect(evidence.has("pi-remote-1-cross-root-final.json")).toBe(true); restarted = true; }, observe: () => {}, capture: async () => {}, evidence: async (name: string, value: unknown) => { if (name.endsWith("incomplete-terminal.json")) { diagnosticAttempts++; if (diagnosticSaveFails) throw new Error("diagnostic storage unavailable"); } evidence.set(name, value); }, remoteBootstrap, registerCleanupAssertion: (fn: () => Promise<any>) => cleanup.push(fn) } as any);
|
||||
if (missingLF) {
|
||||
await expect(call).rejects.toThrow("Public managed-file API contains the exact native-write bytes");
|
||||
expect(evidence.get("pi-agent-files-first-save.json").personal.content).toBe(personal);
|
||||
expect(Buffer.byteLength(personal)).toBe(32);
|
||||
expect(restarted).toBe(false); expect(runs).toHaveLength(1);
|
||||
return;
|
||||
}
|
||||
if (incomplete) {
|
||||
await expect(call).rejects.toBe(terminalError);
|
||||
expect(cleanup).toHaveLength(1);
|
||||
await expect(cleanup[0]!()).rejects.toBe(terminalError);
|
||||
expect(closed).toBe(1); expect(restarted).toBe(false);
|
||||
expect(evidence.has("pi-remote-1-incomplete-terminal.json")).toBe(!diagnosticSaveFails);
|
||||
expect(diagnosticAttempts).toBe(diagnosticSaveFails ? 2 : 1);
|
||||
return;
|
||||
}
|
||||
const result = await call;
|
||||
expect(result.checks.every(check => check.passed)).toBe(true); expect(captures).toEqual(runIds.map(runId => ({ runId }))); expect(readAfterFinish).toBe(true); expect(cleanup).toHaveLength(2); for (const fn of cleanup) await fn();
|
||||
} finally { diagnostic.mockRestore(); await rm(root, { recursive: true, force: true }); }
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { expect, it, vi } from "vitest";
|
||||
import { gradePiRestartCompletion, observePiRestartPending, runPiPendingControllerRestart } from "./pi-native-restart-flow.js";
|
||||
import { validatePrpStructuredRunResult } from "../../packages/paperclip-runner/src/protocol/replay-contract.js";
|
||||
@@ -17,7 +18,7 @@ function fixture(adapter = "acpx-runtime-sidecar") {
|
||||
normalizedSessionId: "session", sourceInstanceId: "runner", sourceSeq, sourceEventId: `runner:run:${sourceSeq}`, payload } },
|
||||
});
|
||||
const state = { issue: { id: "issue", companyId: "company", status: "in_progress" },
|
||||
runs: [{ id: "run", companyId: "company", nativeIssueId: "issue", runtimeMode: "native", status: "running", nativeSessionId: "session", runnerInstanceId: "runner" } as Row],
|
||||
runs: [{ id: "run", companyId: "company", nativeIssueId: "issue", runtimeMode: "native", status: "running", nativeSessionId: "session", runnerInstanceId: "runner", processPid: 200, processGroupId: 200, processStartedAt: "2026-10-02T13:55:27.000Z" } as Row],
|
||||
interactions: [{ id: "interaction", companyId: "company", issueId: "issue", kind: "ask_user_questions", status: "pending", result: null,
|
||||
sourceRunId: "run", continuationPolicy: "none", resolverPolicy: "human_only", idempotencyKey: "paperclip-runner-question:run:request",
|
||||
payload: { runtimeRequestId: "request", questionSet } } as Row] };
|
||||
@@ -50,6 +51,56 @@ it.each(["acpx-runtime", "acpx-runtime-sidecar"])("accepts mixed runner/control-
|
||||
expect(gradePiRestartCompletion(f.state, f.events, pending, "hidden", proof)).toBe(true);
|
||||
});
|
||||
|
||||
function actualPendingPrefix() {
|
||||
return JSON.parse(readFileSync(new URL("./fixtures/pi-native-restart-actual-prefix.json", import.meta.url), "utf8")) as {
|
||||
state: { issue: Row; runs: Row[]; interactions: Row[] }; events: Row[];
|
||||
};
|
||||
}
|
||||
|
||||
it.each(["omitted", "null"])("accepts the actual pre-start submission with %s turn ID", representation => {
|
||||
const f = actualPendingPrefix();
|
||||
if (representation === "null") f.events[0]!.payload.prpEvent.turnId = null;
|
||||
expect(observePiRestartPending(f.state, f.events, "company")).toMatchObject({
|
||||
runId: "run", turnId: "turn", nativeSessionId: "session", sourceInstanceId: "runner", createdSourceSeq: 195,
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
["missing start", (f: ReturnType<typeof actualPendingPrefix>) => { f.events.splice(1, 1); }],
|
||||
["start without assigned turn", (f: ReturnType<typeof actualPendingPrefix>) => { delete f.events[1]!.payload.prpEvent.turnId; }],
|
||||
["start with foreign turn", (f: ReturnType<typeof actualPendingPrefix>) => { f.events[1]!.payload.prpEvent.turnId = "other"; }],
|
||||
["foreign submission turn", (f: ReturnType<typeof actualPendingPrefix>) => { f.events[0]!.payload.prpEvent.turnId = "other"; }],
|
||||
["foreign submission session", (f: ReturnType<typeof actualPendingPrefix>) => { f.events[0]!.payload.prpEvent.normalizedSessionId = "other"; }],
|
||||
["foreign submission producer", (f: ReturnType<typeof actualPendingPrefix>) => {
|
||||
Object.assign(f.events[0]!.payload.prpEvent, { sourceInstanceId: "other", sourceEventId: "other:run:6" });
|
||||
}],
|
||||
["submission after start", (f: ReturnType<typeof actualPendingPrefix>) => {
|
||||
f.events[0]!.seq = 29;
|
||||
Object.assign(f.events[0]!.payload.prpEvent, { sourceSeq: 8, sourceEventId: "runner:run:8" });
|
||||
}],
|
||||
["start after request", (f: ReturnType<typeof actualPendingPrefix>) => {
|
||||
f.events[1]!.seq = 223;
|
||||
Object.assign(f.events[1]!.payload.prpEvent, { sourceSeq: 196, sourceEventId: "runner:run:196" });
|
||||
}],
|
||||
["duplicate submission", (f: ReturnType<typeof actualPendingPrefix>) => {
|
||||
const row = structuredClone(f.events[0]!); row.seq = 26;
|
||||
Object.assign(row.payload.prpEvent, { sourceSeq: 7, sourceEventId: "runner:run:7" });
|
||||
Object.assign(f.events[1]!.payload.prpEvent, { sourceSeq: 8, sourceEventId: "runner:run:8" });
|
||||
f.events.push(row);
|
||||
}],
|
||||
["duplicate start", (f: ReturnType<typeof actualPendingPrefix>) => {
|
||||
const row = structuredClone(f.events[1]!); row.seq = 29;
|
||||
Object.assign(row.payload.prpEvent, { sourceSeq: 8, sourceEventId: "runner:run:8" }); f.events.push(row);
|
||||
}],
|
||||
["later unbound turn event", (f: ReturnType<typeof actualPendingPrefix>) => {
|
||||
const row = structuredClone(f.events[1]!); row.seq = 29; row.eventType = "turn.progress";
|
||||
Object.assign(row.payload.prpEvent, { eventType: "turn.progress", turnId: null, sourceSeq: 8, sourceEventId: "runner:run:8" }); f.events.push(row);
|
||||
}],
|
||||
] as const)("rejects %s around an unbound submission", (_label, mutate) => {
|
||||
const f = actualPendingPrefix(); mutate(f);
|
||||
expect(() => observePiRestartPending(f.state, f.events, "company")).toThrow("Pi native restart");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["no native event", (f: ReturnType<typeof fixture>) => { f.events.length = 0; }],
|
||||
["wrong origin", (f: ReturnType<typeof fixture>) => { f.request.origin.provider = "cursor"; }],
|
||||
@@ -113,16 +164,20 @@ it.each([null, {}, { status: "answered", value: "guess" }, { status: "cancelled"
|
||||
},
|
||||
);
|
||||
|
||||
async function flow(failure?: "replaced" | "wrong-file" | "missing-resolution") {
|
||||
async function flow(failure?: "replaced" | "wrong-file" | "missing-resolution" | "replaced-runner", issueRef = "issue") {
|
||||
const f = fixture(), evidence = new Map<string, any>(), checkpoints: string[] = [];
|
||||
Object.assign(f.state.issue, { identifier: "RUN-1" });
|
||||
let answer = "", restarts = 0, submissions = 0, proof: unknown, resolvePost: ((value: unknown) => void) | undefined;
|
||||
let postPredicate: ((value: any) => boolean) | undefined;
|
||||
const composer: any = { count: () => 1, filter: () => composer, locator: () => composer, first: () => composer,
|
||||
fill: async (value: string) => { expect(restarts).toBe(1); answer = value; } };
|
||||
const button: any = { count: () => 1, filter: () => button, click: async () => {
|
||||
expect(restarts).toBe(1); expect(answer).toMatch(/^PI-RESTART-[a-f0-9]{32}$/); submissions++;
|
||||
const request = { url: () => "http://fixture/api/issues/issue/interactions/interaction/respond", method: () => "POST",
|
||||
const request = { url: () => `http://fixture/api/issues/${issueRef}/interactions/interaction/respond`, method: () => "POST",
|
||||
postDataJSON: () => ({ answers: [{ questionId: "answer", optionIds: [], otherText: answer }] }) };
|
||||
expect(postPredicate!({ ...request, url: () => "http://fixture/api/issues/OTHER-1/interactions/interaction/respond" })).toBe(false);
|
||||
expect(postPredicate!({ ...request, url: () => `http://fixture/api/issues/${issueRef}/interactions/other/respond` })).toBe(false);
|
||||
expect(postPredicate!({ ...request, method: () => "GET" })).toBe(false);
|
||||
expect(postPredicate!(request)).toBe(true); resolvePost!(request); proof = f.finish(answer);
|
||||
if (failure === "missing-resolution") f.events.splice(1, 1);
|
||||
} };
|
||||
@@ -130,23 +185,25 @@ async function flow(failure?: "replaced" | "wrong-file" | "missing-resolution")
|
||||
waitForRequest: (predicate: (value: any) => boolean) => { postPredicate = predicate; return new Promise(resolve => { resolvePost = resolve; }); } };
|
||||
const result = runPiPendingControllerRestart({ page, companyId: "company", deadlineAt: Date.now() + 1000,
|
||||
load: async () => structuredClone(f.state), events: async () => structuredClone(f.events),
|
||||
restart: async () => {
|
||||
restart: async identity => {
|
||||
expect(identity).toEqual({ processPid: 200, processGroupId: 200, processStartedAt: "2026-10-02T13:55:27.000Z" });
|
||||
expect(submissions).toBe(0); expect(f.state.runs[0]!.status).toBe("running"); expect(f.state.interactions[0]!.status).toBe("pending");
|
||||
expect(answer).toBe(""); restarts++; checkpoints.push("restart");
|
||||
if (failure === "replaced") f.state.interactions[0]!.id = "replacement";
|
||||
if (failure === "replaced-runner") f.state.runs[0]!.processPid = f.state.runs[0]!.processGroupId = 201;
|
||||
},
|
||||
settle: async () => structuredClone(f.state), readProof: async () => failure === "wrong-file" ? { status: "answered", value: "guess" } : proof,
|
||||
capture: async id => { checkpoints.push(id); }, evidence: async (name, data) => { evidence.set(name, structuredClone(data)); },
|
||||
});
|
||||
if (failure) await expect(result).rejects.toThrow("Pi native restart");
|
||||
else expect((await result).every(check => check.passed)).toBe(true);
|
||||
expect(restarts).toBe(1); expect(submissions).toBe(failure === "replaced" ? 0 : 1);
|
||||
expect(restarts).toBe(1); expect(submissions).toBe(["replaced", "replaced-runner"].includes(failure ?? "") ? 0 : 1);
|
||||
expect(checkpoints.slice(0, 3)).toEqual(["pi-restart-pending", "restart", "reload"]);
|
||||
expect(evidence.has("pi-native-restart-before.json")).toBe(true);
|
||||
expect(evidence.has("pi-native-restart-after.json")).toBe(true);
|
||||
expect(evidence.has("pi-native-restart-checks.json")).toBe(true);
|
||||
expect(JSON.stringify(evidence.get("pi-native-restart-before.json"))).not.toContain("PI-RESTART-");
|
||||
if (failure !== "replaced") expect(evidence.has("pi-native-restart-final.json")).toBe(true);
|
||||
if (!["replaced", "replaced-runner"].includes(failure ?? "")) expect(evidence.has("pi-native-restart-final.json")).toBe(true);
|
||||
}
|
||||
it("restarts while unanswered, reloads, then submits hidden text through the exact browser route", () => flow());
|
||||
it.each(["replaced", "wrong-file", "missing-resolution"] as const)("retains evidence and fails whole flow on %s", failure => flow(failure));
|
||||
it.each(["issue", "RUN-1"])("restarts while unanswered and submits through the exact %s browser route", issueRef => flow(undefined, issueRef));
|
||||
it.each(["replaced", "wrong-file", "missing-resolution", "replaced-runner"] as const)("retains evidence and fails whole flow on %s", failure => flow(failure));
|
||||
@@ -1,3 +1,4 @@
|
||||
import { parseRestartRunnerIdentity, type RestartRunnerIdentity } from "./process-tree-owner.js";
|
||||
import { createHash, randomBytes } from "node:crypto";
|
||||
import { expect, type Page } from "@playwright/test";
|
||||
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
|
||||
@@ -66,7 +67,19 @@ function inspect(state: State, events: readonly Row[], companyId: string, header
|
||||
&& id(question.id) && question.answerMode === "text" && question.header === header
|
||||
&& card.payload?.runtimeRequestId === request.requestId && digest(card.payload.questionSet) === digest(questionSet)
|
||||
&& card.idempotencyKey === `paperclip-runner-question:${run.id}:${request.requestId}`, "canonical Pi input request identity missing");
|
||||
requireProof(rows.filter(x => x.event.turnId != null || x.event.eventType.startsWith("turn.") || x.event.eventType.startsWith("runtime_request.")).every(({ event }) => event.turnId === request.turnId
|
||||
const submitted = rows.filter(x => x.event.eventType === "turn.submitted");
|
||||
const unboundSubmission = submitted.find(x => x.event.turnId == null);
|
||||
if (unboundSubmission) {
|
||||
// The local runner records submission before ACP assigns the provider turn ID.
|
||||
// Only that unique, ordered pre-start receipt may omit the turn identity.
|
||||
const started = rows.filter(x => x.event.eventType === "turn.started");
|
||||
requireProof(submitted.length === 1 && started.length === 1
|
||||
&& started[0]!.event.turnId === request.turnId
|
||||
&& unboundSubmission.row.seq < started[0]!.row.seq && started[0]!.row.seq < opening.row.seq
|
||||
&& unboundSubmission.event.sourceSeq < started[0]!.event.sourceSeq && started[0]!.event.sourceSeq < opening.event.sourceSeq,
|
||||
"unbound submission is not the unique pre-start receipt");
|
||||
}
|
||||
requireProof(rows.filter(x => x.event.turnId != null || x.event.eventType.startsWith("turn.") || x.event.eventType.startsWith("runtime_request.")).every(({ row, event }) => (event.turnId === request.turnId || row === unboundSubmission?.row)
|
||||
&& event.normalizedSessionId === run.nativeSessionId
|
||||
&& event.sourceInstanceId === (event.sourceKind === "runner" ? run.runnerInstanceId : `${run.runnerInstanceId}:control`)), "turn, native session or producer was replaced");
|
||||
requireProof(rows.filter(x => x.event.sourceKind === "control_plane").every(x => x.event.turnId === request.turnId), "control-plane result belongs to another turn");
|
||||
@@ -120,7 +133,7 @@ export function gradePiRestartCompletion(state: State, events: readonly Row[], p
|
||||
|
||||
export async function runPiPendingControllerRestart(input: {
|
||||
page: Page; companyId: string; deadlineAt: number; load(): Promise<State>; events(runId: string): Promise<Row[]>;
|
||||
restart(): Promise<void>; settle(): Promise<State>; readProof(): Promise<unknown>;
|
||||
preserveLocalRunner?: boolean; restart(preserveRunner?: RestartRunnerIdentity): Promise<void>; settle(): Promise<State>; readProof(): Promise<unknown>;
|
||||
capture(id: string, label: string, file: string): Promise<void>; evidence(name: string, data: unknown): Promise<void>;
|
||||
}): Promise<Check[]> {
|
||||
const checks: Check[] = [], answer = `PI-RESTART-${randomBytes(16).toString("hex")}`;
|
||||
@@ -138,15 +151,19 @@ export async function runPiPendingControllerRestart(input: {
|
||||
const composer = () => input.page.getByTestId("question-text-answer-composer").filter({ visible: true });
|
||||
await expect(composer()).toHaveCount(1);
|
||||
await input.capture("pi-restart-pending", "Pi native question before controller restart", "pi-restart-pending.png");
|
||||
await input.restart();
|
||||
const runnerIdentity = input.preserveLocalRunner === false ? undefined : parseRestartRunnerIdentity(before.state.runs[0]);
|
||||
await input.restart(runnerIdentity);
|
||||
await input.page.reload();
|
||||
const after = await snapshot("pi-native-restart-after.json"), resumed = observePiRestartPending(after.state, after.events, input.companyId);
|
||||
check("restart-same-pending-native-request", digest(resumed) === digest(pending), "Controller restart retained the same unanswered request, run, turn, native session and producer");
|
||||
if (runnerIdentity) check("restart-same-live-runner-identity", digest(parseRestartRunnerIdentity(after.state.runs[0])) === digest(runnerIdentity),
|
||||
"The original durable runner PID, process group and start identity survived the controller restart");
|
||||
await expect(composer()).toHaveCount(1);
|
||||
await input.capture("pi-restart-reconnected", "Same Pi question after controller restart", "pi-restart-reconnected.png");
|
||||
await composer().locator('[contenteditable="true"],textarea').first().fill(answer);
|
||||
const route = `/api/issues/${pending.issueId}/interactions/${pending.interactionId}/respond`;
|
||||
const submitted = input.page.waitForRequest(request => new URL(request.url()).pathname === route && request.method() === "POST",
|
||||
const issueRefs = [pending.issueId, after.state.issue.identifier].filter(id);
|
||||
const routes = new Set(issueRefs.map(issueRef => `/api/issues/${encodeURIComponent(issueRef)}/interactions/${pending.interactionId}/respond`));
|
||||
const submitted = input.page.waitForRequest(request => routes.has(new URL(request.url()).pathname) && request.method() === "POST",
|
||||
{ timeout: Math.max(1, input.deadlineAt - Date.now()) });
|
||||
const button = input.page.getByRole("button", { name: after.state.interactions[0]!.payload.questionSet.submitLabel ?? "Submit answers", exact: true }).filter({ visible: true });
|
||||
await expect(button).toHaveCount(1); await button.click();
|
||||
|
||||
@@ -72,6 +72,45 @@ def digest(data):
|
||||
return hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def checked_runner_executable(path):
|
||||
# Production stages a verified preinstalled runner with ln -sfn. Admit
|
||||
# that named link only while its identity and resolved regular file remain
|
||||
# stable; inspect still requires both the pinned hash and /proc/exe inode.
|
||||
before = os.lstat(path)
|
||||
require(stat.S_ISREG(before.st_mode) or stat.S_ISLNK(before.st_mode), 'runner_file_shape')
|
||||
resolved = os.path.realpath(path)
|
||||
content, inode = checked_file(resolved, 256 * 1024 * 1024)
|
||||
after = os.lstat(path)
|
||||
identity = lambda s: (s.st_dev, s.st_ino, s.st_mode, s.st_mtime_ns, s.st_ctime_ns)
|
||||
require(identity(before) == identity(after) and os.path.realpath(path) == resolved, 'runner_link_changed')
|
||||
return content, inode
|
||||
|
||||
|
||||
def parse_closure_metadata(content, expected_pin):
|
||||
# Match production parseNativeAcpxDistributionEntries: the profile pins
|
||||
# canonical entries, not the formatting or outer JSON file bytes.
|
||||
manifest = json.loads(content)
|
||||
entries = manifest.get('entries') if isinstance(manifest, dict) else None
|
||||
require(isinstance(entries, list) and 0 < len(entries) <= 30000, 'closure_inventory')
|
||||
normalized, previous, total = [], '', 0
|
||||
for entry in entries:
|
||||
require(isinstance(entry, dict) and set(entry) == {'path', 'sha256', 'size', 'executable'}, 'closure_entry_shape')
|
||||
path = entry['path']
|
||||
require(isinstance(path, str) and 0 < len(path) <= 4096 and not os.path.isabs(path)
|
||||
and not re.search(r'[\x00-\x1f\x7f\\]', path)
|
||||
and all(part not in ('', '.', '..') for part in path.split('/'))
|
||||
and path > previous and path != 'manifest.json' and not path.startswith('.paperclip-'), 'closure_entry_path')
|
||||
require(isinstance(entry['sha256'], str) and re.fullmatch(r'[a-f0-9]{64}', entry['sha256'])
|
||||
and type(entry['size']) is int and 0 <= entry['size'] <= 384 * 1024 * 1024
|
||||
and type(entry['executable']) is bool, 'closure_entry_metadata')
|
||||
total += entry['size']; require(total <= 1024 * 1024 * 1024, 'closure_tree_bound')
|
||||
previous = path
|
||||
normalized.append({key: entry[key] for key in ('path', 'sha256', 'size', 'executable')})
|
||||
canonical = json.dumps(normalized, separators=(',', ':'), ensure_ascii=False).encode('utf8')
|
||||
require(re.fullmatch(r'[a-f0-9]{64}', expected_pin) and digest(canonical) == expected_pin, 'closure_pin')
|
||||
return normalized
|
||||
|
||||
|
||||
def inspect(config):
|
||||
require(set(config) == {'root', 'binding', 'runtimeEnvironmentLeaseId', 'runnerdSha256', 'closureSha256'}, 'config_keys')
|
||||
binding, expected = config['binding'], config['root']
|
||||
@@ -86,14 +125,12 @@ def inspect(config):
|
||||
and flag(root_args, '--environment-lease-id') == config['runtimeEnvironmentLeaseId']
|
||||
and flag(root_args, '--lifecycle-mode') == 'per_turn', 'root_binding')
|
||||
require(re.fullmatch(re.escape(runtime_root) + r'/sessions/[a-f0-9]{64}/runner', flag(root_args, '--state-dir')), 'root_session')
|
||||
runner_bytes, runner_inode = checked_file(runner, 256 * 1024 * 1024)
|
||||
runner_bytes, runner_inode = checked_runner_executable(runner)
|
||||
executable = os.stat(f'/proc/{root["pid"]}/exe')
|
||||
require((executable.st_dev, executable.st_ino) == runner_inode and 'sha256:' + digest(runner_bytes) == config['runnerdSha256'], 'runner_executable')
|
||||
closure_bytes, _ = checked_file(PACK + '/provider-assets/pi/linux-x64/native-closure.json', 4 * 1024 * 1024)
|
||||
require(digest(closure_bytes) == config['closureSha256'], 'closure_pin')
|
||||
manifest = json.loads(closure_bytes)
|
||||
entries = {e['path']: e for e in manifest['entries']}
|
||||
require(len(entries) == len(manifest['entries']), 'closure_duplicate')
|
||||
admitted_entries = parse_closure_metadata(closure_bytes, config['closureSha256'])
|
||||
entries = {e['path']: e for e in admitted_entries}
|
||||
for name in (NODE, ENTRY, EXTENSION, 'pi-entry.cjs', 'node_modules/pi-acp/dist/index.js', 'node_modules/pi-acp/dist/paperclip-runtime.js'):
|
||||
require(name in entries and re.fullmatch(r'[a-f0-9]{64}', entries[name]['sha256']), 'closure_entry')
|
||||
pids = [int(p) for p in os.listdir('/proc') if p.isdigit() and int(p) > 1]
|
||||
@@ -128,7 +165,15 @@ def inspect(config):
|
||||
continue
|
||||
distribution = parent_args[3][:-len('/pi-entry.cjs')]
|
||||
require(re.fullmatch(r'/tmp/paperclip-acpx-native-[A-Za-z0-9_-]+/distribution', distribution), 'snapshot_path')
|
||||
require(parent_args == [distribution + '/' + NODE, '--require', distribution + '/' + GUARD, distribution + '/pi-entry.cjs'], 'wrapper_parent')
|
||||
require(parent_args[1:] == ['--require', distribution + '/' + GUARD, distribution + '/pi-entry.cjs'], 'wrapper_parent')
|
||||
descriptor = None
|
||||
if parent_args[0] != distribution + '/' + NODE:
|
||||
# Production nativeBootstrap executes the held Node through child
|
||||
# FD 3 (unfenced) or 7 (guardian/credential fences). The child's
|
||||
# cmdline retains /proc/self/fd/N; it does not name the source path.
|
||||
match = re.fullmatch(r'/proc/self/fd/(3|7)', parent_args[0])
|
||||
require(match is not None, 'wrapper_parent')
|
||||
descriptor = f'/proc/{parent}/fd/{match.group(1)}'
|
||||
# Pi sets process.title and overwrites Linux argv. The exact pinned
|
||||
# parent's launch code attests the entrypoint; title alone never selects.
|
||||
require(argv(pid) == ['pi'], 'pi_process_title')
|
||||
@@ -145,6 +190,9 @@ def inspect(config):
|
||||
require((exe.st_dev, exe.st_ino) == identities[NODE], 'pi_executable')
|
||||
parent_exe = os.stat(f'/proc/{parent}/exe')
|
||||
require((parent_exe.st_dev, parent_exe.st_ino) == identities[NODE], 'wrapper_executable')
|
||||
if descriptor is not None:
|
||||
held_exe = os.stat(descriptor)
|
||||
require((held_exe.st_dev, held_exe.st_ino) == identities[NODE], 'wrapper_descriptor')
|
||||
for identity in chain:
|
||||
require(proc(identity['pid'], boot) == identity, 'ancestry_changed')
|
||||
matches.append({'target': table[pid], 'ancestry': chain, 'nodeSha256': 'sha256:' + entries[NODE]['sha256'],
|
||||
|
||||
@@ -32,18 +32,73 @@ def check_cancelled():
|
||||
NAMES = [fault.NODE, fault.ENTRY, fault.EXTENSION, 'pi-entry.cjs', 'node_modules/pi-acp/dist/index.js', 'node_modules/pi-acp/dist/paperclip-runtime.js']
|
||||
|
||||
|
||||
class ClosureMetadataTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.entries = [
|
||||
{'path': name, 'sha256': fault.digest(name.encode()), 'size': len(name), 'executable': name == fault.NODE}
|
||||
for name in sorted(NAMES)
|
||||
]
|
||||
self.canonical = json.dumps(self.entries, separators=(',', ':'), ensure_ascii=False).encode()
|
||||
self.pin = fault.digest(self.canonical)
|
||||
self.manifest = {'entries': self.entries}
|
||||
|
||||
def test_canonical_entries_pin_admits_metadata(self):
|
||||
raw = json.dumps(self.manifest, indent=2).encode()
|
||||
self.assertNotEqual(fault.digest(raw), self.pin)
|
||||
self.assertEqual(fault.parse_closure_metadata(raw, self.pin), self.entries)
|
||||
|
||||
def test_metadata_formatting_and_property_order_preserve_pin(self):
|
||||
reordered = {'entries': [dict(reversed(list(entry.items()))) for entry in self.entries]}
|
||||
for manifest in [self.manifest, reordered]:
|
||||
for indent in [None, 2, 4]:
|
||||
with self.subTest(indent=indent):
|
||||
self.assertEqual(fault.parse_closure_metadata(json.dumps(manifest, indent=indent).encode(), self.pin), self.entries)
|
||||
|
||||
def test_raw_metadata_hash_is_not_a_closure_pin(self):
|
||||
raw = json.dumps(self.manifest, indent=2).encode()
|
||||
with self.assertRaisesRegex(RuntimeError, 'closure_pin'):
|
||||
fault.parse_closure_metadata(raw, fault.digest(raw))
|
||||
|
||||
def test_changed_entry_is_rejected_by_the_pinned_digest(self):
|
||||
changed = json.loads(json.dumps(self.manifest))
|
||||
changed['entries'][0]['sha256'] = '0' * 64
|
||||
with self.assertRaisesRegex(RuntimeError, 'closure_pin'):
|
||||
fault.parse_closure_metadata(json.dumps(changed).encode(), self.pin)
|
||||
|
||||
def test_unsafe_duplicate_unsorted_and_invalid_metadata_fail_closed(self):
|
||||
mutations = [
|
||||
lambda m: m['entries'][0].update(path='../escape'),
|
||||
lambda m: m['entries'][0].update(path='.paperclip-native-entry.cjs'),
|
||||
lambda m: m['entries'].insert(1, m['entries'][0].copy()),
|
||||
lambda m: m['entries'].reverse(),
|
||||
lambda m: m['entries'][0].update(size=True),
|
||||
lambda m: m['entries'][0].update(executable=1),
|
||||
lambda m: m['entries'][0].update(extra='foreign'),
|
||||
]
|
||||
for index, mutate in enumerate(mutations):
|
||||
changed = json.loads(json.dumps(self.manifest))
|
||||
mutate(changed)
|
||||
with self.subTest(index=index), self.assertRaises(RuntimeError):
|
||||
fault.parse_closure_metadata(json.dumps(changed).encode(), self.pin)
|
||||
for invalid in [None, [], {}, {'entries': []}]:
|
||||
with self.subTest(invalid=invalid), self.assertRaises(RuntimeError):
|
||||
fault.parse_closure_metadata(json.dumps(invalid).encode(), self.pin)
|
||||
|
||||
|
||||
class IdentityTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.files = {DIST + '/' + n: (n.encode(), (1, 100 + i)) for i, n in enumerate(NAMES)}
|
||||
self.files[RUNTIME + '/bin/paperclip-runnerd'] = (b'runner', (1, 90))
|
||||
closure = json.dumps({'entries': [{'path': n, 'sha256': fault.digest(n.encode()), 'size': len(n)} for n in NAMES]}).encode()
|
||||
entries = [{'path': n, 'sha256': fault.digest(n.encode()), 'size': len(n), 'executable': n == fault.NODE} for n in sorted(NAMES)]
|
||||
closure = json.dumps({'entries': entries}, indent=2).encode()
|
||||
self.files[fault.PACK + '/provider-assets/pi/linux-x64/native-closure.json'] = (closure, (1, 80))
|
||||
self.config = {'root': ROOT, 'binding': {'remoteCwd': '/workspace', 'runId': 'run'}, 'runtimeEnvironmentLeaseId': 'workspace-id',
|
||||
'runnerdSha256': 'sha256:' + fault.digest(b'runner'), 'closureSha256': fault.digest(closure)}
|
||||
'runnerdSha256': 'sha256:' + fault.digest(b'runner'), 'closureSha256': fault.digest(json.dumps(entries, separators=(',', ':'), ensure_ascii=False).encode())}
|
||||
self.table = {21: ROOT, 22: PARENT, 23: TARGET}
|
||||
self.args = {21: [RUNTIME + '/bin/paperclip-runnerd', '--run-id', 'run', '--environment-lease-id', 'workspace-id', '--lifecycle-mode', 'per_turn', '--state-dir', RUNTIME + '/sessions/' + 'a' * 64 + '/runner'],
|
||||
22: [DIST + '/' + fault.NODE, '--require', DIST + '/' + fault.GUARD, DIST + '/pi-entry.cjs'], 23: ['pi']}
|
||||
self.patches = [patch.object(fault, 'proc', side_effect=lambda pid, boot: self.table[pid]), patch.object(fault, 'argv', side_effect=lambda pid: self.args[pid]),
|
||||
patch.object(fault, 'checked_runner_executable', side_effect=lambda p: self.files[p]),
|
||||
patch.object(fault, 'checked_file', side_effect=lambda p, *a: self.files[p]), patch.object(Path, 'read_text', return_value=BOOT),
|
||||
patch.object(os, 'listdir', return_value=['21', '22', '23']), patch.object(os, 'getpgid', return_value=21),
|
||||
patch.object(os, 'lstat', return_value=SimpleNamespace(st_mode=0o40500)), patch.object(os.path, 'realpath', side_effect=lambda p: p),
|
||||
@@ -57,6 +112,28 @@ class IdentityTests(unittest.TestCase):
|
||||
self.assertFalse(receipt['originalChildArgvAvailable'])
|
||||
self.assertEqual(receipt['ancestry'], [TARGET, PARENT, ROOT])
|
||||
|
||||
def test_descriptor_launch_attests_the_same_pinned_wrapper(self):
|
||||
for descriptor in [3, 7]:
|
||||
self.args[22][0] = f'/proc/self/fd/{descriptor}'
|
||||
with self.subTest(descriptor=descriptor):
|
||||
self.assertEqual(fault.inspect(self.config)['target'], TARGET)
|
||||
|
||||
def test_descriptor_launch_rejects_foreign_missing_or_unbound_fd(self):
|
||||
for name in ['/proc/self/fd/8', '/proc/99/fd/7', '/foreign/node']:
|
||||
self.args[22][0] = name
|
||||
with self.subTest(name=name), self.assertRaisesRegex(RuntimeError, 'wrapper_parent'):
|
||||
fault.inspect(self.config)
|
||||
self.args[22][0] = '/proc/self/fd/7'
|
||||
stat_original = os.stat
|
||||
for problem in ['foreign', 'missing']:
|
||||
def descriptor_stat(path):
|
||||
if path == '/proc/22/fd/7':
|
||||
if problem == 'missing': raise FileNotFoundError(path)
|
||||
return SimpleNamespace(st_dev=1, st_ino=999)
|
||||
return stat_original(path)
|
||||
with self.subTest(problem=problem), patch.object(os, 'stat', side_effect=descriptor_stat), self.assertRaises((RuntimeError, FileNotFoundError)):
|
||||
fault.inspect(self.config)
|
||||
|
||||
def test_wrong_metadata_never_selects(self):
|
||||
for field, value in [('closureSha256', '0' * 64), ('runtimeEnvironmentLeaseId', 'foreign'), ('runnerdSha256', 'sha256:' + '0' * 64)]:
|
||||
with self.subTest(field=field), self.assertRaises(RuntimeError): fault.inspect({**self.config, field: value})
|
||||
@@ -82,6 +159,43 @@ class IdentityTests(unittest.TestCase):
|
||||
with patch.object(os, 'listdir', return_value=['21', '22', '23', '24']), self.assertRaisesRegex(RuntimeError, 'unique_pi_child'): fault.inspect(self.config)
|
||||
|
||||
|
||||
class RunnerExecutableTests(unittest.TestCase):
|
||||
def test_regular_and_preinstalled_link_resolve_to_the_same_inode(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
executable = Path(tmp).resolve() / 'installed-runner'
|
||||
executable.write_bytes(b'pinned runner')
|
||||
link = executable.with_name('runtime-runner')
|
||||
link.symlink_to(executable)
|
||||
content, inode = fault.checked_runner_executable(str(link))
|
||||
self.assertEqual(content, b'pinned runner')
|
||||
self.assertEqual((content, inode), fault.checked_runner_executable(str(executable)))
|
||||
|
||||
def test_retargeted_link_is_rejected_after_read(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
executable = Path(tmp).resolve() / 'installed-runner'
|
||||
executable.write_bytes(b'pinned runner')
|
||||
foreign = executable.with_name('foreign-runner')
|
||||
foreign.write_bytes(b'foreign')
|
||||
link = executable.with_name('runtime-runner')
|
||||
link.symlink_to(executable)
|
||||
original = fault.checked_file
|
||||
def replace_during_read(*args):
|
||||
result = original(*args)
|
||||
link.unlink()
|
||||
link.symlink_to(foreign)
|
||||
return result
|
||||
with patch.object(fault, 'checked_file', side_effect=replace_during_read), self.assertRaisesRegex(RuntimeError, 'runner_link_changed'):
|
||||
fault.checked_runner_executable(str(link))
|
||||
|
||||
def test_missing_link_and_directory_never_admit_an_executable(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp).resolve()
|
||||
link = root / 'runtime-runner'
|
||||
link.symlink_to(root / 'missing')
|
||||
with self.assertRaises(FileNotFoundError): fault.checked_runner_executable(str(link))
|
||||
with self.assertRaisesRegex(RuntimeError, 'runner_file_shape'): fault.checked_runner_executable(str(root))
|
||||
|
||||
|
||||
class PidfdTests(unittest.TestCase):
|
||||
def test_changed_or_retired_pidfd_never_signals_and_always_closes(self):
|
||||
before = {'target': TARGET}
|
||||
@@ -103,6 +217,18 @@ class PidfdTests(unittest.TestCase):
|
||||
|
||||
@unittest.skipUnless(sys.platform == 'linux' and hasattr(os, 'pidfd_open'), 'Hosted native Linux calibration required')
|
||||
def test_real_title_overwrite_then_exact_child_pidfd(self):
|
||||
self.calibrate_real_child()
|
||||
|
||||
@unittest.skipUnless(sys.platform == 'linux' and hasattr(os, 'pidfd_open'), 'Hosted native Linux descriptor calibration required')
|
||||
def test_real_descriptor_launch_then_exact_child_pidfd(self):
|
||||
for descriptor in [3, 7]:
|
||||
with self.subTest(descriptor=descriptor): self.calibrate_real_child(descriptor)
|
||||
|
||||
@unittest.skipUnless(sys.platform == 'linux' and hasattr(os, 'pidfd_open'), 'Hosted native Linux preinstalled-link calibration required')
|
||||
def test_real_preinstalled_runner_link_then_exact_child_pidfd(self):
|
||||
self.calibrate_real_child(7, runner_link=True)
|
||||
|
||||
def calibrate_real_child(self, descriptor=None, runner_link=False):
|
||||
node = shutil.which('node'); self.assertIsNotNone(node)
|
||||
base = Path(tempfile.mkdtemp(prefix='pi-fault-calibration-', dir='/tmp'))
|
||||
snapshot = Path(tempfile.mkdtemp(prefix='paperclip-acpx-native-', dir='/tmp'))
|
||||
@@ -115,19 +241,34 @@ class PidfdTests(unittest.TestCase):
|
||||
for name in NAMES + [fault.GUARD]:
|
||||
p = distribution / name; p.parent.mkdir(parents=True, exist_ok=True); p.write_text('// fixture\n')
|
||||
shutil.copyfile(node, distribution / fault.NODE); (distribution / fault.NODE).chmod(0o500)
|
||||
shutil.copyfile(node, runtime / 'bin/paperclip-runnerd'); (runtime / 'bin/paperclip-runnerd').chmod(0o500)
|
||||
runner = runtime / 'bin/paperclip-runnerd'
|
||||
if runner_link:
|
||||
installed = base / 'preinstalled-runner'
|
||||
shutil.copyfile(node, installed); installed.chmod(0o500)
|
||||
runner.symlink_to(installed)
|
||||
else:
|
||||
shutil.copyfile(node, runner); runner.chmod(0o500)
|
||||
ready = workspace / 'child.json'; exited = workspace / 'exit.json'
|
||||
(distribution / fault.ENTRY).write_text('process.title="pi"; require("node:fs").writeFileSync(' + json.dumps(str(ready)) + ',JSON.stringify({pid:process.pid}));setInterval(()=>{},1000);')
|
||||
(distribution / 'pi-entry.cjs').write_text('const p=require("node:child_process").spawn(process.execPath,["--require",' + json.dumps(str(distribution / fault.GUARD)) + ',' + json.dumps(str(distribution / fault.ENTRY)) + '],{stdio:"ignore"});p.on("exit",(code,signal)=>require("node:fs").writeFileSync(' + json.dumps(str(exited)) + ',JSON.stringify({code,signal})));process.on("SIGTERM",()=>{if(p.exitCode!==null||p.signalCode!==null)process.exit(0);p.once("exit",()=>process.exit(0));p.kill("SIGTERM")});setInterval(()=>{},1000);')
|
||||
entries = []
|
||||
for name in NAMES:
|
||||
p = distribution / name; content = p.read_bytes(); entries.append({'path': name, 'sha256': fault.digest(content), 'size': len(content)})
|
||||
for name in sorted(NAMES):
|
||||
p = distribution / name; content = p.read_bytes(); entries.append({'path': name, 'sha256': fault.digest(content), 'size': len(content), 'executable': name == fault.NODE})
|
||||
closure = json.dumps({'entries': entries}).encode(); (pack / 'provider-assets/pi/linux-x64/native-closure.json').write_bytes(closure)
|
||||
for directory, dirs, files in os.walk(snapshot):
|
||||
for f in files:
|
||||
p = Path(directory) / f; p.chmod(0o500 if p == distribution / fault.NODE else 0o400)
|
||||
Path(directory).chmod(0o500)
|
||||
script = base / 'root.cjs'; script.write_text('const p=require("node:child_process").spawn(' + json.dumps(str(distribution / fault.NODE)) + ',["--require",' + json.dumps(str(distribution / fault.GUARD)) + ',' + json.dumps(str(distribution / 'pi-entry.cjs')) + '],{stdio:"ignore"});process.on("SIGTERM",()=>{if(p.exitCode!==null||p.signalCode!==null)process.exit(0);p.once("exit",()=>process.exit(0));p.kill("SIGTERM")});setInterval(()=>{},1000);')
|
||||
launch = json.dumps(str(distribution / fault.NODE))
|
||||
setup, stdio, release = '', '"ignore"', ''
|
||||
if descriptor is not None:
|
||||
# Production nativeBootstrap keeps its executable descriptor
|
||||
# at child FD 3, or FD 7 when lifetime/credential fences exist.
|
||||
setup = 'const fd=require("node:fs").openSync(' + launch + ',"r");'
|
||||
launch = json.dumps(f'/proc/self/fd/{descriptor}')
|
||||
stdio = '[' + ','.join(['"ignore"'] * descriptor + ['fd']) + ']'
|
||||
release = 'require("node:fs").closeSync(fd);'
|
||||
script = base / 'root.cjs'; script.write_text(setup + 'const p=require("node:child_process").spawn(' + launch + ',["--require",' + json.dumps(str(distribution / fault.GUARD)) + ',' + json.dumps(str(distribution / 'pi-entry.cjs')) + '],{stdio:' + stdio + '});' + release + 'process.on("SIGTERM",()=>{if(p.exitCode!==null||p.signalCode!==null)process.exit(0);p.once("exit",()=>process.exit(0));p.kill("SIGTERM")});setInterval(()=>{},1000);')
|
||||
check_cancelled()
|
||||
root_child = subprocess.Popen([str(runtime / 'bin/paperclip-runnerd'), str(script), '--run-id', 'fixture', '--environment-lease-id', 'workspace-id', '--lifecycle-mode', 'per_turn', '--state-dir', str(runtime / 'sessions' / ('a' * 64) / 'runner')], cwd=workspace, env={'PATH': '/usr/bin:/bin'}, start_new_session=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try: owned_fds.append(os.pidfd_open(root_child.pid))
|
||||
@@ -143,7 +284,7 @@ class PidfdTests(unittest.TestCase):
|
||||
self.assertTrue(ready.exists()); target = json.loads(ready.read_text())['pid']
|
||||
self.assertEqual(fault.argv(target), ['pi'], 'real Node process.title must overwrite argv')
|
||||
boot = Path('/proc/sys/kernel/random/boot_id').read_text().strip()
|
||||
config = {'root': fault.proc(root_child.pid, boot), 'binding': {'remoteCwd': str(workspace), 'runId': 'fixture'}, 'runtimeEnvironmentLeaseId': 'workspace-id', 'runnerdSha256': 'sha256:' + fault.digest((runtime / 'bin/paperclip-runnerd').read_bytes()), 'closureSha256': fault.digest(closure)}
|
||||
config = {'root': fault.proc(root_child.pid, boot), 'binding': {'remoteCwd': str(workspace), 'runId': 'fixture'}, 'runtimeEnvironmentLeaseId': 'workspace-id', 'runnerdSha256': 'sha256:' + fault.digest((runtime / 'bin/paperclip-runnerd').read_bytes()), 'closureSha256': fault.digest(json.dumps(entries, separators=(',', ':'), ensure_ascii=False).encode())}
|
||||
with patch.object(fault, 'PACK', str(pack)):
|
||||
admitted = fault.inspect(config)
|
||||
self.assertEqual(admitted['target']['pid'], target)
|
||||
|
||||
@@ -15,8 +15,8 @@ it("calibrates the exact Pi fault helper with metadata negatives and native Linu
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.signal).toBeNull();
|
||||
expect(result.status, result.stderr).toBe(0);
|
||||
expect(result.stderr).toContain("Ran 9 tests");
|
||||
expect(result.stderr).toContain("Ran 21 tests");
|
||||
if (process.platform === "linux") expect(result.stderr).not.toContain("skipped");
|
||||
else expect(result.stderr).toContain("skipped=1");
|
||||
else expect(result.stderr).toContain("skipped=3");
|
||||
console.info(result.stderr.trim());
|
||||
}, 30_000);
|
||||
@@ -45,7 +45,7 @@ const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
||||
|
||||
export default defineConfig({
|
||||
testDir: ".",
|
||||
testMatch: "runner.spec.ts",
|
||||
testMatch: process.env.PAPERCLIP_RUNNER_E2E_INSTALLED_STARTUP_ONLY === "1" ? "installed-startup.spec.ts" : "runner.spec.ts",
|
||||
timeout: Number(process.env.PAPERCLIP_RUNNER_E2E_TEST_TIMEOUT_MS ?? 600_000),
|
||||
expect: { timeout: 30_000 },
|
||||
fullyParallel: false,
|
||||
|
||||
@@ -7,9 +7,8 @@ export const PENDING_PROFILE_PREREQUISITES = {
|
||||
"legacy-grok": "Grok identity/auth/skills/session/billing qualification is pending",
|
||||
} as const;
|
||||
|
||||
/** Pi is intentionally absent: no qualified model source exists for a valid profile. */
|
||||
/** Qualification fixtures use explicit models; there is no runtime model roster. */
|
||||
export const UNQUALIFIED_PROFILE_GAPS = {
|
||||
pi: "Pi has no qualified model source, so no valid Product E2E profile is registered.",
|
||||
} as const;
|
||||
|
||||
/** Reject missing remote observer authority before creating a company or run. */
|
||||
@@ -18,7 +17,7 @@ export function assertRemoteNativeEvidencePrerequisites(
|
||||
environment: NodeJS.ProcessEnv,
|
||||
): void {
|
||||
if (!executions.some(execution => execution.environment.id === "daytona"
|
||||
&& ["cursor_native", "native_active_stop", "native_provider_loss"].includes(execution.task.flow))) return;
|
||||
&& ["cursor_native", "pi_native", "pi_controls", "copilot_protection", "native_active_stop", "native_provider_loss"].includes(execution.task.flow))) return;
|
||||
const names = ["PAPERCLIP_E2E_DAYTONA_NODE_SHA256", "PAPERCLIP_E2E_DAYTONA_RUNNERD_SHA256"];
|
||||
const invalid = names.filter(name => !/^sha256:[a-f0-9]{64}$/u.test(environment[name] ?? ""));
|
||||
if (invalid.length) throw new Error(`Native Daytona evidence requires exact image executable digests: ${invalid.join(", ")}`);
|
||||
|
||||
@@ -27,6 +27,38 @@ it.skipIf(process.platform === "win32")("revalidates PID/start before signaling
|
||||
} finally { f.owner.stopObserving(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("admits replacement group members only through a still-owned ancestor", async () => {
|
||||
const f = fixture([row(100, process.pid, 100), row(200, 100, 200)]);
|
||||
try {
|
||||
await f.owner.observe();
|
||||
// The short-lived group leader exited between polls. Its new member is
|
||||
// independently owned through the original, still-live controller.
|
||||
f.replace([row(100, process.pid, 100), row(300, 100, 200)]);
|
||||
await f.owner.signal("SIGKILL");
|
||||
expect(f.signals).toEqual([[200, "SIGKILL"], [100, "SIGKILL"]]);
|
||||
} finally { f.owner.stopObserving(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("refuses replacement groups containing any unowned live member", async () => {
|
||||
const f = fixture([row(100, process.pid, 100), row(200, 100, 200)]);
|
||||
try {
|
||||
await f.owner.observe();
|
||||
f.replace([row(100, process.pid, 100), row(300, 100, 200), row(400, 1, 200)]);
|
||||
await expect(f.owner.signal("SIGKILL")).rejects.toThrow("identity became uncertain");
|
||||
expect(f.signals).toEqual([]);
|
||||
} finally { f.owner.stopObserving(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("does not use a recycled controller as a replacement member's ownership anchor", async () => {
|
||||
const f = fixture([row(100, process.pid, 100), row(200, 100, 200)]);
|
||||
try {
|
||||
await f.owner.observe();
|
||||
f.replace([row(100, process.pid, 100, "recycled-controller"), row(300, 100, 200)]);
|
||||
await expect(f.owner.signal("SIGKILL")).rejects.toThrow("identity became uncertain");
|
||||
expect(f.signals).toEqual([]);
|
||||
} finally { f.owner.stopObserving(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("retains observed descendants when the root exits and excludes the caller group", async () => {
|
||||
const f = fixture([row(100, process.pid, 100), row(200, 100, 200), row(300, 100, 10), row(999, process.pid, 999)]);
|
||||
try {
|
||||
|
||||
@@ -5,6 +5,29 @@ import {
|
||||
type ObservedProcessGroup, type ProcessObservation,
|
||||
} from "./process-tree.js";
|
||||
|
||||
/** Supplied by the company-scoped run API, never by process-name discovery. */
|
||||
export interface RestartRunnerIdentity {
|
||||
processPid: number;
|
||||
processGroupId: number;
|
||||
processStartedAt: string;
|
||||
}
|
||||
export function parseRestartRunnerIdentity(value: unknown): RestartRunnerIdentity {
|
||||
const v = value as RestartRunnerIdentity | null;
|
||||
if (!v || !Number.isSafeInteger(v.processPid) || v.processPid <= 1
|
||||
|| v.processGroupId !== v.processPid || typeof v.processStartedAt !== "string"
|
||||
|| !Number.isFinite(Date.parse(v.processStartedAt))) throw new Error("Invalid restart runner identity");
|
||||
return { processPid: v.processPid, processGroupId: v.processGroupId, processStartedAt: v.processStartedAt };
|
||||
}
|
||||
|
||||
/** Linux run receipts retain milliseconds; ps lstart retains whole seconds. */
|
||||
export function restartProcessStartMatches(observed: string, expected: string, platform = process.platform): boolean {
|
||||
const actualTime = Date.parse(observed), expectedTime = Date.parse(expected);
|
||||
if (!Number.isFinite(actualTime) || !Number.isFinite(expectedTime)) return false;
|
||||
return platform === "linux"
|
||||
? Math.floor(actualTime / 1000) === Math.floor(expectedTime / 1000)
|
||||
: actualTime === expectedTime;
|
||||
}
|
||||
|
||||
export function createProcessTreeOwner(root: ChildProcess, options: {
|
||||
readTable?: () => Promise<ProcessObservation[] | null>;
|
||||
signalGroup?: (pid: number, signal: NodeJS.Signals) => void;
|
||||
@@ -12,6 +35,8 @@ export function createProcessTreeOwner(root: ChildProcess, options: {
|
||||
let groups: ObservedProcessGroup[] = [];
|
||||
let table: ProcessObservation[] = [];
|
||||
let rootStarted: string | undefined;
|
||||
let preserved: ObservedProcessGroup[] = [];
|
||||
let restartRunner: ProcessObservation | undefined;
|
||||
let observing: Promise<void> | undefined;
|
||||
const covered = new Set<number>();
|
||||
const readTable = options.readTable ?? readProcessTable;
|
||||
@@ -30,10 +55,20 @@ export function createProcessTreeOwner(root: ChildProcess, options: {
|
||||
const rootRow = next.find(row => row.pid === root.pid);
|
||||
if (!rootStarted && !exited()) rootStarted = rootRow?.started;
|
||||
const retained = revalidateObservedProcessGroups(groups, next);
|
||||
// Never acquire a recycled group from its numeric ID alone.
|
||||
const trustedAnchors = retained.flatMap(group => group.members
|
||||
.filter(member => next.some(row => row.pid === member.pid && row.started === member.started
|
||||
&& row.processGroupId === group.processGroupId)).map(member => member.pid));
|
||||
if (rootRow && rootStarted === rootRow.started) trustedAnchors.push(rootRow.pid);
|
||||
const ownedNow = new Set(trustedAnchors.flatMap(pid =>
|
||||
observeDescendantProcessTree(next, pid).members.map(member => member.process.pid)));
|
||||
// A short-lived leader can exit between polls while its replacement
|
||||
// remains a descendant of a separately validated owner. Admit that
|
||||
// ancestry, never a recycled numeric group or a mixed ownership group.
|
||||
const lost = groups.filter(group => !retained.includes(group) && next.some(row => row.processGroupId === group.processGroupId && running(row)));
|
||||
if (lost.length) throw new Error("Owned process group identity became uncertain");
|
||||
const refreshed = refreshContinuouslyLiveProcessGroups(retained, next)
|
||||
const uncertain = lost.filter(group => next.some(row => row.processGroupId === group.processGroupId
|
||||
&& running(row) && !ownedNow.has(row.pid)));
|
||||
if (uncertain.length) throw new Error(`Owned process group identity became uncertain: ${uncertain.map(group => group.processGroupId).join(",")}`);
|
||||
const refreshed = refreshContinuouslyLiveProcessGroups([...retained, ...lost], next)
|
||||
.filter(group => group.processGroupId !== callerGroup);
|
||||
const anchors = refreshed.flatMap(group => group.members.map(member => member.pid));
|
||||
if (rootRow && rootStarted === rootRow.started) anchors.push(rootRow.pid);
|
||||
@@ -47,6 +82,14 @@ export function createProcessTreeOwner(root: ChildProcess, options: {
|
||||
}
|
||||
}
|
||||
groups = [...byGroup.values()];
|
||||
// Follow only continuously owned members of the admitted daemon tree.
|
||||
// Orphaned descendants stay owned, but unrelated controller children do not.
|
||||
const retainedPreserved = refreshContinuouslyLiveProcessGroups(revalidateObservedProcessGroups(preserved, next), next);
|
||||
const preservedIds = new Set(retainedPreserved.map(group => group.processGroupId));
|
||||
for (const member of retainedPreserved.flatMap(group => group.members)) {
|
||||
for (const group of observeDescendantProcessTree(next, member.pid).groups) preservedIds.add(group.processGroupId);
|
||||
}
|
||||
preserved = groups.filter(group => preservedIds.has(group.processGroupId));
|
||||
table = next;
|
||||
})().finally(() => { observing = undefined; });
|
||||
return observing;
|
||||
@@ -60,6 +103,33 @@ export function createProcessTreeOwner(root: ChildProcess, options: {
|
||||
function liveGroups() {
|
||||
return groups.filter(group => table.some(row => row.processGroupId === group.processGroupId && running(row)));
|
||||
}
|
||||
async function preserveRunnerForRestart(identity: RestartRunnerIdentity) {
|
||||
const expected = parseRestartRunnerIdentity(identity);
|
||||
await observe();
|
||||
if (restartRunner || exited()) throw new Error("Restart runner admission must precede controller exit");
|
||||
const tree = observeDescendantProcessTree(table, root.pid!);
|
||||
const candidate = tree.members.map(member => member.process).find(row => row.pid === expected.processPid);
|
||||
if (!candidate || !running(candidate) || candidate.kind !== "paperclip-runnerd"
|
||||
|| candidate.processGroupId !== expected.processGroupId
|
||||
|| !restartProcessStartMatches(candidate.started, expected.processStartedAt)
|
||||
|| candidate.processGroupId === root.pid) throw new Error("Restart runner is not the exact owned durable daemon");
|
||||
const subtree = observeDescendantProcessTree(table, candidate.pid);
|
||||
const members = new Set(subtree.members.map(member => member.process.pid));
|
||||
const ids = new Set(subtree.groups.map(group => group.processGroupId));
|
||||
if (table.some(row => ids.has(row.processGroupId) && !members.has(row.pid) && running(row))) {
|
||||
throw new Error("Restart runner shares a process group with an unrelated process");
|
||||
}
|
||||
restartRunner = { ...candidate };
|
||||
preserved = groups.filter(group => ids.has(group.processGroupId));
|
||||
}
|
||||
function assertRestartRunnerAlive() {
|
||||
if (!restartRunner || !table.some(row => row.pid === restartRunner!.pid && row.started === restartRunner!.started
|
||||
&& row.processGroupId === restartRunner!.processGroupId && running(row))) throw new Error("Admitted restart runner did not survive controller restart");
|
||||
}
|
||||
function restartCleanupGroups() {
|
||||
const ids = new Set(preserved.map(group => group.processGroupId));
|
||||
return new Set(liveGroups().filter(group => !ids.has(group.processGroupId)).map(group => group.processGroupId));
|
||||
}
|
||||
function signalSnapshot(signal: NodeJS.Signals, selected?: ReadonlySet<number>) {
|
||||
const currentProcessGroupId = table.find(row => row.pid === process.pid)?.processGroupId ?? null;
|
||||
if (currentProcessGroupId === null) throw new Error("Cleanup caller group identity is unavailable");
|
||||
@@ -118,6 +188,7 @@ export function createProcessTreeOwner(root: ChildProcess, options: {
|
||||
graceComplete = selected.size > 0 && delivered.length === selected.size;
|
||||
}
|
||||
return { observe, signal, liveGroups, gracefulRoots, signalGracefully,
|
||||
preserveRunnerForRestart, assertRestartRunnerAlive, restartCleanupGroups,
|
||||
directGraceDelivered: () => directGraceDelivered, stopObserving: () => { if (timer) clearInterval(timer); } };
|
||||
}
|
||||
|
||||
|
||||
@@ -159,6 +159,14 @@ const diagnosticProcessKinds = new Set([
|
||||
"tsx",
|
||||
]);
|
||||
|
||||
export function diagnosticProcessKind(command: string, platform: NodeJS.Platform = process.platform) {
|
||||
const name = path.basename(command);
|
||||
// Linux comm is limited to 15 bytes. This is a diagnostic role only;
|
||||
// preservation still requires the trusted run PID/start/group and ancestry.
|
||||
if (platform === "linux" && name === "paperclip-runne") return "paperclip-runnerd";
|
||||
return diagnosticProcessKinds.has(name) ? name : "other";
|
||||
}
|
||||
|
||||
export async function readProcessTable(startInspector?: () => ChildProcess): Promise<ProcessObservation[] | null> {
|
||||
if (process.platform === "win32") {
|
||||
return null;
|
||||
@@ -212,8 +220,7 @@ export async function readProcessTable(startInspector?: () => ChildProcess): Pro
|
||||
// A target process can choose its own argv and process name. Emit a
|
||||
// fixed category instead of target-controlled text so diagnostics
|
||||
// can never turn that metadata into a secret-exfiltration channel.
|
||||
const command = path.basename(match[6]!);
|
||||
const kind = diagnosticProcessKinds.has(command) ? command : "other";
|
||||
const kind = diagnosticProcessKind(match[6]!);
|
||||
return {
|
||||
pid: Number(match[1]),
|
||||
parentPid: Number(match[2]),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { explicitlyRequestsFileOutput } from "../../server/src/services/native-runtime/native-deliverable-feedback.js";
|
||||
import { copilotProtectionTasks } from "./copilot-protection-tasks.js";
|
||||
import type { APIRequestContext, APIResponse } from "@playwright/test";
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import { classifyFailure } from "./failure-classifier.js";
|
||||
@@ -11,7 +12,7 @@ afterEach(() => { vi.useRealTimers(); vi.unstubAllEnvs(); });
|
||||
function harness(timeoutMs = REMOTE_FIXTURE_MIN_SETUP_BUDGET_MS + 18_000) {
|
||||
const order: string[] = [];
|
||||
const issue = { id: "issue", companyId: "company", assigneeAgentId: "agent" };
|
||||
const run = { id: "run", companyId: "company", agentId: "agent", status: "running", executionStage: "preparing" };
|
||||
const run = { id: "run", companyId: "company", agentId: "agent", status: "running", executionStage: "preparing", runtimeMode: "native", nativePhase: "provider_running" };
|
||||
const leases = [{ id: "lease", heartbeatRunId: "run", issueId: "issue", status: "active", providerLeaseId: "sandbox" }];
|
||||
const api = { get: vi.fn(async (path: string) => path === "/api/issues/issue" ? issue : path === "/api/heartbeat-runs/run" ? run : leases) };
|
||||
const fixture = {
|
||||
@@ -48,6 +49,19 @@ it("keeps negated bootstrap instructions out of the production file-delivery con
|
||||
expect(explicitlyRequestsFileOutput(`${prompt}\nCreate a downloadable report.txt file.`)).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps async completion immediate with a private sentinel and no delivery wait", () => {
|
||||
const task = copilotProtectionTasks.find(task => task.id === "attached-async-settlement")!;
|
||||
const action = task.buildPrompt("nonce");
|
||||
expect(explicitlyRequestsFileOutput(action)).toBe(false);
|
||||
expect(explicitlyRequestsFileOutput(`${harness().bootstrap.prompt("nonce")}\n${action}`)).toBe(false);
|
||||
expect(action).toContain("attempt to finish immediately without calling read_bash or another waiting tool");
|
||||
expect(action).toContain("Do not modify fixture code, manufacture its result, or start another command.");
|
||||
expect(action).toContain("private diagnostic sentinel, not a requested file deliverable");
|
||||
expect(action).toContain("Do not publish it. Do not attach it.");
|
||||
expect(explicitlyRequestsFileOutput(`${action}\nAttach a downloadable report.txt file.`)).toBe(true);
|
||||
expect(action).toContain("evidence [], verification []");
|
||||
expect(action).not.toContain("register_deliverable");
|
||||
});
|
||||
|
||||
it("keeps the complete readiness/install reserve when a lease arrives at the admission boundary", async () => {
|
||||
vi.useFakeTimers(); vi.setSystemTime(0);
|
||||
@@ -106,6 +120,13 @@ it("waits through queued admission without publishing early", async () => {
|
||||
expect(count).toBe(2); expect(h.fixture.publishAction).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it.each(["provider_running", "observed"])("admits a native %s run while its legacy stage stays preparing", async nativePhase => {
|
||||
const h = harness(); h.bootstrap.prompt("native-ready"); h.run.nativePhase = nativePhase;
|
||||
await expect(h.bootstrap.bindAndRelease(h.request)).resolves.toBe(h.fixture);
|
||||
expect(h.run.executionStage).toBe("preparing");
|
||||
expect(h.bind).toHaveBeenCalledTimes(1); expect(h.fixture.publishAction).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it.each(["", "x".repeat(16385)])("rejects empty or over-bound action after closing only its observer", async action => {
|
||||
const h = harness(); h.bootstrap.prompt("nonce");
|
||||
await expect(h.bootstrap.bindAndRelease({ ...h.request, actionPrompt: () => action })).rejects.toThrow("empty or too large");
|
||||
|
||||
@@ -173,6 +173,8 @@ export function createRemoteNativeBootstrap(input: {
|
||||
if (lastReadError !== undefined) throw lastReadError;
|
||||
return state;
|
||||
},
|
||||
// Native runs can retain the legacy "preparing" stage. The observer's
|
||||
// runtime-ready RPC proves the actual pinned daemon before installation.
|
||||
accept: state => !state.rejection && state.owned && state.run?.status === "running" && Boolean(state.lease)
|
||||
&& Date.now() < admissionDeadlineAt,
|
||||
reject: state => state.rejection,
|
||||
|
||||
@@ -7,8 +7,9 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { Script } from "node:vm";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { REMOTE_FIXTURE_MIN_SETUP_BUDGET_MS, bindRemoteNativeFixture, createRemoteTargetWatch, isRemoteRunRoot, parseRemoteProcStat, remoteNativeFixtureDiagnostics, type RemoteNativeFixtureOptions, type RemoteNativeSnapshot } from "./remote-native-fixtures.js";
|
||||
import { REMOTE_FIXTURE_MIN_SETUP_BUDGET_MS, bindRemoteNativeFixture, validatePiProviderDeathReceipt, createRemoteTargetWatch, isRemoteRunRoot, parseRemoteProcStat, remoteProcEntryDisappeared, remoteNativeFixtureDiagnostics, remoteNativeIncompleteTerminalEvidence, type RemoteNativeFixtureOptions, type RemoteNativeSnapshot } from "./remote-native-fixtures.js";
|
||||
|
||||
import { PI_DISTRIBUTION_CLOSURE_SHA256 } from "../../packages/paperclip-runner/src/drivers/acpx/pi-closure-pins.js";
|
||||
import { createRemoteNativeBootstrap } from "./remote-native-bootstrap.js";
|
||||
|
||||
const hash = (s: string) => `sha256:${createHash("sha256").update(s).digest("hex")}`;
|
||||
@@ -52,6 +53,19 @@ function harness() {
|
||||
setLease(value: Record<string, unknown>) { lease = value; }, lease: () => lease, override(fn: typeof override) { override = fn; } };
|
||||
}
|
||||
|
||||
describe("remote process exit during proc reads", () => {
|
||||
it.each(["ENOENT", "ESRCH"])("accepts confirmed %s process absence", code => {
|
||||
const confirm = vi.fn();
|
||||
expect(remoteProcEntryDisappeared({ code }, confirm)).toBe(true);
|
||||
expect(confirm).not.toHaveBeenCalled();
|
||||
expect(remoteProcEntryDisappeared({ code: "EIO" }, () => { throw { code }; })).toBe(true);
|
||||
});
|
||||
it.each(["EIO", "EACCES", undefined])("rejects an unreadable existing process (%s)", code => {
|
||||
expect(remoteProcEntryDisappeared({ code }, () => ({ isDirectory: () => true }))).toBe(false);
|
||||
expect(remoteProcEntryDisappeared({ code }, () => { throw { code: "EACCES" }; })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("remote native lease admission", () => {
|
||||
it.each(["companyId", "environmentId", "heartbeatRunId", "providerLeaseId", "provider", "status"])("rejects wrong %s before executing any remote command", async key => {
|
||||
const h = harness(); h.setLease({ ...h.lease(), [key]: "foreign" });
|
||||
@@ -232,19 +246,63 @@ describe("remote native lease admission", () => {
|
||||
if (type === "bad-file") end.targets["result.txt"] = { ...end.targets["result.txt"], absent: false, sha256: hash("missing") };
|
||||
h.resolveTerminal(end); await expect(f.finish()).rejects.toThrow();
|
||||
});
|
||||
it("reports only closed terminal failure reasons without weakening retirement proof", async () => {
|
||||
it("retains a validated incomplete terminal receipt without raw file or RPC content", async () => {
|
||||
const h = harness(), f = await bindRemoteNativeFixture(h.options);
|
||||
await f.publishAction("action.txt", "task");
|
||||
const end = { ...structuredClone(h.current), complete: false,
|
||||
processes: { ...h.current.processes, live: [] }, files: {},
|
||||
failureCodes: ["process_pid_reused", "secret-provider-payload", "process_pid_reused", { arbitrary: "payload" }] };
|
||||
const end = { ...structuredClone(h.current), complete: false, watcher: { ...h.current.watcher, complete: false },
|
||||
processes: { ...h.current.processes, live: [] }, files: { "private.txt": "PRIVATE RPC CONTENT" }, untrustedDump: "PRIVATE RPC CONTENT" };
|
||||
h.resolveTerminal(end);
|
||||
const error = await f.finish().catch(error => error);
|
||||
expect(error.message).toContain("terminal_evidence_incomplete:published=true,complete=false");
|
||||
expect(error.message).toContain(":process_pid_reused");
|
||||
expect(error.message).not.toContain("secret-provider-payload");
|
||||
expect(error.message).not.toContain("arbitrary");
|
||||
expect(remoteNativeFixtureDiagnostics(error)).toEqual([{ phase: "wait", code: "terminal_evidence_incomplete" }]);
|
||||
expect(error.message).toBe("remote_native_fixture:terminal_evidence_incomplete");
|
||||
const retained = remoteNativeIncompleteTerminalEvidence(error)!;
|
||||
expect(retained.complete).toBe(false); expect(retained.watcher.complete).toBe(false);
|
||||
expect(retained.processes.liveCount).toBe(0);
|
||||
expect(retained.incompleteReasons).toEqual([]);
|
||||
expect(JSON.stringify(retained)).not.toContain("PRIVATE RPC CONTENT");
|
||||
retained.watcher.complete = true;
|
||||
expect(remoteNativeIncompleteTerminalEvidence(error)!.watcher.complete).toBe(false);
|
||||
expect(remoteNativeIncompleteTerminalEvidence(new Error("PRIVATE RPC CONTENT"))).toBeUndefined();
|
||||
});
|
||||
it("retains only closed incompleteness reasons and rejects raw diagnostics", async () => {
|
||||
const h = harness(), f = await bindRemoteNativeFixture(h.options); await f.publishAction("action.txt", "task");
|
||||
h.resolveTerminal({ ...structuredClone(h.current), complete: false, incompleteReasons: ["unwatched_directory"],
|
||||
processes: { ...h.current.processes, live: [] }, files: {} });
|
||||
const error = await f.finish().catch(error => error);
|
||||
expect(remoteNativeIncompleteTerminalEvidence(error)?.incompleteReasons).toEqual(["unwatched_directory"]);
|
||||
await f.close();
|
||||
const bad = harness(), other = await bindRemoteNativeFixture(bad.options); await other.publishAction("action.txt", "task");
|
||||
bad.resolveTerminal({ ...structuredClone(bad.current), complete: false, incompleteReasons: ["PRIVATE RAW PATH"],
|
||||
processes: { ...bad.current.processes, live: [] }, files: {} });
|
||||
const rejected = await other.finish().catch(error => error);
|
||||
expect(rejected.message).toContain("incomplete_reasons_shape");
|
||||
expect(remoteNativeIncompleteTerminalEvidence(rejected)).toBeUndefined();
|
||||
await other.close();
|
||||
});
|
||||
it("keeps incomplete evidence failed while closing a retired observer after public lease deletion", async () => {
|
||||
const h = harness(), f = await bindRemoteNativeFixture(h.options);
|
||||
await f.publishAction("action.txt", "task");
|
||||
h.resolveTerminal({ ...structuredClone(h.current), complete: false, incompleteReasons: ["unwatched_directory"],
|
||||
watcher: { ...h.current.watcher, complete: false }, processes: { ...h.current.processes, live: [] }, files: {} });
|
||||
await expect(f.finish()).rejects.toThrow("terminal_evidence_incomplete");
|
||||
h.setLease({ ...h.lease(), status: "released", releasedAt: "2026-10-06T01:00:00Z" });
|
||||
const before = h.calls.length;
|
||||
await expect(f.close()).resolves.toBeUndefined();
|
||||
expect(h.calls).toHaveLength(before);
|
||||
await expect(f.finish()).rejects.toThrow("terminal_evidence_incomplete");
|
||||
await expect(f.readFile("result.txt")).rejects.toThrow("unregistered_read");
|
||||
});
|
||||
it.each(["live", "missing-root", "invalid-shape", "unknown-cause", "reused-process", "attached-live"])("does not treat %s evidence as retired cleanup", async kind => {
|
||||
const h = harness(), f = await bindRemoteNativeFixture(h.options);
|
||||
await f.publishAction("action.txt", "task");
|
||||
const end: any = { ...structuredClone(h.current), complete: false, processes: { ...h.current.processes, live: [] }, files: {} };
|
||||
if (kind === "live") end.processes.live = [21];
|
||||
if (kind === "missing-root") end.processes = { captured: false, root: null, journal: [], live: [] };
|
||||
if (kind === "invalid-shape") end.observedAtMs = "invalid";
|
||||
if (kind === "reused-process") end.incompleteReasons = ["process_identity_reused"];
|
||||
if (kind === "attached-live") end.incompleteReasons = ["attached_process_live"];
|
||||
h.resolveTerminal(end); await expect(f.finish()).rejects.toThrow();
|
||||
h.setLease({ ...h.lease(), status: "released", releasedAt: "2026-10-06T01:00:00Z" });
|
||||
await expect(f.close()).rejects.toThrow("lease_scope");
|
||||
});
|
||||
it("keeps a fixture-owned cross-root sentinel distinct from workspace targets", async () => {
|
||||
const h = harness(); h.options.crossRoot = { initialText: "outside sentinel" };
|
||||
@@ -334,7 +392,7 @@ describe("cell deadline and cleanup bounds", () => {
|
||||
});
|
||||
|
||||
describe("independent filesystem and process observations", () => {
|
||||
it.skipIf(process.platform !== "linux")("retains transient create/delete events and detects same-path parent replacement", async () => {
|
||||
it("retains transient create/delete events and detects same-path parent replacement", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "remote-watch-")); const watcher = createRemoteTargetWatch(dir, "denied.txt");
|
||||
try {
|
||||
await writeFile(join(dir, "denied.txt"), "forbidden"); await rm(join(dir, "denied.txt"));
|
||||
@@ -378,7 +436,7 @@ describe("actual generated observer state machine", () => {
|
||||
const handlers: Array<(socket: any) => void> = [], children: any[] = [];
|
||||
const missing = () => { throw Object.assign(new Error("missing"), { code: "ENOENT" }); };
|
||||
const fds = new Map<number, string>(); let nextFd = 50, runtimeInode = 4n;
|
||||
const symbolicLinks = new Set<string>();
|
||||
const symbolicLinks = new Set<string>(), directoryInodes = new Map<string, bigint>();
|
||||
const directories = new Set(["/tmp", "/workspace", "/workspace/.paperclip-runtime", "/workspace/.paperclip-runtime/paperclip-runner", "/workspace/.paperclip-runtime/paperclip-runner/sessions"]);
|
||||
if (!deferStartup) directories.add(config.root);
|
||||
const listeners = new Map<string, (socket: any) => void>();
|
||||
@@ -402,10 +460,10 @@ describe("actual generated observer state machine", () => {
|
||||
lstatSync(path: string) {
|
||||
const directory = directories.has(path);
|
||||
if (!directory && !files.has(path) && !symbolicLinks.has(path)) return missing();
|
||||
return { dev: 1n, ino: path === config.root ? 2n : path === "/workspace/.paperclip-runtime/paperclip-runner" ? runtimeInode : 3n, mtimeNs: 4n, ctimeNs: 5n, isDirectory: () => directory, isFile: () => !directory, isSymbolicLink: () => symbolicLinks.has(path), size: files.get(path)?.length ?? 0 };
|
||||
return { dev: 1n, ino: directoryInodes.get(path) ?? (path === config.root ? 2n : path === "/workspace/.paperclip-runtime/paperclip-runner" ? runtimeInode : 3n), mtimeNs: 4n, ctimeNs: 5n, isDirectory: () => directory, isFile: () => !directory, isSymbolicLink: () => symbolicLinks.has(path), size: files.get(path)?.length ?? 0 };
|
||||
},
|
||||
realpathSync: (path: string) => path,
|
||||
readdirSync(path: string) { if (path === "/proc") return [...proc.keys()].map(String); if (path === "/workspace") return [".paperclip-runtime", ...[...files.keys(), ...symbolicLinks].filter(p => p.startsWith("/workspace/") && !p.slice(11).includes("/")).map(p => p.slice(11))]; if (path === "/workspace/.paperclip-runtime") return ["reusable-sandbox-lease.json", "paperclip-runner", ...[...files.keys()].filter(p => p.startsWith(path + "/") && !p.slice(path.length + 1).includes("/") && !p.endsWith("reusable-sandbox-lease.json")).map(p => p.slice(path.length + 1))]; if (path.startsWith("/workspace/.paperclip-runtime/paperclip-runner")) throw new Error("excluded runtime must not be traversed"); return []; },
|
||||
readdirSync(path: string) { if (path === "/proc") return [...proc.keys()].map(String); if (path.startsWith("/workspace/.paperclip-runtime/paperclip-runner")) throw new Error("excluded runtime must not be traversed"); return [...new Set([...files.keys(), ...directories, ...symbolicLinks].filter(p => p.startsWith(path + "/") && !p.slice(path.length + 1).includes("/")).map(p => p.slice(path.length + 1)))]; },
|
||||
watch(path: string, options: unknown, callback?: (_kind: string, name: string | null) => void) {
|
||||
const entry = { path, callback: (callback ?? options) as (_kind: string, name: string | null) => void, closed: false }; watches.push(entry);
|
||||
return Object.assign(new EventEmitter(), { close: () => { entry.closed = true; } });
|
||||
@@ -432,8 +490,9 @@ describe("actual generated observer state machine", () => {
|
||||
close: vi.fn(() => listeners.clear()),
|
||||
};
|
||||
const net = { createServer(fn: (socket: any) => void) { handlers.push(fn); return server; } };
|
||||
const faultSpawn = vi.fn(() => ({ status: 0, stdout: JSON.stringify(piFaultReceipt()) }));
|
||||
const context = {
|
||||
require(name: string) { if (name === "node:fs") return fs; if (name === "node:net") return net; if (name === "node:child_process") return { spawn: vi.fn(() => { const child = Object.assign(new EventEmitter(), { pid: 88, exitCode: null, signalCode: null, kill: vi.fn() }); children.push(child); return child; }) }; if (name === "node:path") return { join: (...paths: string[]) => paths.join("/"), dirname: (path: string) => path.slice(0, path.lastIndexOf("/")), basename: (path: string) => path.slice(path.lastIndexOf("/") + 1) }; if (name === "node:crypto") return { createHash }; throw new Error("unexpected module"); },
|
||||
require(name: string) { if (name === "node:fs") return fs; if (name === "node:net") return net; if (name === "node:child_process") return { spawnSync: faultSpawn, spawn: vi.fn(() => { const child = Object.assign(new EventEmitter(), { pid: 88, exitCode: null, signalCode: null, kill: vi.fn() }); children.push(child); return child; }) }; if (name === "node:path") return { join: (...paths: string[]) => paths.join("/"), dirname: (path: string) => path.slice(0, path.lastIndexOf("/")), basename: (path: string) => path.slice(path.lastIndexOf("/") + 1) }; if (name === "node:crypto") return { createHash }; throw new Error("unexpected module"); },
|
||||
process: { argv: ["node", `${config.root}/observer.cjs`, Buffer.from(JSON.stringify(config)).toString("base64")], execPath: "/node", hrtime: { bigint: () => 12345n }, exit: vi.fn() },
|
||||
Buffer, __filename: `${config.root}/observer.cjs`,
|
||||
setInterval(fn: () => void) { intervals.push(fn); return 1; }, clearInterval: vi.fn(),
|
||||
@@ -445,8 +504,56 @@ describe("actual generated observer state machine", () => {
|
||||
const replies: any[] = [], socket = Object.assign(new EventEmitter(), { end: (value: string) => replies.push(JSON.parse(value)), destroy: vi.fn() });
|
||||
handlers[0]!(socket); socket.emit("data", Buffer.from(JSON.stringify({ op, nonce: config.nonce, ...args }) + "\n")); return replies;
|
||||
}
|
||||
return { request, proc, files, watches, fs, intervals, timers, config, handlers, children, symbolicLinks, context, server, directories, listeners, install: h.calls.find(c => c.request.op === "install")!, replaceRuntimeRoot() { runtimeInode = 999n; } };
|
||||
return { request, proc, files, watches, fs, intervals, timers, config, handlers, children, faultSpawn, symbolicLinks, directoryInodes, context, server, directories, listeners, install: h.calls.find(c => c.request.op === "install")!, replaceRuntimeRoot() { runtimeInode = 999n; } };
|
||||
}
|
||||
it("keeps existing watched directory notifications complete without losing nested mutations", async () => {
|
||||
const o = await observerHarness(true);
|
||||
o.directories.add(o.config.root); o.files.set(`${o.config.root}/observer.cjs`, Buffer.from(o.install.request.source));
|
||||
o.directories.add("/workspace/existing");
|
||||
new Script(o.install.request.source).runInNewContext(o.context);
|
||||
// Linux emits a parent notification when chmod changes a child's directory
|
||||
// attributes. Its inode and registered recursive watch remain unchanged.
|
||||
for (const w of o.watches) if (w.path === "/workspace") w.callback("change", "existing");
|
||||
expect(o.request("snapshot")[0].result.complete).toBe(true);
|
||||
o.fs.writeFileSync("/workspace/existing/transient.txt", "changed", { flag: "wx" });
|
||||
o.files.delete("/workspace/existing/transient.txt");
|
||||
o.watches.find(w => w.path === "/workspace/existing")!.callback("rename", "transient.txt");
|
||||
const final = o.request("snapshot")[0].result;
|
||||
expect(final.complete).toBe(true); expect(final.watcher.workspaceMutationCount).toBe(3);
|
||||
expect(final.workspace["existing/transient.txt"]).toBeUndefined();
|
||||
});
|
||||
it.each(["new", "replacement", "symlink", "unknown"])("rejects %s directory notifications", async variant => {
|
||||
const o = await observerHarness(true);
|
||||
o.directories.add(o.config.root); o.files.set(`${o.config.root}/observer.cjs`, Buffer.from(o.install.request.source));
|
||||
if (variant !== "new") o.directories.add("/workspace/existing");
|
||||
new Script(o.install.request.source).runInNewContext(o.context);
|
||||
if (variant === "new") o.directories.add("/workspace/existing");
|
||||
if (variant === "replacement") o.directoryInodes.set("/workspace/existing", 999n);
|
||||
if (variant === "symlink") o.symbolicLinks.add("/workspace/existing");
|
||||
for (const w of o.watches) if (w.path === "/workspace") w.callback("change", variant === "unknown" ? null : "existing");
|
||||
const reply = o.request("snapshot")[0];
|
||||
expect(reply.ok && reply.result.complete).toBe(false);
|
||||
});
|
||||
it("executes the actual observer fault phase once with a closed environment and exact identity", async () => {
|
||||
for (const published of [false, true]) {
|
||||
const rejected = await observerHarness();
|
||||
if (published) rejected.request("publish", { path: "action.txt", text: "ask native input" });
|
||||
expect(rejected.request("pi-provider-death", { runtimeEnvironmentLeaseId: published ? "foreign" : "workspace-id" })[0].ok).toBe(false);
|
||||
expect(rejected.faultSpawn).not.toHaveBeenCalled();
|
||||
}
|
||||
const o = await observerHarness();
|
||||
expect(o.request("snapshot")[0].ok).toBe(true);
|
||||
expect(o.request("publish", { path: "action.txt", text: "ask native input" })[0].ok).toBe(true);
|
||||
expect(o.request("pi-provider-death", { runtimeEnvironmentLeaseId: "workspace-id" })[0]).toEqual({ ok: true, result: piFaultReceipt() });
|
||||
expect(o.faultSpawn).toHaveBeenCalledTimes(1);
|
||||
const [program, args, options] = o.faultSpawn.mock.calls[0] as unknown as [string, string[], Record<string, unknown>];
|
||||
expect(program).toBe("/usr/bin/python3"); expect(args.slice(0, 2)).toEqual(["-I", "-c"]);
|
||||
expect(args[2]).toContain("signal.pidfd_send_signal");
|
||||
expect(JSON.parse(Buffer.from(args[3]!, "base64").toString())).toEqual({ root, binding, runtimeEnvironmentLeaseId: "workspace-id", runnerdSha256: hash("runnerd"), closureSha256: PI_DISTRIBUTION_CLOSURE_SHA256["linux-x64"] });
|
||||
expect(options).toMatchObject({ env: { PATH: "/usr/bin:/bin" }, timeout: 8000, maxBuffer: 32768 });
|
||||
expect(o.request("pi-provider-death", { runtimeEnvironmentLeaseId: "workspace-id" })[0].ok).toBe(false);
|
||||
expect(o.faultSpawn).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
async function generatedRpc(o: Awaited<ReturnType<typeof observerHarness>>, request: Record<string, unknown>, mutateSource = (source: string) => source) {
|
||||
const quoted = o.install.command.match(/ -e (.+) '[A-Za-z0-9+/=]+'$/su)![1]!;
|
||||
const source = quoted.slice(1, -1).replaceAll("'\\''", "'");
|
||||
@@ -689,7 +796,7 @@ describe("actual generated observer state machine", () => {
|
||||
const o = await observerHarness(); o.request("snapshot"); const wait = o.request("wait");
|
||||
o.proc.set(21, { ppid: 1, group: 99, ticks: "999", argv: ["/unrelated"] }); o.intervals[0]!(); o.timers.find(t => t.ms === 100)!.fn();
|
||||
expect(wait[0].result.complete).toBe(false); expect(wait[0].result.processes.root.startTicks).toBe("100");
|
||||
expect(wait[0].result.failureCodes).toContain("process_pid_reused");
|
||||
expect(wait[0].result.incompleteReasons).toContain("process_identity_reused");
|
||||
});
|
||||
it("counts transient workspace create/delete and rejects changed setup-file bytes", async () => {
|
||||
const o = await observerHarness(); o.request("snapshot");
|
||||
@@ -707,3 +814,41 @@ describe("actual generated observer state machine", () => {
|
||||
expect(other.request("snapshot")[0].ok).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
function piFaultReceipt() {
|
||||
const parent = { pid: 22, ppid: 21, startTicks: "101", bootId }, target = { pid: 23, ppid: 22, startTicks: "102", bootId };
|
||||
return { schema: "paperclip.e2e.pi-provider-death.v1", binding, runtimeEnvironmentLeaseId: "workspace-id", target, ancestry: [target, parent, root],
|
||||
nodeSha256: hash("node"), entrypointSha256: hash("entry"), closureSha256: PI_DISTRIBUTION_CLOSURE_SHA256["linux-x64"],
|
||||
entrypointAttribution: "pinned_wrapper_parent", originalChildArgvAvailable: false, observedChildTitle: "pi", signalled: true, signal: "SIGKILL", targetKind: "pi_native_child", workerSignalled: false };
|
||||
}
|
||||
describe("exact Pi-child fault capability", () => {
|
||||
it("validates ancestry and honest parent-attested entrypoint identity", () => {
|
||||
const receipt = piFaultReceipt();
|
||||
expect(validatePiProviderDeathReceipt(receipt, binding, root, "workspace-id")).toEqual(receipt);
|
||||
for (const patch of [{ workerSignalled: true }, { target: root }, { originalChildArgvAvailable: true }, { entrypointAttribution: "process_title" },
|
||||
{ binding: { ...binding, runId: "foreign" } }, { runtimeEnvironmentLeaseId: "foreign" }, { closureSha256: "0".repeat(64) },
|
||||
{ ancestry: [receipt.target, { ...receipt.ancestry[1], startTicks: "" }, root] }, { ancestry: [receipt.target, root] },
|
||||
{ ancestry: [receipt.target, { ...receipt.ancestry[1], ppid: 999 }, root] }, { ancestry: [receipt.target, receipt.ancestry[1], { ...root, startTicks: "999" }] }]) {
|
||||
expect(() => validatePiProviderDeathReceipt({ ...receipt, ...patch }, binding, root, "workspace-id")).toThrow();
|
||||
}
|
||||
});
|
||||
it("requires published action and consumes the one-shot capability even on RPC failure", async () => {
|
||||
const h = harness(), fixture = await bindRemoteNativeFixture(h.options);
|
||||
await expect(fixture.terminatePiProvider!("workspace-id")).rejects.toThrow();
|
||||
await fixture.publishAction("action.txt", "Ask the native question");
|
||||
h.override(request => request.op === "pi-provider-death" ? { exitCode: 0, result: JSON.stringify({ ok: false, error: "pi_provider_identity_or_signal_failed" }) } : undefined);
|
||||
await expect(fixture.terminatePiProvider!("workspace-id")).rejects.toThrow();
|
||||
await expect(fixture.terminatePiProvider!("workspace-id")).rejects.toThrow();
|
||||
expect(h.calls.filter(call => call.request.op === "pi-provider-death")).toHaveLength(1);
|
||||
await fixture.close();
|
||||
});
|
||||
it("returns only the exact reviewed fault receipt and never signals the worker", async () => {
|
||||
const h = harness(), fixture = await bindRemoteNativeFixture(h.options);
|
||||
await fixture.publishAction("action.txt", "Ask the native question");
|
||||
h.override(request => request.op === "pi-provider-death" ? { exitCode: 0, result: JSON.stringify({ ok: true, result: piFaultReceipt() }) } : undefined);
|
||||
expect(await fixture.terminatePiProvider!("workspace-id")).toEqual(piFaultReceipt());
|
||||
await expect(fixture.terminatePiProvider!("workspace-id")).rejects.toThrow();
|
||||
expect(h.calls.filter(call => call.request.op === "pi-provider-death")).toHaveLength(1);
|
||||
await fixture.close();
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,10 @@
|
||||
import { createHash, randomBytes } from "node:crypto";
|
||||
import { watch, lstatSync } from "node:fs";
|
||||
import { watch, lstatSync, readFileSync } from "node:fs";
|
||||
import { posix } from "node:path";
|
||||
|
||||
import { PI_DISTRIBUTION_CLOSURE_SHA256 } from "../../packages/paperclip-runner/src/drivers/acpx/pi-closure-pins.js";
|
||||
const PI_FAULT_SOURCE = readFileSync(new URL("./pi-provider-fault.py", import.meta.url), "utf8");
|
||||
|
||||
export const REMOTE_FIXTURE_DAYTONA_SDK_VERSION = "0.203.0";
|
||||
const NODE = "/opt/paperclip-runner/provider-pack/node_modules/node/bin/node";
|
||||
const MAX_OUTPUT = 256 * 1024;
|
||||
@@ -30,8 +33,8 @@ function relative(value: string): string {
|
||||
}
|
||||
// Only closed diagnostic enums cross the remote boundary; SDK errors and output
|
||||
// are never retained. These diagnostics explain failed evidence, not qualification.
|
||||
const RPC_PHASES = ["runtime-ready", "install", "wait", "close", "arm", "publish", "snapshot", "attached", "read", "inject-loss"] as const;
|
||||
const RPC_CODES = ["node_identity", "sentinel_type", "sentinel", "cwd", "runtime_root_identity", "runtime_binary_identity", "proc_bound", "ambiguous_run_root", "runtime_not_ready", "runtime_identity_changed", "invalid_proc_identity", "invalid_proc_fields", "socket_error", "socket_timeout", "output_bound", "rpc_deadline", "remote_unknown", "transport_failure", "invalid_response", "readiness_deadline", "terminal_evidence_incomplete"] as const;
|
||||
const RPC_PHASES = ["runtime-ready", "install", "wait", "close", "arm", "publish", "snapshot", "attached", "read", "pi-provider-death"] as const;
|
||||
const RPC_CODES = ["node_identity", "sentinel_type", "sentinel", "cwd", "runtime_root_identity", "runtime_binary_identity", "proc_bound", "ambiguous_run_root", "runtime_not_ready", "runtime_identity_changed", "invalid_proc_identity", "invalid_proc_fields", "socket_error", "socket_timeout", "output_bound", "rpc_deadline", "remote_unknown", "transport_failure", "invalid_response", "readiness_deadline", "pi_provider_identity_or_signal_failed"] as const;
|
||||
type RpcPhase = typeof RPC_PHASES[number];
|
||||
type RpcCode = typeof RPC_CODES[number];
|
||||
interface RpcDiagnostic { phase: RpcPhase; code: RpcCode }
|
||||
@@ -48,12 +51,16 @@ export function remoteNativeFixtureDiagnostics(error: unknown): RpcDiagnostic[]
|
||||
export interface RemoteNativeAuthority { companyId: string; environmentId: string; runId: string; leaseId: string; sandboxId: string; image: string }
|
||||
export interface RemoteNativeBinding extends RemoteNativeAuthority { remoteCwd: string }
|
||||
export interface RemoteProcessIdentity { pid: number; ppid: number; startTicks: string; bootId: string }
|
||||
const INCOMPLETE_REASONS = ["multiple_run_roots", "run_root_changed", "process_identity_reused", "process_sample_failed", "process_read_io", "process_stat_shape", "process_runtime_binding", "process_terminal_seal_failed", "unknown_watch_name", "setup_file_changed", "workspace_event_bound", "workspace_symlink", "unwatched_directory", "directory_identity_changed", "watch_stat_failed", "workspace_watch_error", "target_watch_incomplete", "receipt_byte_bound", "attached_process_live", "control_request_bound", "observer_ttl"] as const;
|
||||
type IncompleteReason = typeof INCOMPLETE_REASONS[number];
|
||||
const FILESYSTEM_INCOMPLETE_REASONS: readonly IncompleteReason[] = ["unknown_watch_name", "setup_file_changed", "workspace_event_bound", "workspace_symlink", "unwatched_directory", "directory_identity_changed", "watch_stat_failed", "workspace_watch_error", "target_watch_incomplete"];
|
||||
export interface RemoteNativeSnapshot {
|
||||
binding: RemoteNativeBinding;
|
||||
observedAtMs: number;
|
||||
observedMonotonicNs: string;
|
||||
receivedAtMs: number;
|
||||
complete: boolean;
|
||||
incompleteReasons?: IncompleteReason[];
|
||||
workspace: Record<string, string>;
|
||||
targets: Record<string, { absent: boolean; sha256: string | null; parent: { dev: string; ino: string }; mutationCount: number; complete: boolean }>;
|
||||
watcher: { complete: boolean; targetMutationCount: number; workspaceMutationCount: number };
|
||||
@@ -67,6 +74,26 @@ export interface RemoteNativeSnapshot {
|
||||
setup: { path: string; sha256: string | null; published: boolean };
|
||||
attached: { connections: number; failure: string | null; commandExit: { code: number; observedAtMs: number; observedMonotonicNs: string } | null; markerWrittenAtMs: number | null; markerWrittenMonotonicNs: string | null; clientExitedAtMs: number | null; clientExitedMonotonicNs: string | null } | null;
|
||||
}
|
||||
class IncompleteRemoteTerminalEvidenceError extends Error {
|
||||
constructor(readonly snapshot: RemoteNativeSnapshot) {
|
||||
super("remote_native_fixture:terminal_evidence_incomplete");
|
||||
}
|
||||
}
|
||||
/** Retain only a shape-validated public receipt, without raw RPC text or files. */
|
||||
export function remoteNativeIncompleteTerminalEvidence(error: unknown) {
|
||||
if (!(error instanceof IncompleteRemoteTerminalEvidenceError)) return undefined;
|
||||
const row = error.snapshot, root = row.processes.root;
|
||||
return {
|
||||
observedAtMs: row.observedAtMs, receivedAtMs: row.receivedAtMs, complete: row.complete,
|
||||
incompleteReasons: [...(row.incompleteReasons ?? [])],
|
||||
setupPublished: row.setup.published,
|
||||
watcher: { complete: row.watcher.complete, targetMutationCount: row.watcher.targetMutationCount, workspaceMutationCount: row.watcher.workspaceMutationCount },
|
||||
processes: { captured: row.processes.captured, liveCount: row.processes.live.length, journalCount: row.processes.journal.length,
|
||||
root: root ? { pid: root.pid, ppid: root.ppid, startTicks: root.startTicks, bootId: root.bootId } : null },
|
||||
targets: Object.fromEntries(Object.entries(row.targets).map(([name, target]) => [name,
|
||||
{ absent: target.absent, sha256: target.sha256, mutationCount: target.mutationCount, complete: target.complete }])),
|
||||
};
|
||||
}
|
||||
/** Structural subset of pinned SDK0.203.0; caller supplies its authenticated
|
||||
* client. No create/list/delete or arbitrary remote command API is exposed. */
|
||||
export interface RemoteFixtureDaytona {
|
||||
@@ -84,6 +111,17 @@ export function parseRemoteProcStat(pid: number, stat: string, bootId: string):
|
||||
if (!/^\d+$/u.test(fields[19] ?? "") || !/^\d+$/u.test(fields[1] ?? "") || !/^\d+$/u.test(fields[2] ?? "")) throw new Error("invalid_proc_fields");
|
||||
return { pid, ppid: Number(fields[1]), group: Number(fields[2]), state: fields[0]!, startTicks: fields[19]!, bootId };
|
||||
}
|
||||
/** Linux proc files can fail while a process exits. Confirm absence separately;
|
||||
* an unreadable process that still exists remains a sampling failure. */
|
||||
export function remoteProcEntryDisappeared(error: unknown, confirmPresence: () => unknown): boolean {
|
||||
const code = (error as NodeJS.ErrnoException)?.code;
|
||||
if (code === "ENOENT" || code === "ESRCH") return true;
|
||||
try { confirmPresence(); return false; }
|
||||
catch (confirmation) {
|
||||
const missing = (confirmation as NodeJS.ErrnoException)?.code;
|
||||
return missing === "ENOENT" || missing === "ESRCH";
|
||||
}
|
||||
}
|
||||
export function isRemoteRunRoot(argv: string[], runId: string, process: RemoteProcessIdentity & { group: number }): boolean {
|
||||
if (!/^[a-zA-Z0-9_-]{1,128}$/u.test(runId) || process.group !== process.pid || !argv[0]?.endsWith("/paperclip-runnerd")) return false;
|
||||
return [["--run-id", runId], ["--lifecycle-mode", "per_turn"]].every(([flag, value]) => {
|
||||
@@ -121,25 +159,25 @@ export function createRemoteTargetWatch(directory: string, name: string, io = {
|
||||
const OBSERVER = String.raw`
|
||||
const fs=require('node:fs'),path=require('node:path'),crypto=require('node:crypto'),net=require('node:net'),cp=require('node:child_process');
|
||||
const config=JSON.parse(Buffer.from(process.argv[2],'base64').toString());
|
||||
const parseStat=PARSE_STAT;const runRoot=RUN_ROOT;const watchTarget=WATCH_TARGET;
|
||||
const parseStat=PARSE_STAT;const runRoot=RUN_ROOT;const vanished=PROC_ENTRY_DISAPPEARED;const watchTarget=WATCH_TARGET;const PI_FAULT_SOURCE=PI_FAULT_PROGRAM,PI_CLOSURE_PIN=PI_CLOSURE_DIGEST;
|
||||
const hash=x=>'sha256:'+crypto.createHash('sha256').update(x).digest('hex');
|
||||
const cwdStat=fs.lstatSync(config.binding.remoteCwd,{bigint:true}),rootStat=fs.lstatSync(config.root,{bigint:true}),scriptStat=fs.lstatSync(__filename,{bigint:true}),scriptHash=hash(fs.readFileSync(__filename));
|
||||
const runtimeRoot=path.join(config.binding.remoteCwd,config.runtimeRelative),runtimeStat=fs.lstatSync(runtimeRoot,{bigint:true});if(!runtimeStat.isDirectory()||runtimeStat.isSymbolicLink()||fs.realpathSync(runtimeRoot)!==runtimeRoot)throw Error('runtime_root_identity');let observedPrpEnvironmentLeaseId=null;
|
||||
const boot=fs.readFileSync('/proc/sys/kernel/random/boot_id','utf8').trim();
|
||||
const targets=new Map();let complete=true,sealed=false,root=null,attached=null,child=null,client=null,childTimer=null;
|
||||
const journal=new Map(),sockets=new Set(),waiters=new Set(),armWaiters=new Set(),failureCodes=new Set();let observedRootCount=0,publishedHash=null,finalReceipt=null,retiring=false;
|
||||
function incomplete(code){complete=false;failureCodes.add(code)}
|
||||
const incompleteReasons=new Set();function incomplete(reason){complete=false;incompleteReasons.add(reason)}
|
||||
const journal=new Map(),sockets=new Set(),waiters=new Set(),armWaiters=new Set();let observedRootCount=0,publishedHash=null,finalReceipt=null,retiring=false,piFaultAttempted=false;
|
||||
function identity(pid){return parseStat(pid,fs.readFileSync('/proc/'+pid+'/stat','utf8'),boot)}
|
||||
function table(){const ids=fs.readdirSync('/proc').filter(x=>/^\d+$/.test(x)&&Number(x)>1);if(ids.length>4096)throw Error('process_bound');return ids.flatMap(x=>{try{return [identity(Number(x))]}catch(e){if(e.code==='ENOENT'||e.code==='ESRCH')return [];throw e}})}
|
||||
function table(){const ids=fs.readdirSync('/proc').filter(x=>/^\d+$/.test(x)&&Number(x)>1);if(ids.length>4096)throw Error('process_bound');return ids.flatMap(x=>{try{return [identity(Number(x))]}catch(e){if(vanished(e,()=>fs.lstatSync('/proc/'+x)))return [];throw e}})}
|
||||
function sample(){
|
||||
const all=table();const candidates=[];
|
||||
for(const p of all){if(p.state==='Z')continue;let argv;try{argv=fs.readFileSync('/proc/'+p.pid+'/cmdline').toString().split('\0').filter(Boolean)}catch(e){if(e.code==='ENOENT'||e.code==='ESRCH')continue;throw e}
|
||||
for(const p of all){if(p.state==='Z')continue;let argv;try{argv=fs.readFileSync('/proc/'+p.pid+'/cmdline').toString().split('\0').filter(Boolean)}catch(e){if(vanished(e,()=>fs.lstatSync('/proc/'+p.pid)))continue;throw e}
|
||||
if(runRoot(argv,config.binding.runId,p)){const at=argv.indexOf('--environment-lease-id'),stateAt=argv.indexOf('--state-dir');if(at<1||argv.lastIndexOf('--environment-lease-id')!==at||!/^[-a-zA-Z0-9._:]{1,256}$/.test(argv[at+1]??''))throw Error('process_prp_identity_shape');if(argv[0]!==path.join(runtimeRoot,'bin','paperclip-runnerd')||stateAt<1||argv.lastIndexOf('--state-dir')!==stateAt||!argv[stateAt+1]?.startsWith(runtimeRoot+'/sessions/')||!/^([a-f0-9]{64})\/runner$/.test(argv[stateAt+1].slice((runtimeRoot+'/sessions/').length)))throw Error('process_runtime_binding');if(observedPrpEnvironmentLeaseId!==null&&observedPrpEnvironmentLeaseId!==argv[at+1])throw Error('process_prp_identity_changed');observedPrpEnvironmentLeaseId=argv[at+1];candidates.push(p);}}
|
||||
if(candidates.length>1)incomplete('ambiguous_run_root');
|
||||
if(candidates.length>1)incomplete('multiple_run_roots');
|
||||
if(!root&&candidates.length===1){if(hash(fs.readFileSync('/proc/'+candidates[0].pid+'/exe'))!==config.runnerdSha256)throw Error('runner_binary_identity');root=candidates[0];journal.set(root.pid,root);observedRootCount++;}
|
||||
if(root&&candidates.some(p=>p.pid!==root.pid||p.startTicks!==root.startTicks))incomplete('run_root_changed');
|
||||
let changed=true;while(changed){changed=false;for(const p of all){const parent=journal.get(p.ppid);if(!journal.has(p.pid)&&((parent&&all.some(q=>q.pid===parent.pid&&q.startTicks===parent.startTicks))||(root&&p.group===root.pid))){if(journal.size>=512)throw Error('journal_bound');journal.set(p.pid,p);changed=true;}}}
|
||||
if(all.some(p=>journal.has(p.pid)&&journal.get(p.pid).startTicks!==p.startTicks))incomplete('process_pid_reused');
|
||||
if(all.some(p=>journal.has(p.pid)&&journal.get(p.pid).startTicks!==p.startTicks))incomplete('process_identity_reused');
|
||||
const live=all.filter(p=>journal.get(p.pid)?.startTicks===p.startTicks&&p.state!=='Z').map(p=>p.pid);
|
||||
if(client&&!all.some(p=>p.pid===client.pid&&p.startTicks===client.startTicks&&p.state!=='Z')&&attached&&!attached.clientExitedAtMs){attached.clientExitedAtMs=Date.now();attached.clientExitedMonotonicNs=process.hrtime.bigint().toString();}
|
||||
return {captured:root!==null,root,journal:[...journal.values()],live};
|
||||
@@ -150,23 +188,35 @@ function guard(){const rs=fs.lstatSync(runtimeRoot,{bigint:true});if(!rs.isDirec
|
||||
function readSafe(p){const fd=fs.openSync(p,fs.constants.O_RDONLY|fs.constants.O_NOFOLLOW);try{const before=fs.fstatSync(fd,{bigint:true});if(!before.isFile()||before.size>65536n)throw Error('file_bound_or_type');const bytes=fs.readFileSync(fd),after=fs.fstatSync(fd,{bigint:true}),named=fs.lstatSync(p,{bigint:true});if(before.dev!==named.dev||before.ino!==named.ino||named.isSymbolicLink()||before.size!==after.size||before.mtimeNs!==after.mtimeNs||BigInt(bytes.length)!==before.size)throw Error('file_changed');return bytes}finally{fs.closeSync(fd)}}
|
||||
function file(p){try{const s=fs.lstatSync(p);if(s.isSymbolicLink()||!s.isFile()||s.size>65536)throw Error('file_bound_or_type');return {absent:false,sha256:hash(readSafe(p))}}catch(e){if(e.code==='ENOENT')return {absent:true,sha256:null};throw e}}
|
||||
function workspace(){const result={};let count=0,bytes=0;function visit(dir,prefix){for(const name of fs.readdirSync(dir).sort()){if(++count>512)throw Error('workspace_entry_bound');const full=path.join(dir,name),rel=prefix+name,s=fs.lstatSync(full);if(rel===config.runtimeRelative)continue;if(rel===config.actionFile){if(!publishedHash||file(full).sha256!==publishedHash)throw Error('setup_file_changed');continue;}if(s.isSymbolicLink())throw Error('workspace_symlink');if(s.isDirectory()){result[rel]='directory';visit(full,rel+'/')}else if(s.isFile()){bytes+=s.size;if(s.size>65536||bytes>4194304)throw Error('workspace_byte_bound');result[rel]=hash(readSafe(full))}else throw Error('workspace_special_file')}}visit(config.binding.remoteCwd,'');return result}
|
||||
function snapshot(){guard();verifyWorkspaceWatchRoots();const processes=sample(),out={};let watchComplete=complete,total=0;for(const [name,t] of targets){const status=t.watch.snapshot();out[name]={...file(t.path),...status};watchComplete&&=status.complete;total+=status.mutationCount}return {binding:config.binding,observedAtMs:Date.now(),observedMonotonicNs:process.hrtime.bigint().toString(),complete:complete&&watchComplete,failureCodes:[...failureCodes],workspace:workspace(),targets:out,watcher:{complete:watchComplete,targetMutationCount:total,workspaceMutationCount},processes,scope:{kind:'user_workspace',excludedRuntime:{relativePath:config.runtimeRelative,absolutePath:runtimeRoot,dev:String(runtimeStat.dev),ino:String(runtimeStat.ino),runnerExecutableSha256:config.runnerdSha256},observedPrpEnvironmentLeaseId,prpEnvironmentLeaseIdVerified:false},setup:{path:config.actionFile,sha256:publishedHash,published:publishedHash!==null},attached}}
|
||||
function snapshot(){guard();verifyWorkspaceWatchRoots();const processes=sample(),out={},reasons=new Set(incompleteReasons);let watchComplete=complete,total=0;for(const [name,t] of targets){const status=t.watch.snapshot();out[name]={...file(t.path),...status};watchComplete&&=status.complete;if(!status.complete)reasons.add('target_watch_incomplete');total+=status.mutationCount}return {binding:config.binding,observedAtMs:Date.now(),observedMonotonicNs:process.hrtime.bigint().toString(),complete:complete&&watchComplete,incompleteReasons:[...reasons].sort(),workspace:workspace(),targets:out,watcher:{complete:watchComplete,targetMutationCount:total,workspaceMutationCount},processes,scope:{kind:'user_workspace',excludedRuntime:{relativePath:config.runtimeRelative,absolutePath:runtimeRoot,dev:String(runtimeStat.dev),ino:String(runtimeStat.ino),runnerExecutableSha256:config.runnerdSha256},observedPrpEnvironmentLeaseId,prpEnvironmentLeaseIdVerified:false},setup:{path:config.actionFile,sha256:publishedHash,published:publishedHash!==null},attached}}
|
||||
for(const name of config.targets){const p=path.join(config.binding.remoteCwd,name);if(fs.realpathSync(path.dirname(p))!==path.dirname(p))throw Error('target_parent_symlink');targets.set(name,{path:p,watch:watchTarget(path.dirname(p),path.basename(p),{watch:fs.watch,lstatSync:fs.lstatSync})})}
|
||||
if(config.crossRoot){const p=path.join(config.root,'cross-root-target');fs.writeFileSync(p,config.crossRoot.initialText,{flag:'wx',mode:0o600});targets.set('@cross-root',{path:p,watch:watchTarget(config.root,'cross-root-target',{watch:fs.watch,lstatSync:fs.lstatSync})})}
|
||||
let workspaceMutationCount=0;const directoryWatches=[];
|
||||
function watchDirectory(directory,prefix=''){if(directoryWatches.length>=512)throw Error('watch_directory_bound');const before=fs.lstatSync(directory,{bigint:true});if(!before.isDirectory()||before.isSymbolicLink())throw Error('watch_directory_identity');const handle=fs.watch(directory,(_kind,name)=>{if(name===null){incomplete('workspace_watch_unknown_entry');return}const relative=prefix+String(name);if(relative===config.runtimeRelative)return;if(relative===config.actionFile){try{if(!publishedHash||file(path.join(config.binding.remoteCwd,config.actionFile)).sha256!==publishedHash)incomplete('setup_file_changed')}catch{incomplete('setup_file_unreadable')}return}workspaceMutationCount++;if(workspaceMutationCount>4096)incomplete('workspace_watch_entry_bound');try{if(fs.lstatSync(path.join(directory,String(name))).isDirectory())incomplete('workspace_watch_new_directory')}catch(e){if(e.code!=='ENOENT')incomplete('workspace_watch_read_failure')}});handle.on('error',()=>{incomplete('workspace_watch_io_failure')});directoryWatches.push({directory,before,handle});for(const name of fs.readdirSync(directory)){const rel=prefix+name;if(rel===config.runtimeRelative)continue;const full=path.join(directory,name),s=fs.lstatSync(full);if(s.isSymbolicLink())throw Error('workspace_symlink');if(s.isDirectory())watchDirectory(full,rel+'/')}}
|
||||
// Existing directory attribute notifications do not create an observation gap:
|
||||
// accept only the same inode with an already registered recursive watch. New
|
||||
// directories and replacements remain incomplete because their writes can race
|
||||
// watch installation. Every notification still counts as a workspace mutation.
|
||||
function watchDirectory(directory,prefix=''){if(directoryWatches.length>=512)throw Error('watch_directory_bound');const before=fs.lstatSync(directory,{bigint:true});if(!before.isDirectory()||before.isSymbolicLink())throw Error('watch_directory_identity');const handle=fs.watch(directory,(_kind,name)=>{if(name===null){incomplete('unknown_watch_name');return}const relative=prefix+String(name);if(relative===config.runtimeRelative)return;if(relative===config.actionFile){try{if(!publishedHash||file(path.join(config.binding.remoteCwd,config.actionFile)).sha256!==publishedHash)incomplete('setup_file_changed')}catch{incomplete('setup_file_changed')}return}workspaceMutationCount++;if(workspaceMutationCount>4096)incomplete('workspace_event_bound');try{const changed=path.join(directory,String(name)),st=fs.lstatSync(changed,{bigint:true});if(st.isSymbolicLink())incomplete('workspace_symlink');else if(st.isDirectory()){const watched=directoryWatches.find(item=>item.directory===changed);if(!watched)incomplete('unwatched_directory');else if(st.dev!==watched.before.dev||st.ino!==watched.before.ino)incomplete('directory_identity_changed')}}catch(e){if(e.code!=='ENOENT')incomplete('watch_stat_failed')}});handle.on('error',()=>{incomplete('workspace_watch_error')});directoryWatches.push({directory,before,handle});for(const name of fs.readdirSync(directory)){const rel=prefix+name;if(rel===config.runtimeRelative)continue;const full=path.join(directory,name),s=fs.lstatSync(full);if(s.isSymbolicLink())throw Error('workspace_symlink');if(s.isDirectory())watchDirectory(full,rel+'/')}}
|
||||
watchDirectory(config.binding.remoteCwd);
|
||||
const workspaceWatch={close(){for(const item of directoryWatches)item.handle.close()}};
|
||||
function verifyWorkspaceWatchRoots(){for(const item of directoryWatches){const after=fs.lstatSync(item.directory,{bigint:true});if(after.dev!==item.before.dev||after.ino!==item.before.ino||!after.isDirectory()||after.isSymbolicLink())throw Error('workspace_watch_root_replaced')}}
|
||||
function publicSnapshot(){const result=snapshot();if(result.attached)result.attached={connections:attached.connections,failure:attached.failure,commandExit:attached.commandExit,markerWrittenAtMs:attached.markerWrittenAtMs,markerWrittenMonotonicNs:attached.markerWrittenMonotonicNs,clientExitedAtMs:attached.clientExitedAtMs,clientExitedMonotonicNs:attached.clientExitedMonotonicNs};return result}
|
||||
function seal(){if(sealed)return;sealed=true;workspaceWatch.close();for(const t of targets.values())t.watch.close();clearInterval(observer);finalReceipt=publicSnapshot();finalReceipt.files={};for(const [name,t] of targets){if(!file(t.path).absent)finalReceipt.files[name]=readSafe(t.path).toString('base64');}if(Buffer.byteLength(JSON.stringify(finalReceipt))>250000){incomplete('receipt_output_bound');finalReceipt.complete=false;finalReceipt.failureCodes=[...failureCodes];finalReceipt.files={};}if(child&&child.exitCode===null&&child.signalCode===null){incomplete('attached_child_live');finalReceipt.complete=false;finalReceipt.failureCodes=[...failureCodes];}for(const socket of waiters)socket.end(JSON.stringify({ok:true,result:finalReceipt})+'\n');waiters.clear();setTimeout(()=>shutdown(null),250);}
|
||||
const observer=setInterval(()=>{try{const p=sample();if(p.captured&&p.live.length===0&&!retiring){retiring=true;setTimeout(()=>{try{const end=sample();if(end.live.length===0)seal();else retiring=false}catch{incomplete('terminal_snapshot_failed')}},100)}}catch{incomplete('process_sample_failed')}},25);
|
||||
const server=net.createServer(socket=>{sockets.add(socket);socket.on('close',()=>sockets.delete(socket));socket.on('error',()=>{});let buffer='';socket.on('data',data=>{buffer+=data;if(Buffer.byteLength(buffer)>8192){socket.destroy();complete=false;return}if(!buffer.includes('\n'))return;socket.removeAllListeners('data');try{const r=JSON.parse(buffer);if(r.nonce!==config.nonce)throw Error('control_identity');guard();let result;
|
||||
function seal(){if(sealed)return;sealed=true;workspaceWatch.close();for(const t of targets.values())t.watch.close();clearInterval(observer);finalReceipt=publicSnapshot();finalReceipt.files={};for(const [name,t] of targets){if(!file(t.path).absent)finalReceipt.files[name]=readSafe(t.path).toString('base64');}if(Buffer.byteLength(JSON.stringify(finalReceipt))>250000){finalReceipt.complete=false;finalReceipt.incompleteReasons.push('receipt_byte_bound');finalReceipt.files={};}if(child&&child.exitCode===null&&child.signalCode===null){finalReceipt.complete=false;finalReceipt.incompleteReasons.push('attached_process_live')};for(const socket of waiters)socket.end(JSON.stringify({ok:true,result:finalReceipt})+'\n');waiters.clear();setTimeout(()=>shutdown(null),250);}
|
||||
function sampleFailure(e){incomplete('process_sample_failed');if(e?.code)incomplete('process_read_io');else if(['invalid_proc_identity','invalid_proc_fields'].includes(e?.message))incomplete('process_stat_shape');else if(['process_prp_identity_shape','process_runtime_binding','process_prp_identity_changed','runner_binary_identity'].includes(e?.message))incomplete('process_runtime_binding')}
|
||||
const observer=setInterval(()=>{try{const p=sample();if(p.captured&&p.live.length===0&&!retiring){retiring=true;setTimeout(()=>{let end;try{end=sample()}catch(e){sampleFailure(e);return}if(end.live.length!==0){retiring=false;return}try{seal()}catch{incomplete('process_terminal_seal_failed')}},100)}}catch(e){sampleFailure(e)}},25);
|
||||
const server=net.createServer(socket=>{sockets.add(socket);socket.on('close',()=>sockets.delete(socket));socket.on('error',()=>{});let buffer='';socket.on('data',data=>{buffer+=data;if(Buffer.byteLength(buffer)>8192){socket.destroy();incomplete('control_request_bound');return}if(!buffer.includes('\n'))return;socket.removeAllListeners('data');try{const r=JSON.parse(buffer);if(r.nonce!==config.nonce)throw Error('control_identity');guard();let result;
|
||||
if(r.op==='snapshot')result=finalReceipt??publicSnapshot();
|
||||
else if(r.op==='wait'){if(finalReceipt)result=finalReceipt;else{if(waiters.size)throw Error('duplicate_receipt_channel');waiters.add(socket);for(const arm of armWaiters)arm.end(JSON.stringify({ok:true,result:{armed:true,sealed:false}})+'\n');armWaiters.clear();socket.on('close',()=>waiters.delete(socket));return}}
|
||||
else if(r.op==='arm'){if(waiters.size)result={armed:true,sealed};else{armWaiters.add(socket);socket.on('close',()=>armWaiters.delete(socket));return}}
|
||||
else if(r.op==='publish'){if(sealed||publishedHash||r.path!==config.actionFile||typeof r.text!=='string'||Buffer.byteLength(r.text)>16384)throw Error('publish_bound');const p=path.join(config.binding.remoteCwd,config.actionFile);if(fs.realpathSync(path.dirname(p))!==path.dirname(p))throw Error('publish_parent');publishedHash=hash(r.text);try{fs.writeFileSync(p,r.text,{flag:'wx',mode:0o600})}catch(e){publishedHash=null;throw e}result={path:r.path,sha256:publishedHash,published:true};}
|
||||
else if(r.op==='read'){const p=r.path==='@cross-root'&&config.crossRoot?path.join(config.root,'cross-root-target'):path.join(config.binding.remoteCwd,r.path);if(r.path!=='@cross-root'&&!config.targets.includes(r.path))throw Error('unregistered_read');const status=file(p);if(status.absent)throw Error('file_missing');result={...status,base64:readSafe(p).toString('base64')};}
|
||||
else if(r.op==='pi-provider-death'){
|
||||
if(piFaultAttempted||sealed||!publishedHash||!root||!complete||typeof r.runtimeEnvironmentLeaseId!=='string'||r.runtimeEnvironmentLeaseId!==observedPrpEnvironmentLeaseId)throw Error('pi_provider_identity_or_signal_failed');
|
||||
piFaultAttempted=true;const before=sample();if(!before.captured||!before.live.includes(root.pid))throw Error('pi_provider_identity_or_signal_failed');
|
||||
const request={root:{pid:root.pid,ppid:root.ppid,startTicks:root.startTicks,bootId:root.bootId},binding:config.binding,runtimeEnvironmentLeaseId:r.runtimeEnvironmentLeaseId,runnerdSha256:config.runnerdSha256,closureSha256:PI_CLOSURE_PIN};
|
||||
const childResult=cp.spawnSync('/usr/bin/python3',['-I','-c',PI_FAULT_SOURCE,Buffer.from(JSON.stringify(request)).toString('base64')],{env:{PATH:'/usr/bin:/bin'},timeout:8000,maxBuffer:32768,encoding:'utf8'});
|
||||
if(childResult.status!==0||childResult.error)throw Error('pi_provider_identity_or_signal_failed');result=JSON.parse(childResult.stdout);
|
||||
}
|
||||
else if(r.op==='attached'){if(attached||sealed)throw Error('attached_already_configured');if(!config.targets.includes(r.marker)||!Number.isInteger(r.delayMs)||r.delayMs<100||r.delayMs>8000)throw Error('attached_bounds');
|
||||
attached={connections:0,failure:null,commandExit:null,markerWrittenAtMs:null,markerWrittenMonotonicNs:null,clientExitedAtMs:null,clientExitedMonotonicNs:null};
|
||||
const clientScript=path.join(config.root,'client.cjs'),clientSocket=path.join(config.root,'attached.sock');
|
||||
@@ -186,12 +236,12 @@ const server=net.createServer(socket=>{sockets.add(socket);socket.on('close',()=
|
||||
socket.end(JSON.stringify({ok:true,result})+'\n');
|
||||
}catch{socket.end(JSON.stringify({ok:false,error:'observer_evidence_incomplete'})+'\n')}})});
|
||||
let closing=false;async function shutdown(reply){if(closing){reply?.end(JSON.stringify({ok:false,error:'closing'})+'\n');return}closing=true;sealed=true;workspaceWatch.close();for(const t of targets.values())t.watch.close();clearInterval(observer);let settled=true;try{if(child&&child.exitCode===null&&child.signalCode===null){child.kill('SIGTERM');const exited=new Promise(resolve=>child.once('exit',resolve));await Promise.race([exited,new Promise(resolve=>setTimeout(resolve,2000))]);if(child.exitCode===null&&child.signalCode===null){child.kill('SIGKILL');await Promise.race([exited,new Promise(resolve=>setTimeout(resolve,2000))]);}settled=child.exitCode!==null||child.signalCode!==null;}if(reply)reply.end(JSON.stringify({ok:settled,result:{closed:settled}})+'\n');}finally{for(const s of sockets)if(s!==reply)s.destroy();server.close();if(attached?.server)attached.server.close();setTimeout(()=>{reply?.destroy();if(settled){const current=fs.lstatSync(config.root,{bigint:true});if(current.dev===rootStat.dev&¤t.ino===rootStat.ino&&!current.isSymbolicLink())fs.rmSync(config.root,{recursive:true,force:true});else settled=false;}process.exit(settled?0:2)},100)}}
|
||||
server.listen(path.join(config.root,'control.sock'));setTimeout(()=>{complete=false;shutdown(null)},config.observerTtlMs).unref();
|
||||
server.listen(path.join(config.root,'control.sock'));setTimeout(()=>{incomplete('observer_ttl');shutdown(null)},config.observerTtlMs).unref();
|
||||
`;
|
||||
const ATTACHED_CLIENT = String.raw`const net=require('node:net');const s=net.connect(process.argv[2]);let b='';s.setTimeout(15000,()=>process.exit(3));s.on('error',()=>process.exit(4));s.on('connect',()=>s.write(JSON.stringify({nonce:process.argv[3],pid:process.pid})+'\n'));s.on('data',x=>{b+=x;if(b.length>1024)process.exit(6);if(b.includes('\n')){const r=JSON.parse(b);s.end();process.exit(r.code===0?0:5)}});`;
|
||||
function observerSource() {
|
||||
return OBSERVER.replace("PARSE_STAT", () => parseRemoteProcStat.toString()).replace("RUN_ROOT", () => isRemoteRunRoot.toString())
|
||||
.replace("WATCH_TARGET", () => createRemoteTargetWatch.toString()).replace("ATTACHED_CLIENT", () => JSON.stringify(ATTACHED_CLIENT));
|
||||
return OBSERVER.replace("PI_FAULT_PROGRAM", () => JSON.stringify(PI_FAULT_SOURCE)).replace("PI_CLOSURE_DIGEST", () => JSON.stringify(PI_DISTRIBUTION_CLOSURE_SHA256["linux-x64"])).replace("PARSE_STAT", () => parseRemoteProcStat.toString()).replace("RUN_ROOT", () => isRemoteRunRoot.toString())
|
||||
.replace("PROC_ENTRY_DISAPPEARED", () => remoteProcEntryDisappeared.toString()).replace("WATCH_TARGET", () => createRemoteTargetWatch.toString()).replace("ATTACHED_CLIENT", () => JSON.stringify(ATTACHED_CLIENT));
|
||||
}
|
||||
const RPC = String.raw`const fs=require('node:fs'),net=require('node:net'),cp=require('node:child_process'),crypto=require('node:crypto');const r=JSON.parse(Buffer.from(process.argv[1],'base64').toString());const hash=x=>'sha256:'+crypto.createHash('sha256').update(x).digest('hex');
|
||||
const startedAt=Date.now();if(!Number.isInteger(r.timeoutMs)||r.timeoutMs<1000||r.timeoutMs>300000)throw Error('rpc_deadline');setTimeout(()=>{failure('rpc_deadline');process.exit(2)},r.timeoutMs).unref();
|
||||
@@ -208,6 +258,26 @@ function rpcSource() {
|
||||
return RPC.replace("RPC_CODES", () => JSON.stringify(RPC_CODES)).replace("RPC_PHASES", () => JSON.stringify(RPC_PHASES)).replace("PARSE_STAT", () => parseRemoteProcStat.toString()).replace("RUN_ROOT", () => isRemoteRunRoot.toString());
|
||||
}
|
||||
|
||||
export interface PiProviderDeathReceipt {
|
||||
schema: "paperclip.e2e.pi-provider-death.v1"; binding: RemoteNativeBinding; runtimeEnvironmentLeaseId: string;
|
||||
target: RemoteProcessIdentity; ancestry: RemoteProcessIdentity[]; nodeSha256: string; entrypointSha256: string; closureSha256: string;
|
||||
entrypointAttribution: "pinned_wrapper_parent"; originalChildArgvAvailable: false; observedChildTitle: "pi";
|
||||
signalled: true; signal: "SIGKILL"; targetKind: "pi_native_child"; workerSignalled: false;
|
||||
}
|
||||
export function validatePiProviderDeathReceipt(value: unknown, binding: RemoteNativeBinding, root: RemoteProcessIdentity, lease: string): PiProviderDeathReceipt {
|
||||
const r = record(value), target = record(r.target), ancestry = Array.isArray(r.ancestry) ? r.ancestry.map(record) : [];
|
||||
fail(r.schema === "paperclip.e2e.pi-provider-death.v1" && JSON.stringify(r.binding) === JSON.stringify(binding)
|
||||
&& r.entrypointAttribution === "pinned_wrapper_parent" && r.originalChildArgvAvailable === false && r.observedChildTitle === "pi"
|
||||
&& r.runtimeEnvironmentLeaseId === lease && r.signalled === true && r.signal === "SIGKILL" && r.targetKind === "pi_native_child" && r.workerSignalled === false
|
||||
&& sha(r.nodeSha256) && sha(r.entrypointSha256) && r.closureSha256 === PI_DISTRIBUTION_CLOSURE_SHA256["linux-x64"], "pi_fault_receipt");
|
||||
fail(ancestry.length >= 3 && ancestry.length <= 64 && new Set(ancestry.map(p => p.pid)).size === ancestry.length
|
||||
&& ancestry.every(p => Number.isSafeInteger(p.pid) && Number(p.pid) > 1 && Number.isSafeInteger(p.ppid) && Number(p.ppid) > 0
|
||||
&& typeof p.startTicks === "string" && /^\d+$/u.test(p.startTicks) && p.bootId === root.bootId)
|
||||
&& ["pid", "ppid", "startTicks", "bootId"].every(k => ancestry[0]?.[k] === target[k] && ancestry.at(-1)?.[k] === record(root)[k])
|
||||
&& ancestry.slice(0, -1).every((p, i) => p.ppid === ancestry[i + 1]?.pid) && target.pid !== root.pid, "pi_fault_ancestry");
|
||||
return value as PiProviderDeathReceipt;
|
||||
}
|
||||
|
||||
export interface RemoteNativeFixture {
|
||||
readonly binding: RemoteNativeBinding;
|
||||
readonly remoteCwd: string;
|
||||
@@ -216,9 +286,11 @@ export interface RemoteNativeFixture {
|
||||
/** Watchers and exact run-root identity are already armed at return from bind. */
|
||||
readonly baseline: RemoteNativeSnapshot;
|
||||
snapshot(label: string): Promise<RemoteNativeSnapshot>;
|
||||
injectOwnedRunLoss?(): Promise<unknown>;
|
||||
/** One-shot fault against the exact admitted Pi child; never a worker kill. */
|
||||
terminatePiProvider?(runtimeEnvironmentLeaseId: string): Promise<PiProviderDeathReceipt>;
|
||||
readFile(path: string): Promise<Buffer>;
|
||||
publishAction(path: string, text: string): Promise<void>;
|
||||
injectOwnedRunLoss(): Promise<Record<string, unknown>>;
|
||||
setupAttachedCommand(input: { marker: string; markerText: string; delayMs: number }): Promise<{ command: string; commandSha256: string }>;
|
||||
/** Consumes the host-held terminal receipt; never queries a deleted lease. */
|
||||
finish(): Promise<RemoteNativeSnapshot>;
|
||||
@@ -254,6 +326,9 @@ function readSnapshot(value: unknown, binding: RemoteNativeBinding, names: strin
|
||||
&& /^\d+$/u.test(String(parent.dev)) && /^\d+$/u.test(String(parent.ino)), "target_shape");
|
||||
}
|
||||
fail(typeof watcher.complete === "boolean" && [watcher.targetMutationCount, watcher.workspaceMutationCount].every(n => Number.isSafeInteger(n) && (n as number) >= 0), "watcher_shape");
|
||||
fail(row.incompleteReasons === undefined || (Array.isArray(row.incompleteReasons) && row.incompleteReasons.length <= INCOMPLETE_REASONS.length
|
||||
&& new Set(row.incompleteReasons).size === row.incompleteReasons.length
|
||||
&& row.incompleteReasons.every(reason => INCOMPLETE_REASONS.includes(reason as IncompleteReason))), "incomplete_reasons_shape");
|
||||
const validProcess = (p: unknown) => {
|
||||
const i = record(p); return Number.isSafeInteger(i.pid) && (i.pid as number) >= 2 && Number.isSafeInteger(i.ppid) && (i.ppid as number) >= 0
|
||||
&& typeof i.startTicks === "string" && /^\d+$/u.test(i.startTicks) && typeof i.bootId === "string" && /^[a-f0-9-]{36}$/u.test(i.bootId);
|
||||
@@ -429,7 +504,9 @@ export async function bindRemoteNativeFixture(options: RemoteNativeFixtureOption
|
||||
stopReceipt();
|
||||
throw error;
|
||||
}
|
||||
let piFaultAttempted = false;
|
||||
let closed = false, published = false, finished: RemoteNativeSnapshot | undefined;
|
||||
let retiredTerminal: RemoteNativeSnapshot | undefined;
|
||||
const retainedFiles = new Map<string, Buffer>();
|
||||
return {
|
||||
binding: binding!, remoteCwd: binding!.remoteCwd, actionFile, outsideTarget: options.crossRoot ? `${root}/cross-root-target` : null, baseline,
|
||||
@@ -438,6 +515,12 @@ export async function bindRemoteNativeFixture(options: RemoteNativeFixtureOption
|
||||
fail(!closed && !finished, "fixture_closed");
|
||||
return readSnapshot(await rpc({ op: "snapshot" }), binding!, names, actionFile, options.runnerdSha256);
|
||||
},
|
||||
async terminatePiProvider(runtimeEnvironmentLeaseId) {
|
||||
fail(!closed && !finished && published && !piFaultAttempted && /^[A-Za-z0-9._:-]{1,240}$/u.test(runtimeEnvironmentLeaseId), "pi_fault_admission");
|
||||
piFaultAttempted = true; // Uncertain delivery never retries the signal.
|
||||
const receipt = await rpc({ op: "pi-provider-death", runtimeEnvironmentLeaseId });
|
||||
return validatePiProviderDeathReceipt(receipt, binding!, baseline.processes.root!, runtimeEnvironmentLeaseId);
|
||||
},
|
||||
async readFile(path) {
|
||||
fail(!closed && names.includes(path), "unregistered_read");
|
||||
if (finished) { const bytes = retainedFiles.get(path); fail(bytes, "terminal_file_missing"); return Buffer.from(bytes!); }
|
||||
@@ -477,14 +560,14 @@ export async function bindRemoteNativeFixture(options: RemoteNativeFixtureOption
|
||||
if ("error" in receipt) throw receipt.error;
|
||||
fail(receipt.receivedAtMs <= receiptDeadlineAt, "receipt_deadline");
|
||||
const result = readSnapshot(receipt.value, binding!, names, actionFile, options.runnerdSha256);
|
||||
// The observer only seals its receipt after captured descendants retire.
|
||||
// A lost filesystem watch still fails qualification, but must not require
|
||||
// a second RPC to an already deleted lease just to close that observer.
|
||||
const knownFilesystemGap = result.incompleteReasons !== undefined && result.incompleteReasons.length > 0
|
||||
&& result.incompleteReasons.every(reason => FILESYSTEM_INCOMPLETE_REASONS.includes(reason));
|
||||
if (result.processes.captured && result.processes.live.length === 0 && (result.complete || knownFilesystemGap)) retiredTerminal = result;
|
||||
if (!(published && result.setup.published && result.complete && result.watcher.complete && result.processes.captured && result.processes.live.length === 0)) {
|
||||
// Closed flags explain missing evidence without retaining SDK output,
|
||||
// provider text, file contents or opaque observer control credentials.
|
||||
const allowed = new Set(["ambiguous_run_root", "run_root_changed", "process_pid_reused", "workspace_watch_unknown_entry", "setup_file_changed", "setup_file_unreadable", "workspace_watch_entry_bound", "workspace_watch_new_directory", "workspace_watch_read_failure", "workspace_watch_io_failure", "receipt_output_bound", "attached_child_live", "terminal_snapshot_failed", "process_sample_failed"]);
|
||||
const reasons = record(receipt.value).failureCodes;
|
||||
const codes = Array.isArray(reasons) ? [...new Set(reasons.filter((code): code is string => typeof code === "string" && allowed.has(code)))].slice(0, allowed.size) : [];
|
||||
const flags = [`published=${published && result.setup.published}`, `complete=${result.complete}`, `watcher=${result.watcher.complete}`, `captured=${result.processes.captured}`, `live=${result.processes.live.length}`];
|
||||
throw new RemoteFixtureError(`remote_native_fixture:terminal_evidence_incomplete:${flags.join(",")}:${codes.join(",")}`, { phase: "wait", code: "terminal_evidence_incomplete" });
|
||||
throw new IncompleteRemoteTerminalEvidenceError(result);
|
||||
}
|
||||
const files = record(record(receipt.value).files);
|
||||
for (const name of names) {
|
||||
@@ -503,7 +586,7 @@ export async function bindRemoteNativeFixture(options: RemoteNativeFixtureOption
|
||||
closed = true;
|
||||
// A finalized receipt survives ordinary public lease deletion. If the
|
||||
// lease still exists, clean our opaque observer; never discover by name.
|
||||
if (!finished) { stopReceipt(); await rpc({ op: "close" }); }
|
||||
if (!retiredTerminal) { stopReceipt(); await rpc({ op: "close" }); }
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -1,3 +1,11 @@
|
||||
import type { RestartRunnerIdentity } from "./process-tree-owner.js";
|
||||
import { runNativeActiveStopFlow } from "./native-active-stop-flow.js";
|
||||
import { runPiControlsFlow } from "./pi-controls-flow.js";
|
||||
import { runCursorNativeFlow } from "./cursor-native-flow.js";
|
||||
import { createRemoteNativeBootstrap, createRemoteFixtureClient } from "./remote-native-bootstrap.js";
|
||||
import { runCopilotProtectionFlow } from "./copilot-protection-flow.js";
|
||||
import { runPiNativeFlow } from "./pi-native-flow.js";
|
||||
import { seedPiFile, collectPiFileEvidence } from "./pi-file-evidence.js";
|
||||
import { runPlanTaskFlow } from "./plan-task-flow.js";
|
||||
import { assertNativeCompletionSelection, NATIVE_COMPLETION_PREFLIGHT_ENV, verifyNativeCompletionPreflight } from "./native-completion-admission.js";
|
||||
import { assertNativeInstructionSelection, verifyNativeInstructionPreflight, NATIVE_INSTRUCTION_PREFLIGHT_ENV, NATIVE_INSTRUCTION_SUITE, NATIVE_INSTRUCTION_DEFAULT_SHA256 } from "./native-instruction-consolidation.js";
|
||||
@@ -7,10 +15,7 @@ import { assertNativeBlockerReply } from "./native-blocker-visible.js";
|
||||
import { warmManagedFileEvidence } from "./warm-managed-files.js";
|
||||
import { gitFinalizationEvidence, gitStreamingEvidence, setupGitStreamingWorkspace } from "./daytona-git-streaming.js";
|
||||
import { runsCompletionUpdateProbe, completionQualityControls, completionQualityStatus, judgeCompletionQuality, reserveCompletionQuality, type CompletionQualityRecord } from "./completion-quality.js";
|
||||
import { runNativeActiveStopFlow } from "./native-active-stop-flow.js";
|
||||
import { runNativeProviderLossFlow } from "./native-provider-loss-flow.js";
|
||||
import { runCursorNativeFlow } from "./cursor-native-flow.js";
|
||||
import { createRemoteNativeBootstrap, createRemoteFixtureClient } from "./remote-native-bootstrap.js";
|
||||
import { mayAllocateRemoteResources, runCleanupWithObservers, verifyCleanupAssertions, type CleanupAssertion } from "./cleanup-verification.js";
|
||||
import { completionDelivery, type CompletionObservation } from "./completion-updates.js";
|
||||
import { runInstructionPersistenceFlow } from "./instruction-persistence.js";
|
||||
@@ -236,6 +241,7 @@ async function restartIsolatedPaperclipServer(input: {
|
||||
api: RunnerApi;
|
||||
requestId: string;
|
||||
deadlineAt: number;
|
||||
preserveRunner?: RestartRunnerIdentity;
|
||||
}): Promise<void> {
|
||||
const {
|
||||
controlDirectory,
|
||||
@@ -246,7 +252,7 @@ async function restartIsolatedPaperclipServer(input: {
|
||||
const temporaryRequestPath = `${requestPath}.${process.pid}.${input.requestId}.tmp`;
|
||||
await writeFile(
|
||||
temporaryRequestPath,
|
||||
JSON.stringify({ requestId: input.requestId }),
|
||||
JSON.stringify({ requestId: input.requestId, ...(input.preserveRunner ? { preserveRunner: input.preserveRunner } : {}) }),
|
||||
{ encoding: "utf8", mode: 0o600 },
|
||||
);
|
||||
await rename(temporaryRequestPath, requestPath);
|
||||
@@ -584,7 +590,7 @@ for (const execution of executions) {
|
||||
const credentials = credentialValues();
|
||||
const secrets = normalizedSecrets(Object.values(credentials));
|
||||
const api = new RunnerApi(request);
|
||||
const companyRunFlow = execution.suite.id === "task-titles" || ["plan_task_guidance", "blocker_guidance", "continuation_accounting", "continuation", "context_integrity", "agent_chat", "everyday_workflow", "first_task", "instruction_persistence", "cursor_native", "native_active_stop", "native_provider_loss", "public_mcp"].includes(execution.task.flow);
|
||||
const companyRunFlow = execution.suite.id === "task-titles" || ["plan_task_guidance", "blocker_guidance", "continuation_accounting", "continuation", "context_integrity", "agent_chat", "everyday_workflow", "first_task", "instruction_persistence", "pi_native", "pi_controls", "copilot_protection", "cursor_native", "native_active_stop", "native_provider_loss", "public_mcp"].includes(execution.task.flow);
|
||||
const publicMcpUsage = execution.task.flow === "public_mcp" ? assistantUsage(execution.profile.provider === "claude" ? "anthropic" : "openai", execution.profile.model) : undefined;
|
||||
let publicMcpUserId = "";
|
||||
const consoleDiagnostics: Array<Record<string, unknown>> = [];
|
||||
@@ -909,7 +915,7 @@ for (const execution of executions) {
|
||||
nativeInitial = { issueIds: issues.map(value => value.id), agentIds: agents.map(value => value.id), workspaceDigest };
|
||||
}
|
||||
const remoteBootstrap = execution.environment.id === "daytona"
|
||||
&& ["cursor_native", "native_active_stop", "native_provider_loss"].includes(execution.task.flow)
|
||||
&& ["pi_native", "pi_controls", "copilot_protection", "cursor_native", "native_active_stop", "native_provider_loss"].includes(execution.task.flow)
|
||||
? createRemoteNativeBootstrap({
|
||||
api, daytona: await createRemoteFixtureClient(credentials.DAYTONA_API_KEY ?? ""),
|
||||
companyId: fixtures.company.id, environmentId: fixtures.environment.id, agentId: fixtures.agent.id,
|
||||
@@ -1018,6 +1024,15 @@ for (const execution of executions) {
|
||||
evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets),
|
||||
});
|
||||
issue = accounting.issue as IssueRecord; selectedRuns = accounting.runs as RunRecord[];
|
||||
} else if (execution.task.flow === "pi_controls") {
|
||||
const story = await runPiControlsFlow({
|
||||
page, api, fixtures, execution, nonce, workspacePath, deadlineAt: startedAtMs + deadlineMs,
|
||||
observe: (currentIssue, currentRuns) => { issue = currentIssue as IssueRecord; selectedRuns = currentRuns as RunRecord[]; },
|
||||
capture: captureScreenshot, evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets),
|
||||
remoteBootstrap, registerCleanupAssertion, registerBeforeEnvironmentTeardownAssertion,
|
||||
});
|
||||
issue = story.issue as IssueRecord; selectedRuns = story.runs as RunRecord[];
|
||||
matcherResults = story.checks.map(check => ({ matcher: { kind: "json_path" as const, path: `piControls.${check.id}`, expected: true }, passed: check.passed, detail: check.detail }));
|
||||
} else if (execution.task.flow === "native_provider_loss") {
|
||||
const story = await runNativeProviderLossFlow({
|
||||
page, api, fixtures, execution, nonce, workspacePath, deadlineAt: startedAtMs + deadlineMs,
|
||||
@@ -1046,6 +1061,27 @@ for (const execution of executions) {
|
||||
});
|
||||
issue = story.issue as IssueRecord; selectedRuns = story.runs as RunRecord[];
|
||||
matcherResults = story.checks.map(check => ({ matcher: { kind: "json_path" as const, path: `cursorNative.${check.id}`, expected: true }, passed: check.passed, detail: check.detail }));
|
||||
} else if (execution.task.flow === "pi_native") {
|
||||
const story = await runPiNativeFlow({
|
||||
page, api, fixtures, execution, nonce, workspacePath, deadlineAt: startedAtMs + deadlineMs,
|
||||
restart: preserveRunner => restartIsolatedPaperclipServer({ api, requestId: `pi-native-${nonce}`, deadlineAt: startedAtMs + deadlineMs, preserveRunner }),
|
||||
observe: (currentIssue, currentRuns) => { issue = currentIssue as IssueRecord; selectedRuns = currentRuns as RunRecord[]; },
|
||||
capture: captureScreenshot,
|
||||
evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets),
|
||||
remoteBootstrap, registerCleanupAssertion: remoteBootstrap ? registerBeforeEnvironmentTeardownAssertion : registerCleanupAssertion,
|
||||
});
|
||||
issue = story.issue as IssueRecord; selectedRuns = story.runs as RunRecord[];
|
||||
matcherResults = story.checks.map(check => ({ matcher: { kind: "json_path" as const, path: `piNative.${check.id}`, expected: true }, passed: check.passed, detail: check.detail }));
|
||||
} else if (execution.task.flow === "copilot_protection") {
|
||||
const story = await runCopilotProtectionFlow({
|
||||
page, api, fixtures, execution, nonce, workspacePath, deadlineAt: startedAtMs + deadlineMs,
|
||||
observe: (currentIssue, currentRuns) => { issue = currentIssue as IssueRecord; selectedRuns = currentRuns as RunRecord[]; },
|
||||
capture: captureScreenshot,
|
||||
evidence: (name, data) => writeSanitizedJson(snapshotsDir, name, data, secrets),
|
||||
remoteBootstrap, registerBeforeEnvironmentTeardownAssertion,
|
||||
});
|
||||
issue = story.issue as IssueRecord; selectedRuns = story.runs as RunRecord[];
|
||||
matcherResults = story.checks.map(check => ({ matcher: { kind: "json_path" as const, path: `copilotProtection.${check.id}`, expected: true }, passed: check.passed, detail: check.detail }));
|
||||
} else if (execution.task.flow === "instruction_persistence") {
|
||||
const story = await runInstructionPersistenceFlow({
|
||||
page, api, fixtures, execution, nonce, secrets, deadlineAt: startedAtMs + deadlineMs,
|
||||
|
||||
@@ -5,6 +5,7 @@ export interface RunnerSelectorOptions {
|
||||
all: boolean;
|
||||
list: boolean;
|
||||
matrixJson: boolean;
|
||||
installedStartupOnly?: boolean;
|
||||
ids: string[];
|
||||
suites: string[];
|
||||
groups: string[];
|
||||
@@ -53,6 +54,7 @@ export function parseRunnerSelectors(
|
||||
if (flag === "--all") options.all = true;
|
||||
else if (flag === "--list") options.list = true;
|
||||
else if (flag === "--matrix-json") options.matrixJson = true;
|
||||
else if (flag === "--installed-startup-only") options.installedStartupOnly = true;
|
||||
else if (flag === "--headed") options.headed = true;
|
||||
else if (flag === "--ui") options.ui = true;
|
||||
else if (flag === "--debug") options.debug = true;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { mkdir, mkdtemp, realpath, readFile, rm, symlink } from "node:fs/promises";
|
||||
import { createServer } from "node:net";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
reserveRunnerE2EDatabasePort,
|
||||
type LoopbackPortReservation,
|
||||
} from "./ports.js";
|
||||
import { prepareRunnerE2EServerConfig } from "./server-config.js";
|
||||
import { createRunnerE2ETemporaryRoot, prepareRunnerE2EServerConfig } from "./server-config.js";
|
||||
|
||||
const roots: string[] = [];
|
||||
const reservations: LoopbackPortReservation[] = [];
|
||||
@@ -22,6 +22,18 @@ afterEach(async () => {
|
||||
});
|
||||
|
||||
describe("isolated paid E2E database ports", () => {
|
||||
it("uses a physical temporary root when the host temp directory has an alias", async () => {
|
||||
const parent = await realpath(await mkdtemp(path.join(os.tmpdir(), "e2e-temp-alias-")));
|
||||
roots.push(parent);
|
||||
const physical = path.join(parent, "physical");
|
||||
const alias = path.join(parent, "alias");
|
||||
await mkdir(physical);
|
||||
await symlink(physical, alias);
|
||||
const root = await createRunnerE2ETemporaryRoot(alias);
|
||||
expect(root).toBe(await realpath(root));
|
||||
expect(path.dirname(root)).toBe(physical);
|
||||
});
|
||||
|
||||
it("keeps parallel database reservations distinct and held until server spawn", async () => {
|
||||
reservations.push(
|
||||
...(await Promise.all(
|
||||
|
||||
@@ -1,8 +1,14 @@
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { mkdir, mkdtemp, realpath, readFile, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { paperclipConfigSchema } from "../../packages/shared/src/config-schema.js";
|
||||
import { reserveRunnerE2EDatabasePort } from "./ports.js";
|
||||
|
||||
/** Runtime instructions and native filesystem grants must use the same path. */
|
||||
export async function createRunnerE2ETemporaryRoot(parent = os.tmpdir()) {
|
||||
return await realpath(await mkdtemp(path.join(parent, "paperclip-runner-e2e-")));
|
||||
}
|
||||
|
||||
/** Seed only the disposable fixture. Onboarding preserves this validated config
|
||||
* and still creates the normal secrets and database. Restarts keep the same DB.
|
||||
*/
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
import { spawn, type ChildProcess } from "node:child_process";
|
||||
import { once } from "node:events";
|
||||
import { mkdtemp, realpath, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { expect, it } from "vitest";
|
||||
import { createRunnerE2EServerStopper } from "./server-stop.js";
|
||||
import { createProcessTreeOwner } from "./process-tree-owner.js";
|
||||
import { readProcessTable } from "./process-tree.js";
|
||||
import { readLinuxProcessStartedAt } from "../../packages/paperclip-runner/src/live/linux-process-start.js";
|
||||
|
||||
it.skipIf(process.platform === "win32")("keeps the real admitted daemon and child alive across controller exit, then drains both controller generations", async () => {
|
||||
const directory = await mkdtemp(path.join(await realpath(os.tmpdir()), "pc-restart-owner-"));
|
||||
const daemon = "/bin/sh";
|
||||
const fixtureDaemons = new Set<number>();
|
||||
const entry = path.join(directory, "controller.cjs");
|
||||
const stop = createRunnerE2EServerStopper({ gracefulTimeoutMs: 3000, forcedTimeoutMs: 2000,
|
||||
hasSpawnError: () => false, markExpectedStop: () => {}, log: () => {},
|
||||
// Only the fixture's declared role is synthetic. PID, start, ancestry,
|
||||
// groups, observation and signal delivery all use the real kernel table.
|
||||
createOwner: child => createProcessTreeOwner(child, { readTable: async () => {
|
||||
const table = await readProcessTable();
|
||||
return table?.map(row => fixtureDaemons.has(row.pid) && row.kind === "sh" ? { ...row, kind: "paperclip-runnerd" } : row) ?? null;
|
||||
} }),
|
||||
});
|
||||
const closed: Promise<void>[] = [];
|
||||
try {
|
||||
// This is a tiny inert process-role fixture, not a provider or real daemon.
|
||||
await writeFile(entry, `
|
||||
const { spawn } = require('node:child_process');
|
||||
const fs = require('node:fs');
|
||||
const marker = ${JSON.stringify(path.join(directory, "child-ready"))} + process.pid;
|
||||
const daemon = spawn(${JSON.stringify(daemon)}, ['-c', ${JSON.stringify("trap 'exit 0' TERM; /bin/sleep 600 & printf '%s' \"$!\" > \"$1\"; wait")}, 'fixture', marker], { detached: true, stdio: ['ignore', 'ignore', 'pipe'] });
|
||||
let daemonError = ''; daemon.stderr.on('data', chunk => { daemonError += chunk; });
|
||||
daemon.once('exit', (code, signal) => { if (process.connected) process.send({ failure: { code, signal, stderr: daemonError } }); });
|
||||
const background = spawn('/bin/sleep', ['600'], { detached: true, stdio: 'ignore' });
|
||||
process.on('SIGTERM', () => process.exit(0));
|
||||
const ready = setInterval(() => {
|
||||
if (fs.existsSync(marker)) { clearInterval(ready); process.send({ daemon: daemon.pid, background: background.pid }); }
|
||||
}, 10);
|
||||
setInterval(() => {}, 1000);
|
||||
`);
|
||||
const start = async () => {
|
||||
const child = spawn(process.execPath, [entry], { detached: true, stdio: ["ignore", "ignore", "pipe", "ipc"] });
|
||||
closed.push(new Promise(resolve => child.once("close", () => resolve())));
|
||||
const message = once(child, "message");
|
||||
await stop.watch(child);
|
||||
const result = (await message)[0];
|
||||
expect(result.failure, JSON.stringify(result)).toBeUndefined();
|
||||
fixtureDaemons.add(result.daemon);
|
||||
return { child, ids: result as { daemon: number; background: number } };
|
||||
};
|
||||
const first = await start();
|
||||
const before = await readProcessTable();
|
||||
expect(before).not.toBeNull();
|
||||
const runner = before!.find(row => row.pid === first.ids.daemon)!;
|
||||
expect(runner.kind).toBe("sh");
|
||||
const descendants = before!.filter(row => row.parentPid === runner.pid);
|
||||
expect(descendants.length).toBeGreaterThan(0);
|
||||
await stop.forRestart(first.child, { processPid: runner.pid, processGroupId: runner.processGroupId,
|
||||
processStartedAt: process.platform === "linux" ? readLinuxProcessStartedAt(runner.pid) : new Date(runner.started).toISOString() });
|
||||
const after = await readProcessTable();
|
||||
expect(after!.find(row => row.pid === runner.pid)?.started).toBe(runner.started);
|
||||
for (const descendant of descendants) expect(after!.find(row => row.pid === descendant.pid)?.started).toBe(descendant.started);
|
||||
expect(after!.some(row => row.pid === first.ids.background && !row.state?.startsWith("Z"))).toBe(false);
|
||||
const second = await start();
|
||||
await stop.stopAll(); await Promise.all(closed);
|
||||
const final = await readProcessTable();
|
||||
const expectedGone = [first.child.pid, first.ids.daemon, first.ids.background, ...descendants.map(row => row.pid),
|
||||
second.child.pid, second.ids.daemon, second.ids.background];
|
||||
expect(final!.filter(row => expectedGone.includes(row.pid) && !row.state?.startsWith("Z"))).toEqual([]);
|
||||
} finally {
|
||||
try { await stop.stopAll(); await Promise.all(closed); }
|
||||
finally { await rm(directory, { recursive: true, force: true }); }
|
||||
}
|
||||
}, 15000);
|
||||
@@ -0,0 +1,186 @@
|
||||
import type { ChildProcess } from "node:child_process";
|
||||
import { expect, it, vi } from "vitest";
|
||||
import { createProcessTreeOwner, parseRestartRunnerIdentity, restartProcessStartMatches } from "./process-tree-owner.js";
|
||||
import { createRunnerE2EServerStopper } from "./server-stop.js";
|
||||
import { diagnosticProcessKind, type ProcessObservation } from "./process-tree.js";
|
||||
|
||||
const started = "2026-10-02T13:55:27.000Z";
|
||||
const identity = { processPid: 200, processGroupId: 200, processStartedAt: started };
|
||||
const row = (pid: number, parentPid: number, group = pid, kind = "node"): ProcessObservation =>
|
||||
({ pid, parentPid, processGroupId: group, started, kind, state: "S" });
|
||||
function fixture() {
|
||||
let table = [row(process.pid, 1, 10), row(100, process.pid), row(200, 100, 200, "paperclip-runnerd"),
|
||||
row(201, 200), row(300, 100), row(900, process.pid)];
|
||||
const signals: Array<[number, string]> = [];
|
||||
const roots: Array<ChildProcess> = [];
|
||||
const owners: Array<ReturnType<typeof createProcessTreeOwner>> = [];
|
||||
const root = (pid: number) => {
|
||||
const child = { pid, exitCode: null as number | null, signalCode: null as NodeJS.Signals | null,
|
||||
kill: (signal: string) => {
|
||||
signals.push([pid, signal]); child.exitCode = 0;
|
||||
table = table.filter(row => row.pid !== pid).map(row => row.parentPid === pid ? { ...row, parentPid: 1 } : row);
|
||||
return true;
|
||||
} };
|
||||
const handle = child as unknown as ChildProcess;
|
||||
roots.push(handle); return handle;
|
||||
};
|
||||
const stop = createRunnerE2EServerStopper({ gracefulTimeoutMs: 100, forcedTimeoutMs: 100,
|
||||
hasSpawnError: () => false, markExpectedStop: () => {}, log: () => {},
|
||||
createOwner: child => {
|
||||
const owner = createProcessTreeOwner(child, { readTable: async () => table.map(row => ({ ...row })),
|
||||
signalGroup: (group, signal) => { signals.push([group, signal]); table = table.filter(row => row.processGroupId !== group); } });
|
||||
owners.push(owner); return owner;
|
||||
} });
|
||||
return { stop, child: root(100), root, signals, owners, table: () => table,
|
||||
replace: (next: ProcessObservation[]) => { table = next; }, close: () => owners.forEach(owner => owner.stopObserving()) };
|
||||
}
|
||||
|
||||
it("matches Linux process birth receipts at ps precision without accepting another second", () => {
|
||||
expect(restartProcessStartMatches(started, "2026-10-02T13:55:27.731Z", "linux")).toBe(true);
|
||||
for (const expected of ["2026-10-02T13:55:26.999Z", "2026-10-02T13:55:28.000Z", "invalid"]) {
|
||||
expect(restartProcessStartMatches(started, expected, "linux")).toBe(false);
|
||||
}
|
||||
expect(restartProcessStartMatches("invalid", started, "linux")).toBe(false);
|
||||
expect(restartProcessStartMatches(started, "2026-10-02T13:55:27.731Z", "darwin")).toBe(false);
|
||||
expect(restartProcessStartMatches(started, started, "darwin")).toBe(true);
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("preserves only the admitted daemon tree across restart and finally retires old and new owners", async () => {
|
||||
const f = fixture();
|
||||
try {
|
||||
await f.stop.watch(f.child);
|
||||
await f.stop.forRestart(f.child, identity);
|
||||
expect(f.signals).toEqual([[100, "SIGTERM"], [300, "SIGTERM"]]);
|
||||
expect(f.table().map(row => row.pid)).toContain(200);
|
||||
// The daemon forks a new private group after controller loss; it stays owned.
|
||||
f.replace([...f.table(), row(202, 201), row(400, process.pid), row(401, 400)]);
|
||||
const replacement = f.root(400); await f.stop.watch(replacement);
|
||||
await f.stop.stopAll();
|
||||
expect(f.table().map(row => row.pid).sort()).toEqual([process.pid, 900].sort());
|
||||
expect(f.signals.filter(([pid]) => pid === 200)).toEqual([[200, "SIGTERM"]]);
|
||||
expect(f.signals.findIndex(([pid]) => pid === 400)).toBeLessThan(f.signals.findIndex(([pid]) => pid === 200));
|
||||
expect(f.signals.some(([pid]) => pid === 202)).toBe(true);
|
||||
expect(f.signals.some(([pid]) => pid === 900)).toBe(false);
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32").each([
|
||||
["foreign controller", (rows: ProcessObservation[]) => { rows.find(row => row.pid === 200)!.parentPid = 900; }],
|
||||
["reused PID", (rows: ProcessObservation[]) => { rows.find(row => row.pid === 200)!.started = "2026-10-02T13:55:28.000Z"; }],
|
||||
["wrong role", (rows: ProcessObservation[]) => { rows.find(row => row.pid === 200)!.kind = "node"; }],
|
||||
["mixed process group", (rows: ProcessObservation[]) => { rows.push(row(901, 900, 200)); }],
|
||||
["same controller group", (rows: ProcessObservation[]) => { rows.find(row => row.pid === 200)!.processGroupId = 100; }],
|
||||
["missing runner", (rows: ProcessObservation[]) => { rows.splice(rows.findIndex(row => row.pid === 200), 1); }],
|
||||
] as const)("rejects %s without sending a restart signal", async (_name, mutate) => {
|
||||
const f = fixture();
|
||||
try {
|
||||
const rows = f.table(); mutate(rows); f.replace(rows);
|
||||
await expect(f.stop.forRestart(f.child, identity)).rejects.toThrow(/runner|daemon|process group/);
|
||||
expect(f.signals).toEqual([]);
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("fails if the admitted daemon dies instead of pretending recovery preserved it", async () => {
|
||||
const f = fixture();
|
||||
try {
|
||||
const kill = f.child.kill.bind(f.child);
|
||||
f.child.kill = signal => { const result = kill(signal); f.replace(f.table().filter(row => row.pid !== 200)); return result; };
|
||||
await expect(f.stop.forRestart(f.child, identity)).rejects.toThrow("did not survive");
|
||||
await f.stop.stopAll();
|
||||
expect(f.table().map(row => row.pid).sort()).toEqual([process.pid, 900].sort());
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("does not transfer ownership to a reused retained daemon group during final cleanup", async () => {
|
||||
const f = fixture();
|
||||
try {
|
||||
await f.stop.forRestart(f.child, identity);
|
||||
f.replace(f.table().map(row => row.pid === 200 ? { ...row, started: "2026-10-02T14:00:00.000Z" } : row));
|
||||
await expect(f.stop.stopAll()).rejects.toThrow("cleanup incomplete");
|
||||
expect(f.signals.some(([pid]) => pid === 200)).toBe(false);
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
it.each([null, {}, { ...identity, processPid: 0 }, { ...identity, processGroupId: 100 }, { ...identity, processStartedAt: "bad" }])(
|
||||
"rejects malformed restart identity %j", value => { expect(() => parseRestartRunnerIdentity(value)).toThrow(); });
|
||||
|
||||
it.skipIf(process.platform === "win32")("keeps the first final-cleanup deadline after an inspection failure and repeated stop", async () => {
|
||||
const f = fixture();
|
||||
const clock = vi.spyOn(Date, "now");
|
||||
try {
|
||||
await f.stop.forRestart(f.child, identity);
|
||||
clock.mockReturnValue(10_000);
|
||||
const owner = f.owners[0]!;
|
||||
const observe = owner.observe;
|
||||
let fail = true;
|
||||
owner.observe = async () => { if (fail) { fail = false; throw new Error("one inspection failure"); } await observe(); };
|
||||
await expect(f.stop(f.child)).rejects.toThrow("inspection failure");
|
||||
clock.mockReturnValue(20_000);
|
||||
await f.stop(f.child);
|
||||
expect(f.signals.filter(([pid]) => pid === 200)).toEqual([[200, "SIGKILL"]]);
|
||||
} finally { clock.mockRestore(); f.close(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("final cancellation interrupts restart and retires its preserved descendants", async () => {
|
||||
const f = fixture();
|
||||
let cleanup: Promise<void> | undefined;
|
||||
try {
|
||||
const kill = f.child.kill.bind(f.child);
|
||||
f.child.kill = signal => { const result = kill(signal); cleanup = f.stop.stopAll(); return result; };
|
||||
await expect(f.stop.forRestart(f.child, identity)).rejects.toThrow("Final shutdown interrupted");
|
||||
await cleanup;
|
||||
expect(f.table().map(row => row.pid).sort()).toEqual([process.pid, 900].sort());
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32")("final cleanup still retires the old runner when replacement startup failed", async () => {
|
||||
const f = fixture();
|
||||
try {
|
||||
await f.stop.forRestart(f.child, identity);
|
||||
const replacement = f.root(400);
|
||||
Object.assign(replacement, { exitCode: 1 });
|
||||
await f.stop.watch(replacement);
|
||||
await f.stop.stopAll();
|
||||
expect(f.table().map(row => row.pid).sort()).toEqual([process.pid, 900].sort());
|
||||
expect(f.signals.filter(([pid]) => pid === 200)).toEqual([[200, "SIGTERM"]]);
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
|
||||
it("normalizes only the exact Linux comm truncation, never a nearby role name", () => {
|
||||
expect(diagnosticProcessKind("/tmp/paperclip-runne", "linux")).toBe("paperclip-runnerd");
|
||||
expect(diagnosticProcessKind("/tmp/paperclip-runnerd", "darwin")).toBe("paperclip-runnerd");
|
||||
expect(diagnosticProcessKind("paperclip-runne", "darwin")).toBe("other");
|
||||
for (const name of ["paperclip-runn", "paperclip-runner", "paperclip-runneX", "paperclip-runnerd-other"]) {
|
||||
expect(diagnosticProcessKind(name, "linux")).toBe("other");
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
it.skipIf(process.platform === "win32")("still drains the retained daemon after replacement cleanup reports failure", async () => {
|
||||
const f = fixture();
|
||||
try {
|
||||
await f.stop.forRestart(f.child, identity);
|
||||
f.replace([...f.table(), row(400, process.pid)]);
|
||||
const replacement = f.root(400); await f.stop.watch(replacement);
|
||||
const latestOwner = f.owners[1]!;
|
||||
latestOwner.observe = async () => { throw new Error("replacement inspection failed"); };
|
||||
await expect(f.stop.stopAll()).rejects.toThrow("cleanup incomplete");
|
||||
expect(f.signals.filter(([pid]) => pid === 200)).toEqual([[200, "SIGTERM"]]);
|
||||
expect(f.signals.findIndex(([pid]) => pid === 400)).toBeLessThan(f.signals.findIndex(([pid]) => pid === 200));
|
||||
} finally { f.close(); }
|
||||
});
|
||||
|
||||
|
||||
it.skipIf(process.platform === "win32")("does not reuse an already-retired ordinary restart controller's old cleanup deadline", async () => {
|
||||
const f = fixture();
|
||||
const clock = vi.spyOn(Date, "now").mockReturnValue(10_000);
|
||||
try {
|
||||
await f.stop(f.child);
|
||||
clock.mockReturnValue(20_000);
|
||||
f.replace([...f.table(), row(400, process.pid), row(401, 400)]);
|
||||
const replacement = f.root(400); await f.stop.watch(replacement);
|
||||
await f.stop.stopAll();
|
||||
expect(f.signals.filter(([pid]) => pid === 401)).toEqual([[401, "SIGTERM"]]);
|
||||
} finally { clock.mockRestore(); f.close(); }
|
||||
});
|
||||
@@ -46,6 +46,10 @@ function stopper(gracefulTimeoutMs = 2_000) {
|
||||
});
|
||||
return { stop, logs, errors };
|
||||
}
|
||||
function expectSuccessfulWrapperReceipt(wrapper: ChildProcess, messages: unknown[], diagnostics = "") {
|
||||
expect([wrapper.exitCode, wrapper.signalCode], diagnostics).toEqual([0, null]);
|
||||
expect(messages, diagnostics).toContainEqual({ exitCode: 0, signalCode: null });
|
||||
}
|
||||
afterEach(async () => {
|
||||
for (const child of children.splice(0)) {
|
||||
if (child.exitCode !== null || child.signalCode !== null) continue;
|
||||
@@ -153,15 +157,19 @@ it.skipIf(process.platform === "win32")("lets launcher cancellation join wrapper
|
||||
// during the async close even though the helper sends only one signal.
|
||||
const messages: unknown[] = [];
|
||||
wrapper.on("message", message => messages.push(message));
|
||||
const exited = once(wrapper, "exit");
|
||||
// Process exit is not IPC completion. Register both barriers before
|
||||
// cancellation so the final worker receipt is drained before asserting it.
|
||||
const closed = once(wrapper, "close");
|
||||
const disconnected = once(wrapper, "disconnect");
|
||||
await owner.observe();
|
||||
const cancellation = stopOwnedProcessTree(wrapper, owner, 2_000, 2_000);
|
||||
await new Promise(resolve => setTimeout(resolve, 250));
|
||||
if (wrapper.connected) wrapper.send("finish");
|
||||
await exited;
|
||||
await Promise.all([closed, disconnected]);
|
||||
await cancellation;
|
||||
expect(messages, wrapperErrors).toContainEqual({ message: "close-complete", pid: workerPid });
|
||||
expect(messages).toContainEqual({ exitCode: 0, signalCode: null });
|
||||
const diagnostics = `${wrapperErrors}\nwrapper exit=${wrapper.exitCode} signal=${wrapper.signalCode} connected=${wrapper.connected}`;
|
||||
expect(messages, diagnostics).toContainEqual({ message: "close-complete", pid: workerPid });
|
||||
expectSuccessfulWrapperReceipt(wrapper, messages, diagnostics);
|
||||
expect(() => process.kill(workerPid!, 0)).toThrow();
|
||||
} finally {
|
||||
owner.stopObserving();
|
||||
@@ -169,6 +177,21 @@ it.skipIf(process.platform === "win32")("lets launcher cancellation join wrapper
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["failed", "missing"] as const)("rejects a %s worker receipt even when the wrapper closes successfully", async receipt => {
|
||||
const { entry } = await fixture(receipt === "failed"
|
||||
? "process.send({ exitCode: 23, signalCode: null }, () => process.exit(0));"
|
||||
: "process.exit(0);");
|
||||
const wrapper = start(entry);
|
||||
const messages: unknown[] = [];
|
||||
wrapper.on("message", message => messages.push(message));
|
||||
const closed = once(wrapper, "close");
|
||||
const disconnected = once(wrapper, "disconnect");
|
||||
await Promise.all([closed, disconnected]);
|
||||
expect([wrapper.exitCode, wrapper.signalCode]).toEqual([0, null]);
|
||||
expect(messages).toEqual(receipt === "failed" ? [{ exitCode: 23, signalCode: null }] : []);
|
||||
expect(() => expectSuccessfulWrapperReceipt(wrapper, messages)).toThrow();
|
||||
});
|
||||
|
||||
|
||||
it.skipIf(process.platform === "win32")("escalates the owned server group so a hung descendant cannot remain", async () => {
|
||||
const { entry } = await fixture(`
|
||||
@@ -255,8 +278,8 @@ it.skipIf(process.platform === "win32")("recovers a failed cleanup without repea
|
||||
});
|
||||
try {
|
||||
const failed = expect(stop(child)).rejects.toThrow("transient inspection failure");
|
||||
// Process-table inspection can take longer than 100ms on a loaded host.
|
||||
// Keep the child alive until the intended inspection failure actually occurs.
|
||||
// Hold the child until the one-shot failure is actually injected. A fixed
|
||||
// delay can let it exit during the preceding successful inspection.
|
||||
await inspectionFailed;
|
||||
if (child.connected) child.send("finish");
|
||||
await failed;
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { ChildProcess } from "node:child_process";
|
||||
import { createProcessTreeOwner, incompleteTreeFallback } from "./process-tree-owner.js";
|
||||
import { createProcessTreeOwner, incompleteTreeFallback, type RestartRunnerIdentity } from "./process-tree-owner.js";
|
||||
|
||||
// The wrapper receives Playwright's group signal; only it signals Paperclip.
|
||||
export const runnerE2EServerDetached = process.platform !== "win32";
|
||||
@@ -16,6 +16,10 @@ export function createRunnerE2EServerStopper(options: {
|
||||
type State = {
|
||||
owner: ReturnType<typeof createProcessTreeOwner>;
|
||||
promise?: Promise<void>;
|
||||
restartPromise?: Promise<void>;
|
||||
finalRequested?: boolean;
|
||||
finalPhaseStarted?: boolean;
|
||||
finalComplete?: boolean;
|
||||
gracefulDeadline?: number;
|
||||
forcedDeadline?: number;
|
||||
gracefulSent: boolean;
|
||||
@@ -23,16 +27,21 @@ export function createRunnerE2EServerStopper(options: {
|
||||
escalationLogged: boolean;
|
||||
};
|
||||
const states = new WeakMap<ChildProcess, State>();
|
||||
const watched = new Set<ChildProcess>();
|
||||
let finalGraceDeadline: number | undefined;
|
||||
let finalForcedDeadline: number | undefined;
|
||||
let finalAllPromise: Promise<void> | undefined;
|
||||
const exited = (child: ChildProcess) => child.exitCode !== null || child.signalCode !== null || options.hasSpawnError(child);
|
||||
function watch(child: ChildProcess) {
|
||||
let state = states.get(child);
|
||||
if (!state) {
|
||||
state = { owner: (options.createOwner ?? createProcessTreeOwner)(child), gracefulSent: false, groupSignals: new Set(), escalationLogged: false };
|
||||
states.set(child, state);
|
||||
watched.add(child);
|
||||
}
|
||||
return state;
|
||||
}
|
||||
async function stopOnce(child: ChildProcess, state: State, signal: NodeJS.Signals) {
|
||||
async function stopOnce(child: ChildProcess, state: State, signal: NodeJS.Signals, restarting = false) {
|
||||
state.gracefulDeadline ??= Date.now() + options.gracefulTimeoutMs;
|
||||
await state.owner.observe();
|
||||
if (!exited(child) && !state.gracefulSent) {
|
||||
@@ -40,22 +49,28 @@ export function createRunnerE2EServerStopper(options: {
|
||||
}
|
||||
while (Date.now() < state.gracefulDeadline) {
|
||||
await state.owner.observe();
|
||||
if (restarting && state.finalRequested) throw new Error("Final shutdown interrupted controller restart");
|
||||
if (exited(child)) {
|
||||
if (restarting) {
|
||||
state.owner.assertRestartRunnerAlive();
|
||||
if (!state.owner.restartCleanupGroups().size) return;
|
||||
}
|
||||
if (!state.owner.liveGroups().length) { state.owner.stopObserving(); return; }
|
||||
// The leader is gone. Retire only descendants whose start identities
|
||||
// we observed while they belonged to this server, including private groups.
|
||||
const unsignaled = new Set(state.owner.liveGroups().map(group => group.processGroupId).filter(pid => !state.groupSignals.has(pid)));
|
||||
const unsignaled = new Set((restarting ? [...state.owner.restartCleanupGroups()] : state.owner.liveGroups().map(group => group.processGroupId)).filter(pid => !state.groupSignals.has(pid)));
|
||||
for (const pid of await state.owner.signal("SIGTERM", unsignaled)) state.groupSignals.add(pid);
|
||||
}
|
||||
await wait(25);
|
||||
}
|
||||
if (restarting) throw new Error("Controller restart did not retire non-durable children within its graceful deadline");
|
||||
if (!state.escalationLogged) {
|
||||
options.log(`\nPaperclip did not stop within ${options.gracefulTimeoutMs}ms; sending SIGKILL\n`);
|
||||
state.escalationLogged = true;
|
||||
}
|
||||
if (runnerE2EServerDetached) await state.owner.signal("SIGKILL");
|
||||
else if (!exited(child)) child.kill("SIGKILL");
|
||||
state.forcedDeadline ??= Date.now() + options.forcedTimeoutMs;
|
||||
state.forcedDeadline ??= finalForcedDeadline ?? Date.now() + options.forcedTimeoutMs;
|
||||
do {
|
||||
await state.owner.observe();
|
||||
if (exited(child) && !state.owner.liveGroups().length) { state.owner.stopObserving(); return; }
|
||||
@@ -66,7 +81,19 @@ export function createRunnerE2EServerStopper(options: {
|
||||
const stop = (child: ChildProcess, signal: NodeJS.Signals = "SIGTERM"): Promise<void> => {
|
||||
options.markExpectedStop(child);
|
||||
const state = watch(child);
|
||||
state.promise ??= Promise.resolve().then(() => stopOnce(child, state, signal)).catch(async error => {
|
||||
state.finalRequested = true;
|
||||
state.promise ??= Promise.resolve().then(async () => {
|
||||
if (state.restartPromise && !state.finalPhaseStarted) {
|
||||
await state.restartPromise.catch(() => {});
|
||||
state.finalPhaseStarted = true;
|
||||
// A completed restart is a separate phase, not elapsed cleanup time.
|
||||
state.gracefulDeadline = finalGraceDeadline ?? Date.now() + options.gracefulTimeoutMs;
|
||||
state.forcedDeadline = undefined;
|
||||
state.groupSignals.clear();
|
||||
}
|
||||
await stopOnce(child, state, signal);
|
||||
state.finalComplete = true;
|
||||
}).catch(async error => {
|
||||
try {
|
||||
await incompleteTreeFallback(error, child, state, state.gracefulDeadline!, options.forcedTimeoutMs);
|
||||
} finally {
|
||||
@@ -77,5 +104,39 @@ export function createRunnerE2EServerStopper(options: {
|
||||
repeating the graceful signal. */
|
||||
return state.promise;
|
||||
};
|
||||
return Object.assign(stop, { watch: (child: ChildProcess) => watch(child).owner.observe() });
|
||||
return Object.assign(stop, {
|
||||
watch: (child: ChildProcess) => watch(child).owner.observe(),
|
||||
forRestart: (child: ChildProcess, identity: RestartRunnerIdentity): Promise<void> => {
|
||||
options.markExpectedStop(child);
|
||||
const state = watch(child);
|
||||
if (state.finalRequested || state.restartPromise) return Promise.reject(new Error("Restart already requested or final cleanup active"));
|
||||
state.restartPromise = (async () => {
|
||||
await state.owner.preserveRunnerForRestart(identity);
|
||||
options.log(`Controller restart preserves admitted runner pid=${identity.processPid} pgid=${identity.processGroupId} started=${identity.processStartedAt}\n`);
|
||||
await stopOnce(child, state, "SIGTERM", true);
|
||||
})();
|
||||
return state.restartPromise;
|
||||
},
|
||||
stopAll: (signal: NodeJS.Signals = "SIGTERM"): Promise<void> => {
|
||||
// Replacement controller drain precedes old daemon retirement. All
|
||||
// generations share the original final-cleanup window, including retries.
|
||||
finalGraceDeadline ??= Math.min(Date.now() + options.gracefulTimeoutMs,
|
||||
...[...watched].map(child => states.get(child)!).filter(state => state.finalRequested && !state.finalComplete
|
||||
&& (!state.restartPromise || state.finalPhaseStarted)).map(state => state.gracefulDeadline ?? Infinity));
|
||||
finalForcedDeadline ??= finalGraceDeadline + options.forcedTimeoutMs;
|
||||
for (const child of watched) {
|
||||
const state = states.get(child)!;
|
||||
state.finalRequested = true;
|
||||
state.gracefulDeadline ??= finalGraceDeadline;
|
||||
}
|
||||
finalAllPromise ??= Promise.resolve().then(async () => {
|
||||
const failures: unknown[] = [];
|
||||
for (const child of [...watched].reverse()) {
|
||||
try { await stop(child, signal); } catch (error) { failures.push(error); }
|
||||
}
|
||||
if (failures.length) throw new AggregateError(failures, "Owned server cleanup incomplete");
|
||||
}).catch(error => { finalAllPromise = undefined; throw error; });
|
||||
return finalAllPromise;
|
||||
},
|
||||
});
|
||||
}
|
||||
+24
-16
@@ -1,3 +1,6 @@
|
||||
import { parseRestartRunnerIdentity, type RestartRunnerIdentity } from "./process-tree-owner.js";
|
||||
import { usesInstalledCli, verifyInstalledCli } from "./installed-cli.js";
|
||||
import { runnerMatrix } from "./catalog.js";
|
||||
import { createRunnerE2EServerStopper, runnerE2EServerDetached } from "./server-stop.js";
|
||||
import { runnerE2ETypeScriptProcessArgs } from "./web-server-command.js";
|
||||
import { qualifyLegacyClaudeCli } from "./legacy-claude-cli.js";
|
||||
@@ -26,6 +29,12 @@ const configPath = required("PAPERCLIP_CONFIG");
|
||||
const port = required("PAPERCLIP_RUNNER_E2E_PORT");
|
||||
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
||||
const paperclipCli = path.join(repositoryRoot, "tests/runner-e2e/server-entry.ts");
|
||||
const executionIds: string[] = JSON.parse(process.env.PAPERCLIP_RUNNER_E2E_EXECUTION_IDS ?? "[]");
|
||||
const selectedExecutions = usesInstalledCli(process.env) ? executionIds.map(id => {
|
||||
const execution = runnerMatrix.find(row => row.id === id);
|
||||
if (!execution) throw new Error("Unknown installed CLI proof execution");
|
||||
return execution;
|
||||
}) : [];
|
||||
const {
|
||||
controlDirectory,
|
||||
restartRequestPath,
|
||||
@@ -128,9 +137,11 @@ async function startServer() {
|
||||
if (shutdownRequested()) {
|
||||
throw new Error("Refusing to start Paperclip after wrapper shutdown");
|
||||
}
|
||||
// Recheck bytes and dependency resolution on every controller restart.
|
||||
const installed = await verifyInstalledCli(process.env, selectedExecutions);
|
||||
const candidate = spawn(
|
||||
process.execPath,
|
||||
installedRelease?.args ?? runnerE2ETypeScriptProcessArgs(repositoryRoot, paperclipCli, ["onboard", "--yes", "--run"]),
|
||||
installed ? [installed.entry, "onboard", "--yes", "--run"] : runnerE2ETypeScriptProcessArgs(repositoryRoot, paperclipCli, ["onboard", "--yes", "--run"]),
|
||||
{
|
||||
cwd: installedRelease?.cwd ?? repositoryRoot,
|
||||
env: definedServerEnvironment,
|
||||
@@ -237,6 +248,7 @@ async function waitForHealthToStop() {
|
||||
|
||||
interface RestartRequest {
|
||||
requestId: string;
|
||||
preserveRunner?: RestartRunnerIdentity;
|
||||
}
|
||||
|
||||
async function readRestartRequest(): Promise<RestartRequest | null> {
|
||||
@@ -262,7 +274,8 @@ async function readRestartRequest(): Promise<RestartRequest | null> {
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return { requestId };
|
||||
const preserveRunner = (value as { preserveRunner?: unknown }).preserveRunner;
|
||||
return { requestId, ...(preserveRunner === undefined ? {} : { preserveRunner: parseRestartRunnerIdentity(preserveRunner) }) };
|
||||
}
|
||||
|
||||
async function writeRestartAck(
|
||||
@@ -284,12 +297,13 @@ async function writeRestartAck(
|
||||
await rename(temporaryAckPath, restartAckPath);
|
||||
}
|
||||
|
||||
async function restartServer(requestId: string) {
|
||||
async function restartServer({ requestId, preserveRunner }: RestartRequest) {
|
||||
activeRestartRequestId = requestId;
|
||||
appendLog(`\nRestart request ${requestId}: stopping Paperclip\n`);
|
||||
const previous = child;
|
||||
if (!previous) throw new Error("No Paperclip server is available to restart");
|
||||
await stopServer(previous);
|
||||
if (preserveRunner) await stopServer.forRestart(previous, preserveRunner);
|
||||
else await stopServer(previous);
|
||||
if (child === previous) child = null;
|
||||
// Do not mistake an orphaned old server for a healthy replacement. The port
|
||||
// must stop answering before the next launcher is allowed to start.
|
||||
@@ -339,13 +353,12 @@ async function supervise() {
|
||||
const request = await readRestartRequest();
|
||||
if (request && request.requestId !== lastRestartRequestId) {
|
||||
lastRestartRequestId = request.requestId;
|
||||
await restartServer(request.requestId);
|
||||
await restartServer(request);
|
||||
}
|
||||
await delay(200);
|
||||
}
|
||||
|
||||
const running = child;
|
||||
if (running) await stopServer(running, shutdownSignal ?? "SIGTERM");
|
||||
await stopServer.stopAll(shutdownSignal ?? "SIGTERM");
|
||||
}
|
||||
|
||||
let exitCode = 0;
|
||||
@@ -364,15 +377,10 @@ try {
|
||||
);
|
||||
}
|
||||
}
|
||||
const running = child;
|
||||
if (running) {
|
||||
try {
|
||||
await stopServer(running);
|
||||
} catch (stopError) {
|
||||
appendLog(
|
||||
`Failed to stop Paperclip after supervisor failure: ${stopError instanceof Error ? stopError.message : String(stopError)}\n`,
|
||||
);
|
||||
}
|
||||
try {
|
||||
await stopServer.stopAll();
|
||||
} catch (stopError) {
|
||||
appendLog(`Failed to stop Paperclip after supervisor failure: ${stopError instanceof Error ? stopError.message : String(stopError)}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -34,7 +34,10 @@ export type RunnerTaskFlow =
|
||||
| "plan_approval_completion"
|
||||
| "warm_three_turn"
|
||||
| "instruction_persistence"
|
||||
| "pi_native"
|
||||
| "pi_controls"
|
||||
| "native_active_stop"
|
||||
| "copilot_protection"
|
||||
| "native_provider_loss"
|
||||
| "cursor_native";
|
||||
|
||||
|
||||
@@ -1,4 +1,22 @@
|
||||
import { expect, type Page } from "@playwright/test";
|
||||
import { expect, type Locator, type Page } from "@playwright/test";
|
||||
|
||||
/** Rich editors interpret pasted Markdown; fill inserts literal paragraph text. */
|
||||
export async function fillTaskPrompt(editor: Locator, prompt: string): Promise<void> {
|
||||
if (!/^ {0,3}(?:`{3,}|~{3,})/m.test(prompt)
|
||||
|| await editor.evaluate(element => element.tagName === "TEXTAREA")) {
|
||||
await editor.fill(prompt);
|
||||
return;
|
||||
}
|
||||
await editor.fill("");
|
||||
await editor.focus();
|
||||
await editor.evaluate((element, text) => {
|
||||
const clipboardData = new DataTransfer();
|
||||
clipboardData.setData("text/plain", text);
|
||||
const event = new ClipboardEvent("paste", { clipboardData, bubbles: true, cancelable: true });
|
||||
element.dispatchEvent(event);
|
||||
if (!event.defaultPrevented) throw new Error("Task editor did not accept the Markdown paste");
|
||||
}, prompt);
|
||||
}
|
||||
|
||||
export async function createTaskThroughUi(input: {
|
||||
page: Page;
|
||||
@@ -46,9 +64,7 @@ export async function createTaskThroughUi(input: {
|
||||
const titleInput = dialog.getByRole("textbox", { name: "Task title", exact: true });
|
||||
if (await titleInput.isVisible()) await titleInput.fill(input.title);
|
||||
else if (input.requireExplicitTitle) throw new Error("This title-preservation case requires a visible explicit title input");
|
||||
await dialog
|
||||
.getByRole("textbox", { name: "editable markdown", exact: true })
|
||||
.fill(input.prompt);
|
||||
await fillTaskPrompt(dialog.getByRole("textbox", { name: "editable markdown", exact: true }), input.prompt);
|
||||
if (input.workMode !== "standard") {
|
||||
await dialog.getByRole("button", { name: "Add to composer", exact: true }).click();
|
||||
await input.page.getByTestId(input.workMode === "planning" ? "composer-add-plan" : "composer-add-ask").click();
|
||||
|
||||
@@ -21,3 +21,12 @@ export function runnerE2EWebServerCommand(repositoryRoot: string) {
|
||||
export function runnerE2ETypeScriptProcessArgs(repositoryRoot: string, entry: string, args: string[] = []) {
|
||||
return ["--import", path.join(repositoryRoot, "cli/node_modules/tsx/dist/loader.mjs"), entry, ...args];
|
||||
}
|
||||
|
||||
// pnpm's generated playwright bin shim adds NODE_PATH, even when the caller
|
||||
// rejected ambient injection. The installed lane invokes the public JS bin
|
||||
// with the current Node directly, keeping the WebServer environment closed.
|
||||
export function runnerE2EPlaywrightInvocation(repositoryRoot: string, args: string[], installed: boolean) {
|
||||
return installed
|
||||
? { command: process.execPath, args: [path.join(repositoryRoot, "node_modules/@playwright/test/cli.js"), ...args] }
|
||||
: { command: "pnpm", args: ["exec", "playwright", ...args] };
|
||||
}
|
||||
@@ -15,12 +15,12 @@ const everydayOracleImage =
|
||||
"python@sha256:9d2e5553305c7c7b0097999bb17187c69b921ccd6bc9d40e4bb5ebe652c00285";
|
||||
|
||||
describe("public repository paid workflow security", () => {
|
||||
it("keeps the manual EC2 image build credential-free and pins the authorized target", async () => {
|
||||
it("keeps the manual hosted Linux image build credential-free and pins the authorized target", async () => {
|
||||
const workflow = await readFile(path.join(repositoryRoot, ".github/workflows/docker-runner-check.yml"), "utf8");
|
||||
const manual = workflow.slice(workflow.indexOf(" authorize_manual:"));
|
||||
expect(manual.match(/AWS_CI_TRUSTED_USER_IDS/gu)).toHaveLength(2);
|
||||
expect(manual.match(/test "\$REPOSITORY_ID" = 1170821064/gu)).toHaveLength(2);
|
||||
expect(manual).toContain('runs-on: runs-on/fleet=paperclip-public-pr-x64/env=public-ci');
|
||||
expect(manual.slice(manual.indexOf(' manual_image:'))).toContain('runs-on: ubuntu-latest');
|
||||
expect(manual).toContain('repos/$REPOSITORY/git/ref/heads/$TARGET_BRANCH');
|
||||
expect(manual).toContain('ref: ${{ needs.authorize_manual.outputs.target_sha }}');
|
||||
expect(manual).toContain('SOURCE_SHA: ${{ needs.authorize_manual.outputs.target_sha }}');
|
||||
|
||||
Reference in new issue
Block a user